Medicine Quality Scientific Literature Surveyor methodology

We conducted systematic reviews of Pubmed, Google Scholar, Embase, World Health Organization, United States Pharmacopeia, Medical Regulatory Agencies (MRA) websites and other resources – using specific terms developed for each type of medicines– to identify and include English, French and Spanish languages scientific reports. 

Any study describing assays to determine quality, discussion of sampling methodology, medical product legislation, and in vivo or in vitro tests for quality was included.

Studies with results from several countries or locations are included under each specific country/location. Studies with results for a whole region without specific country or location data, are excluded from the database.   

Study locations and years are based on information presented in publications or as shared with us by the authors. Geolocations were generated from place names using utilities from LatLong.net.

Selected details about the publication, survey data and their interpretation were initially entered in a Microsoft Access 2007 database for the Antimalarial, Antiretrovirals and Antidiabetics. Those data were subsequently uploaded using an Online Data Manager developed by the IDDO IT team. From June 2017 onwards, new publications data were then entered in the Online Data Manager directly.

Data Categories

Antimalarials

• Artemisinin derivatives – monotherapy
• Artemisinin-based combination therapies (ACTs)
• Non-artemisinins. Categories are mutually exclusive and individual drugs or combinations are listed
• Tetracylines

Antidiabetics

• Alpha-glucosidase inhibitors
• Biguanides
• Combination: biguanides, sulphonylureas
• Combination: biguanides, thiazolidinediones
• DPP-4 inhibitors
• GLP-1 agonists
• Insulin and analogues
• Medical devices-diabetes
• SGLT2 inhibitors
• Sulphonylureas
• Thiazolidinediones

Antiretrovirals

• Nucleoside reverse-transcriptase inhibitors (NRTIs)
• Non nucleoside reverse-transcriptase inhibitors (NNRTIs)
• Integrase strand transfer inhibitors (INSTI)
• Protease inhibitors
• Combination NRTI-INSTI
• Combination NRTI-NNRTI
• Combination NRTI-NRTI
• Combination NRTI-PI
• Combination PI-PI

Antibiotics

• Aminoglycoside antibacterials
• Amphenicols
• Beta-lactam antibacterials, cephalosporins
• Beta-lactam antibacterials, penicillins
• Beta-lactam antibacterials, carbapenems
• Combination antibacterial(s) and anti-inflammatory drugs
• Combination of Cephalosporins and Fluoroquinolones
• Combinations of other Antibacterials
• Combinations of anti-infectives +/- anti-inflammatory medicines
• Combinations of beta-lactam and beta-lactamase inhibitor
• Drugs for treatment of lepra
• Macrolides, Lincosamides and Streptogramins*
• Other antibacterials, Glycopeptide
• Other antibacterials, Imidazole derivatives
• Other antibacterials, nitrofuran derivatives
• Tetracyclines

Anti-tuberculosis

• Single Drug preparation - Anti-tuberculosis
• Fixed Drug Combination - Anti-tuberculosis
• Aminoglycoside Antibacterials
• Beta-Lactam Antibacterials, Penicillins
• Quinolone Antibacterials
• Other Antibacterials
• Macrolides, Lincosamides and Streptogramins

Veterinary medicines

• Aminoglycoside Antibacterials
• Macrolides, Lincosamides and Streptogramins
• Other Antibacterials
• Quinolone Antibacterials
• Agents against protozoal diseases
• Amphenicols
• Anesthetics, General
• Anesthetics, Local
• Anti-adrenal preparations
• Antidepressant
• Antiepileptics
• Antifungal
• Antiglaucoma preparations and miotics
• Antihelmintics
• Antinematoda Agents
• Benzoylphenyl urea derivatives
• Cephalosporin
• Combination antibacterial(s) and anti-inflammatory drugs
• Insulin and analogues
• Combinations of Antibacterials
• Macrocyclic Lactones
• Other Analgesics and Antipyretics
• Sulfonamides
• Sulfonamides and Trimethoprim
• Tetracyclines
• Vaccines, Unspecified
• Viral Vaccines

Cardiovascular medicines and medical devices

• Adrenergic and dopaminergic agents
• Angiotensin II receptor blockers (ARBs)
• Angiotensin-converting enzyme (ACE) inhibitors
• Antiadrenergics agents, centrally acting
• Antiarrhythmics, Class I and III
• Antithrombotic Agent
• Benzothiazepine derivatives, Selective calcium channel blockers
• Beta blocking agents
• Ca channel blockers: Phenylalkylamine derivatives
• Ca channels blockers, Dihydropyridine derivatives
• Cardiac Glycosides
• Cardiovascular Device
• Cardiovascular medicines-unspecified
• Combination of cardiovascular medicines
• Diuretics
• HMG CoA reductase inhibitors
• Hydrazinophthalazine derivatives
• Other Analgesics and Antipyretics
• Potassium-sparing Agents
• Vasodilatator-Organic nitrates

Vaccines

• Bacterial Vaccine
• Cholera Vaccine
• Viral Vaccines

Report type 

• Original research article – research article published in a scientific journal (excluding review, meta-analysis and article describing/assessing an analytical technique)
• Public alert – health warnings and articles issued by National Medicines Regulatory Agencies (MRA) – can also be a manufacturer or a World Health Organization alert
• Analytical techniques – article describing an analytical technique for quality testing or sampling
• Reviews – compilation of articles on medicine quality using published reports
• Other – article related to medicine quality that do not fit into any other category
• Short Communication – posters notes or short communications about antimalarial quality (scientific work)
• Theses – theses for pharmaceutical, medical, chemistry, or equivalent degree
• Report – national and international organisation surveys not published in scientific journals
• Online Database – database of samples collected and analysed, but results not published in scientific literature (e.g. USP database MQDB)
• Drug regulation/definitions/supply – article regarding medicine regulation and/or definitions and/or the supply of medicines

Collection type

Describes sample collection methodology used in the published studies:

• Convenience survey – non-randomised sample collection methods
• Random survey – randomised outlet and/or sample selection. Only studies with details of randomisation methods are included
• Random and Convenience – some samples from random sampling, some from convenience sampling but there are not enough details to breakdown the samples into Random or Convenience sampling types
• Case reports – patients not responding to medicines or toxic side effects recorded as 100% failure rate and thought to be a consequence of a poor quality medical product. Also includes samples analysed for quality not included in a survey and medical product recalls published by sources other than MRAs
• Laboratory assembled collections – samples assembled in a laboratory to answer a chemical, rather than an epidemiological, question (mostly for development of a new quality technique)
• Recall/warning/alert – recall of products by manufacturers via MRA or by MRAs directly, or by WHO rapid alert
• Control quality routine – samples are collected to be analysed in routine post marketing surveillance (PMS) by MRA (or to a laboratory mandated by the MRA)
• Equivalence study – study to assess the quality of different marketed brands of the same API(s) assuming that the result would represent the quality of the brand as a whole and not an estimate of the prevalence of the quality of individual samples as in 'prevalence surveys'.
• Bioavailability studies – in vivo comparative bioavailability testing for adequate body tissue concentration including the rate and extent to which drug reaches the body tissue compartment
• Stability study – quality test performed on medicines subjected to various storage conditions
• LQAS – lot quality assurance sampling
• Seizure – confiscations by the police, customs or medicines regulatory agency agents
• Others – Other types of methodology
• Unknown
• NA – not applicable

Quality classification

Important consensus has been achieved with the adoption of substandard and falsified (SF) definitions by the World Health Organization at the Seventieth World Health Assembly in 2017. However, in the scientific and lay literature confusion over the terms used to describe medicine quality remains.

IDDO/WWARN uses the term 'falsified' as a synonym for ‘counterfeit’, ‘fake’ or ‘spurious’ to refer to a medical product, produced with criminal intent to mislead, without reference to intellectual property concerns. If authors did not examine packaging we class any samples that failed chemical assays but had detectable stated Active Pharmaceutical Ingredient (API), but without detected wrong active ingredients, as 'substandard or falsified' and not as 'falsified' or 'substandard'. Without packaging analysis, samples that had wrong API or no API we assume to be falsified but acknowledge that this in imperfect. There is a small risk of misclassification of substandard samples (due to gross manufacturing errors) as falsified but thankfully, such catastrophic errors appear to be relatively rare.

For substandard medical products there is currently usually insufficient information to distinguish those resulting from errors in factory production from those degraded due to post-production inappropriate storage. Therefore, substandard medicines may be erroneously categorised as degraded and medicines could be both substandard and degraded.

Quoted from each report without additional analysis, ‘failure rate’ is a coarse measure of medicine quality. It considers pharmaceutical quality attributes, such as the amount of API, dissolution, disintegration, packaging and other tests such as microbiological testing.

Falsified (F) – Samples with fake packaging and the correct stated amount of Active Pharmaceutical Ingredient (API); or those containing a wrong API, or incorrect API quantity, or no API; or samples without genuine packaging available to compare with but contain the wrong or no stated API

Good quality – Samples that did not fail chemical assays and/or packaging tests

Substandard (S) – Samples with genuine packaging but incorrect API quantity or type or defects in dissolution/disintegration or not sterile

Substandard or Falsified – Samples, without reference packaging available for comparison, containing incorrect quantities (>zero %) of the correct API or failure to comply with other quality specifications (e.g. dissolution tests, contents of impurity, sterility etc)

Degraded – Samples with genuine packaging, containing the correct or incorrect quantity of API with chemical evidence of degradation

Diverted-Unregistered – Diverted or Unregistered medicines. These medicines could be good quality, substandard or falsified

Medicine Source 

The type of outlet where samples were collected or the sample source:

• Hospital/health centres – includes public and private hospitals/health centres
• Clinic – outlets that sell medicines and in which a private or public health professional(s) is consulted by patients outside of a hospital
• Private pharmacy – private pharmacy only (excludes private shops, markets that are unauthorised outlets)
• Unregistered/unlicensed outlets/facilities – unlicensed/unregistered outlets/facilities (e.g. shops, market, itinerant drug seller), not primarily a pharmacy  - excludes illegal online website and illegal manufacturers
• Wholesalers/importer/distributors – includes entities importing & distributing medicines/raw materials to lower level facilities (e.g. pharmacies); including procurement centres (public or private) and central medical stores/depot and distributor warehouse, excluding NGO and illegal outlets
• Manufacturer – manufacturer of APIs and medicines; also includes manufacturing warehouses; exclude illegal manufacturing sites
• Customs/border depot – medicines found at the customs or in a warehouse on at a sea or river port or airport, for example
• Non-governmental organisation – non-governmental organisation
• Organisation other – include national or international organization such as the UN agencies (e.g. UNICEF, WHO), not being 'NGO' organisation
• Government outlets – others – governments outlets except hospital, clinic, manufacturer, distributer/wholesaler (e.g. public pharmacy)
• Manufacturing plants, illegal – includes only illegal/unauthorised plants/sites of manufacture of API, medicines, and packaging
• Website – include legal and illegal websites selling medicines
• Other outlet – for example, traditional healers
• Combination of outlets – different types of outlets including private, government, manufacturers, illegal facilities (not enough details in the report to breakdown into different categories)
• All facilities providing medicines included – this category is used for recalls of poor quality samples
• Unknown – there is no detail of the source of the medicines

The web-based mapping application was built used Google’s Web toolkit and Google Maps.

SQL query generated input files for mapping application.

There are important limitations on the use of these data and we have added warning statements to alert the users to these issues.