Click Here for Software to Determine a Surveillance Error Grid With Your Data
David C. Klonoff, MD, Courtney Lias, PhD, Robert Vigersky, MD, William Clarke, MD, Joan Lee Parkes, PhD, David B. Sacks, MD, M. Sue Kirkman, MD, Boris Kovatchev, PhD, and the Error Grid Panel
Currently used error grids for assessing clinical accuracy of blood glucose monitors are based on out-of-date medical practices. Error grids have not been widely embraced by regulatory agencies for clearance of monitors, but this type of tool could be useful for surveillance of the performance of cleared products. Diabetes Technology Society together with representatives from the Food and Drug Administration, the American Diabetes Association, the Endocrine Society, and the Association for the Advancement of Medical Instrumentation, and representatives of academia, industry, and government, have developed a new error grid, called the surveillance error grid (SEG) as a tool to assess the degree of clinical risk from inaccurate blood glucose (BG) monitors.
A total of 206 diabetes clinicians were surveyed about the clinical risk of errors of measured BG levels by a monitor. The impact of such errors on 4 patient scenarios was surveyed. Each monitor/reference data pair was scored and color-coded on a graph per its average risk rating. Using modeled data representative of the accuracy of contemporary meters, the relationships between clinical risk and monitor error were calculated for the Clarke error grid (CEG), Parkes error grid (PEG), and SEG.
SEG action boundaries were consistent across scenarios, regardless of whether the patient was type 1 or type 2 or using insulin or not. No significant differences were noted between responses of adult/pediatric or 4 types of clinicians. Although small specific differences in risk boundaries between US and non-US clinicians were noted, the panel felt they did not justify separate grids for these 2 types of clinicians. The data points of the SEG were classified in 15 zones according to their assigned level of risk, which allowed for comparisons with the classic CEG and PEG. Modeled glucose monitor data with realistic self-monitoring of blood glucose errors derived from meter testing experiments plotted on the SEG when compared to the data plotted on the CEG and PEG produced risk estimates that were more granular and reflective of a continuously increasing risk scale.
The SEG is a modern metric for clinical risk assessments of BG monitor errors that assigns a unique risk score to each monitor data point when compared to a reference value. The SEG allows the clinical accuracy of a BG monitor to be portrayed in many ways, including as the percentages of data points falling into custom-defined risk zones. For modeled data the SEG, compared with the CEG and PEG, allows greater precision for quantifying risk, especially when the risks are low. This tool will be useful to allow regulators and manufacturers to monitor and evaluate glucose monitor performance in their surveillance programs.
Foster City, California: May 20, 2014—Diabetes Technology Society (DTS) is pleased to announce the launch of the DTS Surveillance Program for Cleared Blood Glucose Monitors. This program is intended to protect patients from inaccurate BGM products currently on the market. This surveillance program will provide independent assessment of the performance of cleared blood glucose monitors following Food and Drug Administration (FDA) clearance against accepted standards, and generate information that can assist patients, healthcare providers, and payers in making the right product selection.
After a product has been cleared for use by the FDA, there is currently no systematic post-market surveillance program that monitors for ongoing product quality post clearance. Low-quality BGM systems can negatively impact the ability of people with diabetes to reliably monitor their blood glucose levels and can lead to adverse health effects. Efforts by the FDA to improve accuracy standards for pre-market clearance would have little impact if performance was not maintained post-market, so a post-market surveillance program is critically important to ensuring the accuracy of cleared products for people with diabetes.
A Steering Committee and an Advisory Committee will be formed consisting of experts in blood glucose monitoring from academia, medical practice, clinical chemistry, government, industry, and medical organizations. Patient advocacy group(s) will be represented. A protocol to assess BGM System performance will be developed and then product testing will begin. It is expected that the information generated by this program will inform patients, payers, and regulators as to which cleared products actually perform accurately in a real world standardized environment.
FDA supports a BGM Surveillance program to protect public health. The agency has agreed with the concept of a surveillance plan and has stated that it will act on information that it receives on low-quality BGM products, including from surveillance testing. Initial funds to get the post-market surveillance program started have been committed by Abbott, the global healthcare company. The project will begin this month.
“This surveillance program will provide a significant benefit to both patients and manufacturers,” says David Klonoff, M.D., founder of Diabetes Technology Society and Clinical Professor of Medicine at University of California, San Francisco. “Patients will benefit by having access to more accurate meters on the market and manufacturers committed to delivering accurate products will now have an opportunity to back up claims about quality and accuracy with proof from this independent, third-party testing program. I encourage other manufacturers to come forward to join Abbott in support of this program.”
About Diabetes Technology Society
Diabetes Technology Society is a non-profit educational organization dedicated to promote development and use of technology to help people with diabetes. DTS publishesJournal of Diabetes Science and Technology, the world’s largest scientific journal devoted to new technologies in the fight against diabetes, including glucose monitoring, insulin delivery, the artificial pancreas, software, wireless health solutions, social media, and bioengineered solutions for treating obesity. DTS presents the annual Diabetes Technology Meeting, the Hospital Diabetes Meeting, the Certified Diabetes Technology Clinician certification meeting in the US and in Europe, and the Tissue Response to Implanted Active Medical Devices Meeting. DTS has developed a Personalized Medicine Diabetes Program for the US Air Force. DTS is currently spearheading development of a new error grid for defining clinical accuracy of a blood glucose monitor in cooperation with FDA, American Diabetes Association, The Endocrine Society, and Association for Advancement of Medical Instrumentation.
For additional information, contact Brett McGreevy at Diabetes Technology Society.
650-357-7140(mcgreevy@diabetestechnology.org) (www.diabetestechnology.org)
Foster City, California: June 12, 2014 - Diabetes Technology Society (DTS) is pleased to announce that the Steering Committee has been assembled for its Surveillance Program for Cleared Blood Glucose Monitors. The first Steering Committee meeting will take place next month in Washington DC.
This program is intended to identify poorly performing blood glucose monitoring products on the market. This surveillance program will provide an independent assessment of the performance of cleared blood glucose monitors following Food and Drug Administration (FDA) clearance. The program will generate information that can assist patients, healthcare providers, and payers in making informed product selections. The information will also be provided to FDA, which is the government agency that regulates these products.
After a product has been cleared for use by the FDA, there is currently no systematic post-market surveillance program that monitors for ongoing product quality post-clearance. Poorly performing BGM system can interfere with the ability of people with diabetes to reliably monitor their blood glucose levels, and make correct decision based upon the readings. Inaccurate readings can lead to incorrect actions and therefore to adverse outcomes. Ongoing efforts by the FDA to improve accuracy standards for pre-market clearance would be undermined if performance was not maintained post-market. Therefore, a post-market surveillance program is critically important to ensure the accuracy of cleared products for people with diabetes.
A Steering Committee has been formed consisting of world class experts in blood glucose monitoring, diabetes, and laboratory methods from academia, medical practice, government, industry, and medical organizations. Many of them have experience in establishing surveillance programs and testing verification programs for other medical tests. Patient advocacy group(s) will be represented in a future Advisory Board that will be established soon. A protocol to assess BGM System performance will be developed and then product testing will begin.
“This surveillance program will provide a significant benefit to both patients and manufacturers,” says David Klonoff, M.D., founder of Diabetes Technology Society and a Clinical Professor of Medicine at University of California, San Francisco. “Patients will benefit by having access to information that can assist them in selecting more accurate blood glucose meter systems and manufacturers committed to delivering accurate products will have an opportunity to back up claims about quality and accuracy with verification from this independent testing program.”
About Diabetes Technology Society
Diabetes Technology Society is a non-profit educational organization dedicated to promote development and use of technology to help people with diabetes. DTS publishes Journal of Diabetes Science and Technology, the world’s largest scientific journal devoted to new technologies in the fight against diabetes, including glucose monitoring, insulin delivery, the artificial pancreas, software, wireless health solutions, social media, and bioengineered solutions for treating obesity. DTS presents the annual Diabetes Technology Meeting, the Hospital Diabetes Meeting, the Certified Diabetes Technology Clinician certification meeting in the US and in Europe, and the Tissue Response to Implanted Active Medical Devices Meeting. DTS has developed a Personalized Medicine Diabetes Program for the US Air Force. DTS is currently spearheading development of a new error grid for defining clinical accuracy of blood glucose monitors in cooperation with FDA, American Diabetes Association, The Endocrine Society, and Association for Advancement of Medical Instrumentation.
For additional information, contact Brett McGreevy at Diabetes Technology Society.
650-357-7140(mcgreevy@diabetestechnology.org) (www.diabetestechnology.org)
Foster City, California: September 15, 2014– Diabetes Technology Society (DTS) is pleased to announce that it will hold apublic meeting regarding payer liability for funding inaccurate blood glucose monitors.This meeting, which is free to attend, is an extension of the DTS Surveillance Program for Cleared Blood Glucose Monitors, which was launched on May 20, 2014. The event will be held onNovember 5, 2014in Bethesda, Maryland, at the Residence Inn Downtown Bethesda.
After a product has been cleared by the Food and Drug Administration (FDA), there is currently no systematic post-market surveillance program available to monitor ongoing product quality. As a result, some patients diagnosed with diabetes are using blood glucose meters that do not perform to the standard of their FDA clearance. These devices can interfere with the patient’s ability to accurately monitor their blood glucose levels, which can lead to incorrect decisions about treatment and therefore adverse health effects.
The surveillance program will provide an independent assessment of the performance of cleared blood glucose monitors and generate information that can assist patients, payers, and healthcare providers in making informed product selections.The meeting onNovember 5, 2014will cover the outcomes and consequences for payers and patients who use products that have not maintained their performance level since being cleared by the FDA. The meeting will include perspectives from payers, clinicians, attorneys, scientists, the FDA, patients, and other members of the diabetes community.
Admission to the meeting onNovember 5, 2014is free; however, registration will be necessary to attend the event as space is limited. To register, please visit:www.diabetestechnology.org/surveillancepayersmeeting.
Foster City, California: December 18, 2014 - Diabetes Technology Society (DTS) is pleased to announce the Steering Committee of the DTS Surveillance Program for Cleared Blood Glucose Monitors met in Herndon, Virginia last week to work on the program’s testing protocol. In the coming days the latest version of the protocol will be circulated to the committee members for final approval.
We are excited to reach this stage of developing the protocol and look forward to begin testing.
Best regards,
David C. Klonoff, MD
President, Diabetes Technology Society
Foster City, California: March 3, 2015 - Diabetes Technology Society (DTS) is pleased to announce that it has completed its work on the protocol of the DTS Surveillance Program for Cleared Blood Glucose Monitors.
Best regards,
David C. Klonoff, MD
President, Diabetes Technology Society
June 8, 2016 - Burlingame, California - Diabetes Technology Society (DTS) is pleased to announce the commencement of testing for the DTS Surveillance Program for Cleared Blood Glucose Monitor systems (BGMS). This program is intended to provide information to patients, healthcare professionals, payers, and regulators about the performance of BGMS products currently on the market.
A Steering Committee consisting of experts in blood glucose monitoring from academia, medical practice, clinical chemistry, government, industry, and medical organizations worked together to create the protocol. The committee engaged in meetings and teleconferences to determine how to conduct BGMS surveillance studies in a scientifically sound manner that is in compliance with good clinical practice and all relevant regulations. A clinical surveillance protocol was created that contains performance targets and analytical accuracy testing studies with marketed BGMS products conducted by qualified clinical and laboratory research sites.
According to David C. Klonoff, M.D., Medical Director of the Diabetes Research Institute at Mills-Peninsula Medical Center in San Mateo, California and Chair of the Steering Committee, "Patients will benefit from reviewing the results of this study, which uses the most rigorous consensus protocol for screening cleared BGMs that has ever been developed. The study will determine which BGMSs, that are among the most widely used in the US, meet strict performance standards which are similar to those which are currently used for regulatory purposes."
Abbott has contributed funding to the program as part of the company’s commitment to people with diabetes.
About Diabetes Technology Society
Diabetes Technology Society (DTS) is a nonprofit organization committed to promoting development and use of technology in the fight against diabetes. The DTS mission is to spearhead collaborative efforts by experts in academia, clinical practice, industry, and government to accelerate development of practical technology for treating, monitoring, diagnosing, and preventing diabetes mellitus and its complications. DTS is also a leader in medical device cybersecurity.
For additional information visit https://www.diabetestechnology.org, or contact Brett McGreevy, Administrator, Diabetes Technology Society,(650) 692-7100,(mcgreevy@diabetestechnology.org).
DIABETES TECHNOLOGY SOCIETY (DTS) RELEASES RESULTS OF BLOOD GLUCOSE MONITOR SYSTEM SURVEILLANCE TESTING
Burlingame, California - June 23, 2017 - As part of the DTS Blood Glucose Monitor System Surveillance Program (Klonoff DC, et al. J Diabetes Sci Technol 2015: 9(4): 895) the accuracy of 18 blood glucose monitoring systems (BGMSs) marketed in the USA was assessed across a wide range of blood glucose levels in the hands of trained professionals. Products selected for testing represented 90% of IMS Rx data, or 90% of Nov 2014 OIG Report of Medicare Mail-order shares; and most retail private label brands. Products were acquired off-the-shelf, independent of the manufacturer. Products were tested according to a protocol that was developed by an expert panel in blood glucose monitoring surveillance testing from the US government, professional organizations, and academia.
The protocol specified that to be compliant a blood glucose value must be within 15% of a reference plasma value for a blood glucose >100 mg/dl and within 15 mg/dl of a reference plasma value for a blood glucose <100 mg/dl. These protocol criteria were similar to: 1) (for all blood glucose levels) the latest ISO standard (ISO15197-2013, "In vitro diagnostic test systems — Requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus"); and 2) (for a blood glucose >100 mg/dl) the latest FDA guidance (FDA 2016, "Self-Monitoring of Blood Glucose Test Systems for Over-the-Counter Use"). The number of compliant readings needed to pass depended on the number of trials. For a study of 100 trials, at least 91 readings were required to be within 15% or 15 mg/dL of the reference value. The project was supported by a grant from Abbott Diabetes Care.
Three accuracy studies were performed on each of the 18 selected BGM Systems. Approximately 100 specimens were tested for each trial for each BGM System. A total of 1035 subjects participated. A Seal of Approval was awarded to any product that passed all 3 of the 3 studies. 6 of 18 BGMs received a Seal of Approval. These 6 BGMs all achieved an overall compliance rate of 95% or higher. The data is available at:
https://www.diabetestechnology.org/surveillance.shtml
Diabetes Technology Society is a non-profit organization dedicated to promoting the development and use of technology for people with diabetes. DTS presents the Diabetes Technology Meeting, the Digital Diabetes Congress, the International Hospital Diabetes Meeting, and MEDSec: The Internet of Medical Things. DTS spearheaded development of DTSec, the first consensus cybersecurity manufacturer's standard for any medical device. The official journal of DTS is "Journal of Diabetes Science and Technology", the world’s largest scientific journal focused on diabetes technology.
For additional information visit https://www.diabetestechnology.org or contact Angel Payal, Administrator, Diabetes Technology Society: 650-692-7100 <payal@diabetestechnology.org>