Advanced Ultrasound in Diagnosis and Therapy ›› 2019, Vol. 3 ›› Issue (1): 1-5.doi: 10.37015/AUDT.2019.190801

• Original Research • Previous Articles     Next Articles

Image Features-based Learning Effectively Improves Inter-Observer Agreement for Beginners in Evaluating Thyroid Nodule with Ultrasound

Ying Wang, MDa, Luying Gao, MDa, Yuxin Jiang, MDa, Hui Pan, MDb, Jun Zhao, MAb, Xin Zhou, MAb, Qiong Wu, MMa, Ruyu Liu, MMa, Bo Zhang, MDa,*()   

  1. a Department of Ultrasound, Chinese Academy of Medical Sciences & Peking Union Medical College Hospital, Beijing, China
    b Department of Education, Chinese Academy of Medical Sciences & Peking Union Medical College Hospital, Beijing, China
  • Received:2018-10-10 Online:2019-03-30 Published:2019-03-30
  • Contact: Bo Zhang, MD,


Objective: Thyroid nodules are a common medical problem in China and in other parts of the world. Many guidelines use ultrasound (US) as the first choice for evaluating thyroid nodules. A major limitation of US is operator dependency, resulting in a variety of discrepancies in diagnosing thyroid nodules in the literatures. Risk stratification of thyroid nodules is based on the patterns of US features in the 2015 American Thyroid Association (ATA) management guidelines for adult patients with thyroid nodules. We hypothesize that special training targeting features recognition may increase the inter-observer agreement.
Methods: The study was conducted on 52 participants from Peking Union Medical College Hospital (PUMCH) from March to May 2018.The participants were divided into two groups by their own decision for their convenience. Image featuresbased learning (IF-BL) was used to train the participants to learn special features including shape, margin, echo level, internal structure, calcification, vascularity through 10 standard images based on the 2015 ATA guideline. Group A (27 subjects) received IF-BL during the first month, and Group B (25 subjects) received IF-BL during the second month. All participants evaluated US features and risk stratification in 60 US images of 20 thyroid nodules before and after the training. The test results were graded by a teaching assistant according to the rule of 0.5 points assigned to every feature and 2 points assigned to risk stratification, with a total of 100 points. Inter-observer agreements of US features and risk stratification were assessed and compared before and after the training.
Results: After the first month, Group A had better scores than Group B, the control group of the month (75.4±9.4 vs 68.7±8.4, p = 0.01). At the end of the second month during which both groups were trained, there was no difference of scores between Group A and Group B (74.5±10.4 vs 75.1±7.4, P = 0.78). Scores of all participants were significantly higher than the initial (74.8±9.0 vs 65.8±13.6, P < 0.01). After the training, the kappa values of US features improved from 0.28-0.43 to 0.43-0.75, and those of risk stratification improved from 0.13 to 0.55.
Conclusion: IF-BL can effectively help trainees correctly recognize US features and evaluate the risk stratification of thyroid nodule and can improve the inter-observer agreement.

Key words: Thyroid nodule; Cancer; ATA; Training