a quest for the holy Grail

G. Egghe, Sweetbee

 

 

Analysis of food is mostly based on nutrition databases. Although some of the existing databases contain thousands up to 10 million food items but many drawbacks remain. Food composition tables struggle to cope with the large number of products and the rapid pace of change. This is beautifully illustrated by FoodDB, a database created out of Oxford University (Harrington 2019). It weekly extracts nutritional data and availability of all foods and drinks available on six major UK supermarket websites since November 2017.

Analyses using a single weekly timepoint of 97 368 total products in March 2018 identified 2699 ready meals and pizzas. Longitudinal analyses of 903 pizzas revealed that 10.8% changed their nutritional formulation over 6 months, and 29.9% were either discontinued or new market entries.

Harrington RA, Adhikari V, Rayner M, Scarborough P Nutrient composition databases in the age of big data: foodDB, a comprehensive, real-time database infrastructure. BMJ Open. 2019 Jun 27;9(6):e026652. doi: 10.1136/bmjopen-2018-026652.

One method that is used to keep up with the rapid pace of change is to allow the users to add products to the database. A well known example is My Fitness Pall created by the sport fashion brand Under Armour. It claims to have more than 11 million products in the database but a quick search for “milk” illustrates the problem of this approach. One item is displayed several times with widely different compositions.

In daily practice the app should be easy to use. Apps that require users to scroll through thousands of items are quickly abandoned. Some apps therefore provide search facilities by barcode scanning such as MyFitness Pall (Under Armour) and Lose it.

 

 

 

 

Many commercial apps are based on databases that were (partly) constructed by users. Some organizations have realized that this method introduces a lot of mistakes and have the lists curated by their own nutrition experts.

But even when the food composition is accurately stored it may still not be enough for some user (patient) categories. For diabetic patients carbohydrate content may not be enough. Sugar alcohols for example, used as artificial sweeteners are counted as carbohydrates because, chemically, that is what they are. However, they are very partially absorbed and have only a minimal effect on blood glucose levels. They should therefore not be included in the carbohydrate count for diabetic patients.

So depending on the purpose, food labels and commercial databases may not provide sufficient and useful information.

 

After the correct identification of the food item, the portion size must be quantified. This is easy when a person takes a standard portion ( one bar, one jar of yoghourt, etc) because the volume or weight are known. Nevertheless, the contents of the package does not always reflect the analysis of the so-called standard portion on the package !

Portion size estimation of food on a plate is very inaccurate. This is well studied in diabetic patients for whom correct portion size and hence carbohydrate content of a meal is essential for their treatment. Studies have shown that even experienced adult diabetic patients and health professionals make mean errors between 20 and 40 %.

Brazeau, A.S.; Mircescu, H.; Desjardins, K.; Leroux, C.; Strychar, I.; Ekoé, J.M.; Rabasa-Lhoret, R. Carbohydrate counting accuracy and blood glucose variability in adults with type 1 diabetes. Diabet. Res. Clin. Pract. 2013, 99, 19–23.

Meade, L.T.; Rushton, W.E. Accuracy of carbohydrate counting in adults. Clin. Diabet. 2016, 34, 142–147.

 

This phenomenon is also know in the obesity clinics. Patients on average  underestimate their calorie intake by 400 Cal/day.

 

C. Champagne, G. Bray, A. Kurtz, J. Montiero, E. Tucker, J. Voaufova, and J. Delany. Energy intake and energy expenditure: a controlled study comparing dietitians and nondietitians. J. Am. Diet. Assoc., 2002

D. Schoeller, L. Bandini, and W. Dietz. Inaccuracies in selfreported intake identified by comparison with the doubly labelled water method. Can. J. Physiol. Pharm., 199

 

 

The holy grail is an automatic method for estimating the nutritional contents of a meal from one or more images. This requires the correct identification of the food item and an accurate estimation of its quantity .

A first approach taken was to ask participants/patients to send pictures of their meals to an expert nutritionist to analyse the image. This procedure is hard to organize in real time and is usually used in studies to know what patients are eating without any feedback to the user.

C. K. Martin, H. Han, S. M. Coulon, H. R. Allen, C. M. Champagne, and S. D. Anton. A novel method to remotely measure food intake of free-living individuals in real time: the remote food photography method. British J. of Nutrition, 101(03):446–456, 2009.

Other approaches use crowd sourcing to interpret the image, instead of an expert.

               

Several approaches use computer vision algorithms to recognize food items. Many only work when the food items are well separated and the number of categories is small.

F F. Zhu, M. Bosch, I. Woo, S. Kim, C. J. Boushey, D. S. Ebert, and E. J. Delp. The use of mobile devices in aiding dietary assessment and evaluation. IEEE J. Sel. Top. Signal Process., 4(4):756–766, 2010.

Kong and J. Tan. DietCam: Automatic dietary assessment with mobile camera phones. Pervasive Mob. Comput., 8(1):147–163, 2012.

W. Zhang, Q. Yu, B. Siddiquie, A. Divakaran, and H. Sawhney. ’Snap-n-eat’: Food recognition and nutrition estimation on a smartphone. J. Diabetes Science and Technology, 9(3):525–533, 2015.

Another example is Menu Match, the automatic food recognition in chain restaurants where the menus are known, the portion sizes are standardized and the food composition is given (required by law in the US )

V. Bettadapura, E. Thomaz, A. Parnami, G. D. Abowd, and I. Essa. Leveraging context to support automated food recognition in restaurants. In WACV, pages 580–587, 2015.

Unfortunately, even a perfect picture of a plate cannot perfectly predict what is inside many foods, e.g., a burrito. Consequently, an ideal system needs to ask the user for feedback on inherently ambiguous components of a meal.

A completely automatic system may therefore remain an illusion and the nearest best approach might be a smart ”auto-complete” system.

The Nutrishield app solution will be based on this interpretation of images. With a growing number of data flowing in the system, the estimates will get more accurate by time. This accuracy is essential in further advising the user in his nutritional habits and preferences.