Efficient Labeling in IPython

In my last post I described how to detect speech commands in continuous audio signals. One important ingredient was a good amount of labeled data. Quite early on, it became clear to me that an efficient labeling interface would be extremely helpful. The first iteration of that interface was a simple command-line app, quickly to be replaced by an ipywidgets-based interface.

Now, it offers a graphical user interface directly inside Jupyter notebooks and is easily adaptable to various types of data, such as text, images, or audio. The code is designed to be flexible and simple to use. For example, annotating a set of images is as easy as

>>> from chmp.label import annotate
>>> annotations = annotate(
...     ['./data/img_circle.png', './data/img_square.png'],
...     classes=['square', 'circle'],
...     cls='image',
... )

This call will immediately display a widget showing the current image with a list of buttons to select one of the passed classes, similar to

Screenshot of the labeling
interface.

After clicking on a class button, the current image is annotated and added to the history. Immediately the next image will be displayed. As the user interacts with the widget, their selections are added to the annotations object. After three clicks, it could look like

>>> annotations
[{'index': 0, 'item': '0.png', 'label': 'circle', 'reason': 'order'},
 {'index': 1, 'item': '1.png', 'label': 'circle', 'reason': 'order'},
 {'index': 2, 'item': '2.png', 'label': 'circle', 'reason': 'order'}]

In case of error, it is possible to revisit already labeled images by clicking on the corresponding item in the history. After the new label has been selected, the annotations object will contain both the original label and the re-assigned label. Assume 0.png is re-labeled. In that case, the annotations object will contain two entries for it, similar to

>>> annotations
[{'index': 0, 'item': '0.png', 'label': 'circle', 'reason': 'order'},
 ...,
 {'index': 3, 'item': '0.png', 'label': 'square', 'reason': 'repeat'}]

To only get the latest annotation for an item, use the .get_latest() method of the annotations object, as in

>>> annotations.get_latest()
[{'index': 3, 'item': '0.png', 'label': 'square', 'reason': 'repeat'},
 {'index': 2, 'item': '2.png', 'label': 'circle', 'reason': 'order'},
 {'index': 1, 'item': '1.png', 'label': 'circle', 'reason': 'order'}]

annotate can not only handle images, but also audio files. Simply change the cls parameter in the annotate call to cls='audio'. Now, the widget will show audio controls instead of images. For my experiment with speech commands, it looks similar to

Screenshot of the labeling interface used for
audio.

For more advanced needs, the display mechanism can easily be adapted by passing a function to convert an item to its HTML representation. For example, to show the image file name in red, use

>>> annotations = annotate(
...     ['./data/img_circle.png', './data/img_square.png'],
...     display_value=lambda item: (
...         f'<span style="color: #f00;">image: {item}</span>'
...     ),
...     classes=['square', 'circle'],
... )

In case this short description got you interested, please have a look at the code. If you have any questions or comments, I would love to hear from you on twitter @c_prohm.

Table of contents

    Nov. 2017

  1. Aug. 2017

  2. Mar. 2017

  3. Dec. 2016

  4. Sep. 2016

  5. Apr. 2016

  6. Feb. 2016

  7. Jan. 2016

  8. Apr. 2015

  9. Feb. 2015

  10. Jan. 2015

  11. Dec. 2014

  12. Apr. 2014

  13. Jan. 2014

  14. Dec. 2013

  15. Oct. 2013

  16. Sep. 2013

  17. Aug. 2013

  18. Jun. 2013

  19. May. 2013

  20. Apr. 2013

  21. Mar. 2013

  22. Jan. 2013