This is really cool. The annotation-to-testing-to-annotation-etc. Feedback loop makes a ton of sense, and I'd encourage others who may be confused on this post to look at the Automotus case study https://encord.com/customers/automotus-customer-story/ for the annotation side, but my understanding is the relationship between model outputs and annotation steering is out of scope for that project - do you know of... - Source: Hacker News / 5 months ago

Spacy [0] is a state-of-art / easy-to-use NLP library from the pre-LLM era. This post is the Spacy founder's thoughts on how to integrate LLMs with the kind of problems that "traditional" NLP is used for right now. It's an advertisement for Prodigy [1], their paid tool for using LLMs to assist data labeling. That said, I think I largely agree with the premise, and it's worth reading the entire post. The steps... - Source: Hacker News / 10 months ago

Prodigy AI - Offers software engineers career coaching, skill assessment, and job matching. Visit Prodigy AI. - Source: dev.to / 11 months ago

I essentially want to use a Combo of OCR + NER to attempt to identify this, but I'm not sure NER is well suited for this, as it is not natural language, so there is little context to go off of. I was thinking of perhaps using Prodigy, a data annotation tool, to annotate Candidate Names, Races, etc, and perhaps it will be able to learn off of image data alone wheat these fields tend to look like. Source: about 1 year ago

I come from a similar application area, where I try to tag (annotation/label) a taxonomy of products iteratively. You are trying something slightly different, AFAIU, labeling a flat set of songs, each song with a set of tags from ontology (directed graph)From an application point of view, this is what taxonomists often do, when migrating products from one catalog to another: mapping one taxonomy to another. There... Source: over 1 year ago

I've been doing online tests for about 7 years now. I have most of my stuff on socrative.com. I can give multiple choice, true/false, multiple select, and short answer. I have them turn their desks around during the test so I can see everyone's screen. I run GoGuardian as well so they are locked to the test site. Source: about 2 years ago