Your name keeps your marks in their own file, so several people can mark the same clips and we can measure how much they agree.
This build stores each fix as its own row, but your marks table still uses the old one-blob-per-annotator layout. Until it is migrated the server refuses to serve work, so nobody can save on top of somebody else's record. Clicking below rewrites your existing marks into the new format — nothing is deleted — then reload the page.
Every clip assigned to you — click one to jump to it.
An administrator must upload and designate a public evaluation result.
Loading...
Results are computed offline by eval-forced-alignment
(python -m efa.run --input <published gold set>) and uploaded by an
administrator; this screen only displays them.
Boundary error is how far an aligner put a word's start and end from where a person marked it. Rank by p90, not the median: an aligner that is usually close but sometimes half a second out still cuts training clips mid-word. The dashed row is two people marking the same clip — an aligner inside it is already as good as the humans judging it.
Each test redraws the marked clips 2,000 times and scores both aligners on the same draw, so an easy recording or a mumbling speaker cancels out instead of swamping the difference. Clips are redrawn rather than single boundaries, because boundaries in one clip share a speaker and a recording and are not independent. The p-values are corrected for running several tests at once. Grey rows could be chance; mark more clips to narrow the intervals.
Median error per aligner on each clip. Click a clip to open it.
Loading...
Results are computed offline by eval-forced-alignment
(python -m efa.run --input <published gold set>) and uploaded here;
this screen only displays them.
Boundary error is how far an aligner put a word's start and end from where a person marked it. Rank by p90, not the median: an aligner that is usually close but sometimes half a second out still cuts training clips mid-word. The dashed row is two people marking the same clip — an aligner inside it is already as good as the humans judging it.
Each test redraws the marked clips 2,000 times and scores both aligners on the same draw, so an easy recording or a mumbling speaker cancels out instead of swamping the difference. Clips are redrawn rather than single boundaries, because boundaries in one clip share a speaker and a recording and are not independent. The p-values are corrected for running several tests at once. Grey rows could be chance; mark more clips to narrow the intervals.
Median error per aligner on each clip. Click a clip to open it.
The usual case is accept or nudge one of the two proposals. Move the playhead with the waveform (click, or drag the red playhead), then:
Wrong word? Double-click it to retype, or clear the box to delete it. Use + add word before or + add word after for one the transcript missed — the new word starts selected as "?" so you can type straight over it. Corrected words keep the original and are outlined amber; added ones are outlined green. Both are preserved in the export.
Double-click the start or end label in the info bar (or the buttons) to type an exact time — this may create overlaps. Use untangle words to split overlapping words equally, or reset to baseline to restore the original dataset words and timings.
Drag a blue mark to move it; drag the middle of a word to slide the whole tag — sliding into a neighbouring word pushes it along (and its own neighbour, cascading) rather than overlapping or shrinking it. Drag any mark to make that word current. Click empty space to move the playhead, or drag the red playhead to scrub. Wheel zooms, shift+wheel scrolls, and the bar under the panel pans left and right. fit word snaps to the current word, fit all words shows the whole sentence.
Tick show speech segmentation lines to overlay automatically detected syllable-like boundaries (green). They are only a hint — adjust the sensitivity threshold (higher = fewer lines) and min gap if they do not match the speech.
save keeps a checkpoint of your current work; it is always safe mid-clip. Only done & next clip marks the clip as finished (you can undo with unmark as done) and moves on. The top bar shows how many clips are saved vs done; click the saved count for the full list of your clips and their status. Reloading lands you back on the clip you last saved without marking done.