split_apostrophe_to_units

scripts.fix_tokenization.split_apostrophe_to_units(i, terminals, preterminals, preterminal_parents, tag1, tag2, state)[source]

split token with apostrophe to Elaborator and Center. x’xxx -> x’ xxx