Skip to content

Implement subtitles schema [WIP]#805

Draft
KarthikRIyer wants to merge 16 commits into
AcademySoftwareFoundation:mainfrom
KarthikRIyer:subtitles
Draft

Implement subtitles schema [WIP]#805
KarthikRIyer wants to merge 16 commits into
AcademySoftwareFoundation:mainfrom
KarthikRIyer:subtitles

Conversation

@KarthikRIyer

Copy link
Copy Markdown
Contributor

Link the Issue(s) this Pull Request is related to.

Closes #62

Summarize your change.

This PR adds basic support for Subtitles. I've based it on what I understood from TTML2 and from my experience with Premiere Pro. I've summarized the changes in the issue (#62). Here's an image summarizing the changes (Note change: SerializableObject instead of SerializableObjectWithMetadata):

image

Some concerns:

  • I think storing a Style object with each TimedText is wasteful, but decided to have this approach because we don't have id system in otio yet. In TTML2 there's a section to specify all styles and then refer then in each TimedText. With the current approach conversion from otio to say TTML2 won't be so direct because we don't have a single set of styles. One other thing I thought of was to have a list of styles inside Subtitle and then have a string id in each TimedText. But then each TimedText object will have no style meaning independently/outside the Subtitle. Thoughts?

  • What are Markers used for? Each marker has a markedRange. When we trim an Item or change the sourceRange shouldn't the markedRange of the markers at the start or end of the Item also change? I couldn't find anything that does something like this. Or is this handled in another way? Why I'm asking is if we trim the Subtitle or change its sourceRange, the ranges of TimedTexts at the start or end might change.

Reference associated tests.

Will be added later.

@reinecke

reinecke commented Oct 1, 2020

Copy link
Copy Markdown
Collaborator

Similar to ImageSequenceReference, I wonder if implementing this in an adapter to try out the ergonomics might be a good idea? Also, it might be useful to see some examples of use cases and the code it takes to achieve them using the schema. In ImageSequenceReference, @apetrynet did a lot of iterations trying it out to make sure it worked like he expected and that really helped us make sure we had a usable schema.

@apetrynet

Copy link
Copy Markdown
Contributor

A usage example could be to write an adapter that can use the Subtitle Schema and write out a sub format that ffmpeg supports. I don't think ffmpeg supports TTML2 so converting to srt or something with an adapter might be a good test.

@KarthikRIyer

Copy link
Copy Markdown
Contributor Author

Yes, I was thinking of an srt adapter. Was waiting to hear others' thoughts on the parameters I've included and if we need to implement any other functionality.

I'll write a simple srt adapter and get back to you.

@KarthikRIyer

Copy link
Copy Markdown
Contributor Author

@apetrynet @reinecke I've implemented an srt adapter.

The tests pass for python3, but there are some extra carriage returns added to the strings with python2. Any idea why this is happening?

Comment on lines +42 to +47
class SRTTest(unittest.TestCase):
def setUp(self):
fd, self.tmp_path = tempfile.mkstemp(suffix=".srt", text=True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @KarthikRIyer!
It might be a long shot, but I recognized an error when comparing multi-lined strings. Try adding maxDiff = None as a class variablle in SRTTest

Suggested change
class SRTTest(unittest.TestCase):
def setUp(self):
fd, self.tmp_path = tempfile.mkstemp(suffix=".srt", text=True)
class SRTTest(unittest.TestCase):
maxDiff = None
def setUp(self):
fd, self.tmp_path = tempfile.mkstemp(suffix=".srt", text=True)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This did't work.

maxDiff is just the max length of Diff for the compare function right?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I've had to use it in other tests to make them pass. The error looked similar.

@codecov-io

codecov-io commented Oct 21, 2020

Copy link
Copy Markdown

Codecov Report

Merging #805 into master will decrease coverage by 1.60%.
The diff coverage is 54.85%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #805      +/-   ##
==========================================
- Coverage   84.02%   82.42%   -1.61%     
==========================================
  Files          74       80       +6     
  Lines        3124     3266     +142     
==========================================
+ Hits         2625     2692      +67     
- Misses        499      574      +75     
Flag Coverage Δ
#py27 ?
#py37 ?
#py38 ?
#unittests 82.42% <54.85%> (-1.76%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/opentimelineio/timedTextStyle.h 0.00% <0.00%> (ø)
src/opentimelineio/subtitles.h 3.44% <3.44%> (ø)
src/opentimelineio/subtitles.cpp 38.46% <38.46%> (ø)
src/opentimelineio/timedText.h 46.15% <46.15%> (ø)
src/opentimelineio/timedTextStyle.cpp 58.33% <58.33%> (ø)
src/opentimelineio/timedText.cpp 66.66% <66.66%> (ø)
src/opentimelineio/typeRegistry.cpp 85.29% <100.00%> (ø)
...entimelineio-bindings/otio_serializableObjects.cpp 95.82% <100.00%> (+0.40%) ⬆️
...ntimelineio/opentimelineio-bindings/otio_utils.cpp 86.90% <0.00%> (-0.46%) ⬇️
... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 22e2468...9a0665b. Read the comment docs.

@KarthikRIyer

Copy link
Copy Markdown
Contributor Author

I changed os.open() to io.open() and it worked. But I have no idea why :/

self.assertMultiLineEqual(baseline_data, test_data)

def test_otio(self):
st = otio.adapters.read_from_file(SRT_EXAMPLE_PATH)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be reading SRT_OTIO_EXAMPLE_PATH instead?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, yes. I must've copied the SRT to OTIO test and forgotten to change it to test OTIO to SRT. Thanks for pointing it out. I'll fix it.

@KarthikRIyer

Copy link
Copy Markdown
Contributor Author
Timeline:
    Track1: [----Subtitles----]

If I have something like this and I do:
Timeline.children_if(descended_from=otio.schema.Subtitles)
This returns [Track1, Subtitles]. Is this the intended behaviour? Shouldn't it return just Subtitles?

@darbyjohnston @meshula

@jminor jminor added the roadmap label Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Subtitles & captions

7 participants