I’m working on a program that loads a timestamped Transcript txt file + an Audio file that corresponds to the Transcript. It then builds a timeline divided by scenes. Each scene loads an image or video file inside a ‘playlist’ folder.
I am rewriting a whole program I had created with Processing, just to find out that Processing has severe limitations at the end (i didn’t know about OPENRNDR).
So far I have been successful with images, but I’m having problems with video playback.
My video is loaded and played, but it plays at a much faster framerate than it should. I’m trying to isolate the matter to work on it.
So far, I’m trying to implement this in a separate file:
a video file is loaded, but not immediately played
i press space bar to play it
it plays
i can press space again to pause it, and space again to play it again.
So far i can make it into step 3. when i pause it and try to play it again, the program crashes. Here’s my current code:
import org.openrndr.application
import org.openrndr.color.ColorRGBa
import org.openrndr.ffmpeg.VideoPlayerFFMPEG
fun main() = application {
configure {
width = 960
height = 540
title = "Simple Video Player"
}
program {
// Load the video file
val video = VideoPlayerFFMPEG.fromFile("c:\\videos\\godgangs\\playlist\\01.mp4").apply {
// Set the video to loop indefinitely
ended.listen {
restart()
}
}
// Flag to manage playback state
var isPlaying = false
keyboard.keyDown.listen {
if (it.name == "space") {
if (isPlaying) {
// Pause the video if it's playing
video.pause()
} else {
// Restart the video to ensure it can be played again
video.seek(0.0)
video.play()
}
// Toggle the playback state
isPlaying = !isPlaying
}
}
extend {
// Clear the screen
drawer.clear(ColorRGBa.BLACK)
// Draw the video
video.draw(drawer, 0.0, 0.0, width.toDouble(), height.toDouble())
}
}
}
Note: I cannot code, I’m an artist with almost zero coding skills. I’m using Claude and ChatGPT to write code for me. I have coded entire apps doing this, but I’m afraid 1) these LLMs do not fully understand OPENRNDR or 2) OPENRNDR doesn’t have this video playback options.
If anybody could share some light on this, or just tell me where to look, I’ll be grateful. thanks.
The program you shared almost works. I think that for some reason it does not like executing .play() more than once. I came up with this alternative.
import org.openrndr.KEY_SPACEBAR
import org.openrndr.application
import org.openrndr.color.ColorRGBa
import org.openrndr.ffmpeg.PlayMode
import org.openrndr.ffmpeg.VideoPlayerFFMPEG
enum class State { WAITING, STARTED, PAUSED }
fun main() = application {
program {
// Load the video file. I choose PlayMode.VIDEO to ignore sound.
val video = VideoPlayerFFMPEG.fromFile("/path/to/video/01.mp4", PlayMode.VIDEO).apply {
// loop the video
ended.listen {
restart()
}
}
// Flag to manage playback state
var state = State.WAITING
keyboard.keyDown.listen {
if (it.key == KEY_SPACEBAR) {
when(state) {
State.WAITING -> {
video.play()
state = State.STARTED
}
State.STARTED -> {
video.pause()
state = State.PAUSED
}
State.PAUSED -> {
// seek is optional. Without it it just continues where paused.
video.seek(0.0)
video.resume()
state = State.STARTED
}
}
}
}
extend {
// Clear the screen
drawer.clear(ColorRGBa.BLACK)
// Draw the video
video.draw(drawer, 0.0, 0.0, width.toDouble(), height.toDouble())
}
}
}
Hello, Abe. First, let me thank you for answering my forum post.
I’ve been studying Processing and OPENRNDR and I have figured out you’re quite the legend in these contexts so I’m grateful for your assistance.
I have ran your code and it works perfectly. I will use it as a base to write the rest of my program.
Talking more conceptually here:
Why did I find it much easier to manipulate video playback in Processing than I do with OPENRNDR? Is there something fundamentally different between both languages that I am missing?
If you want, I can send you the full program I have wrote in Processing (which I’m trying to recreate with OPENRNDR).
In the end, what I want to do is much better done with shaders, OPENGL etc - I stumbled upon Processing’s limitations and had to go back to the drawing board.
Anyway, I’m grateful for your assistance. Cheers from Brazil
The code I posted could be simplified if we provided access to the state variable of the video player. I asked if that’s something we could do.
I would be curious to hear about specific aspects you find less intuitive. Maybe we can simplify things?
BTW It is possible to use shaders and do quite demanding projects also in both. For example Philipp Artus’ work is often done in Processing: Work — Philipp Artus .
If your code is public I could quickly take a look.
My Processing program was handling three different media + text rendering at a time:
“Main Media” which were image/video files loaded in the same way of the program above
“overlays” which were image/videos loaded on top of main media with a transparency value
“B-Rolls” which were images/videos that could be loaded on top of main media and overlays
and there was the text rendering that extracted the text form the Transcript.txt that I could render with different effects and alignments.
Everything was working fine but I tried everything and couldn’t make it work with the P2D renderer. It always crashed at some point when rendering videos. I had to switch back to the Default Renderer and that didn’t cut it for me, performance-wise.
I think this was an oversight on my part because I only tried to switch to P2D once the program was too advanced, I didn’t use it from the start.
The program works, but it stutters a lot when it comes to multiple videos and images being rendered at the same time. Also, I couldn’t make your video exporter work with it so exporting was frame by frame and that was also taking too long, because I wrote it to export all the frames offscreen, regardless of the visual output’s performance.
It’s on a private repository but I will find time to make it public anytime soon.
Very interesting to hear about the issues! I understand the struggle.
The video player in Processing hasn’t received much maintenance in recent years. I also had it crash (actually on a live performance).
Video is a challenge. There is so much data going through the cpu, so many codecs, memory being allocated and discarded… Do you use the sound of the videos? If not it can help to remove it from the video files, so there’s less data to process. Also making sure the bitrate is not more than what’s needed.
I think it can also be good to have all videos using the same format. I haven’t studied which codecs work better. I remember in openFrameworks users were using HAP, I think to be able to jump very quickly to any frame in the movie. I think it would be good to do some performance tests: for instance decode 5 or 10 videos simultaneously, see how much CPU it’s used, then try again with a different codec.
It does sound like an interesting challenge
I’m sorry about the P5 video exporter not working. I should make the Gradle branch the default at some point.
