# OPENRNDR and sound (Minim, Beads and TarsosDSP libraries)

**URL:** <https://openrndr.discourse.group/t/openrndr-and-sound-minim-beads-and-tarsosdsp-libraries/35>\
**Category:** Tutorials\
**Created:** [December 3, 2019, 11:28am UTC](https://openrndr.discourse.group/t/openrndr-and-sound-minim-beads-and-tarsosdsp-libraries/35 "2019-12-03T11:28:00Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![kazik](https://yyz2.discourse-cdn.com/free1/user_avatar/openrndr.discourse.group/kazik/32/7_2.png) [@kazik](https://openrndr.discourse.group/u/kazik)\
**Post date:** [December 3, 2019, 11:28am UTC](https://openrndr.discourse.group/t/openrndr-and-sound-minim-beads-and-tarsosdsp-libraries/35/1 "2019-12-03T11:28:00Z")

</div>

It won’t be a question, at least not asked by me. My friend asked me during the OPENRNDR workshop how to use sound with OPENRNDR. and then she immediately googled it. There was no answer over the Internet. So I will write how I do it in my projects with [Minim](http://code.compartmental.net/tools/minim/) library.

Despite what you might think Minim is not bound directly to Processing. It can be integrated with any java derived code. In case of OPENRNDR you need to add this dependency to `build.gradle.kts`:

```auto
dependencies {
    // ...
    compile("net.compartmental.code:minim:2.2.2") {
        exclude(group = "org.apache.maven.plugins", module = "maven-javadoc-plugin")
    }
}

```

And then something like this will do the trick:

```auto
  program {
    val minim = Minim(object : Object() {
      fun sketchPath(fileName: String): String {
        return fileName
      }
      fun createInput(fileName: String): InputStream {
        return FileInputStream(File(fileName))
      }
    })
    val lineIn = minim.lineIn
    val fft = FFT(lineIn.bufferSize(), lineIn.sampleRate())
    //...

```

The `sketchPath` and `createInput` functions should reflect structure of music resources in your project.

---

<div class="post-metadata">

**Author:** ![abe](https://yyz2.discourse-cdn.com/free1/user_avatar/openrndr.discourse.group/abe/32/700_2.png) [@abe](https://openrndr.discourse.group/u/abe)\
**Post date:** [December 3, 2019, 11:44am UTC](https://openrndr.discourse.group/t/openrndr-and-sound-minim-beads-and-tarsosdsp-libraries/35/2 "2019-12-03T11:44:07Z")

</div>

Thanks for the example 🙂

The beads audio library is also not Processing dependent, and they seem to be migrating to gradle:

> **[orsjb/beads](https://github.com/orsjb/beads/tree/migrate_to_gradle)**
>
> The Beads project. Contribute to orsjb/beads development by creating an account on GitHub.

I haven’t tried it recently, but I used it in the past.

---

<div class="post-metadata">

**Author:** ![ricardo](https://yyz2.discourse-cdn.com/free1/user_avatar/openrndr.discourse.group/ricardo/32/8_2.png) [@ricardo](https://openrndr.discourse.group/u/ricardo)\
**Post date:** [December 4, 2019, 1:00pm UTC](https://openrndr.discourse.group/t/openrndr-and-sound-minim-beads-and-tarsosdsp-libraries/35/3 "2019-12-04T13:00:09Z")

</div>

Thanks for sharing @kazik, Beads doesn’t have so many examples out there (not that I’ve found). Minim seems like an easier to use alternative.

For anyone wanting to use beads just add the following to `build.gradle.kts`:

```auto
    repositories {
       maven( url = "https://jitpack.io")
    }

    dependencies {
      compile("com.github.orsjb", "beads", "migrate_to_gradle-SNAPSHOT")
    }

```

If you want to pair it with Open-AL:

```kotlin
    class OpenALIO : AudioIO() {
        private var aqs: AudioSource? = null
        val source: AudioSource
            get() {
                return aqs?: error("not started")
            }
        override fun start(): Boolean {
            println("starting OpenALIO")
            aqs = AudioSystem.createQueueSource(queueSize = 2) {
                this.update()
                val bs = context.bufferSize
                val buffer1 = context.out.getOutBuffer(0)
                val bb = ByteBuffer.allocateDirect(bs * 2)
                bb.order(ByteOrder.nativeOrder())
                for (i in 0 until bs) {
                    bb.putShort((buffer1[i].coerceIn(-1.0f, 1.0f) * 32767).toShort())
                }
                bb.rewind()
                AudioData(AudioFormat.MONO_16, 48000, bb)
            }
            (aqs as AudioQueueSource).play()
            return true
        }
        override fun getAudioInput(p0: IntArray?): UGen {
            TODO("not implemented") //To change body of created functions use File | Settings | File Templates.
        }
    }

```

and use it with:

```kotlin
    val context: AudioContext = AudioContext(
            OpenALIO(),
            48000/60,
            IOAudioFormat(48000.0f, 16, 0, 2)
        )

```

PS: This is courtesy of @edwin… just sharing the solution.

---

<div class="post-metadata">

**Author:** ![edwin](https://avatars.discourse-cdn.com/v4/letter/e/73ab20/32.png) [@edwin](https://openrndr.discourse.group/u/edwin)\
**Post date:** [December 4, 2019, 3:27pm UTC](https://openrndr.discourse.group/t/openrndr-and-sound-minim-beads-and-tarsosdsp-libraries/35/4 "2019-12-04T15:27:54Z")

</div>

I didn’t realize Minim has a synthesis framework too. That was mostly my reason for going with Beads, plus that it was easy to drop-in an audio back-end. I used OpenAL to achieve positional audio with a 4 speaker setup.

---

<div class="post-metadata">

**Author:** ![torb](https://avatars.discourse-cdn.com/v4/letter/t/2bfe46/32.png) [@torb](https://openrndr.discourse.group/u/torb)\
**Post date:** [April 24, 2021, 10:43am UTC](https://openrndr.discourse.group/t/openrndr-and-sound-minim-beads-and-tarsosdsp-libraries/35/5 "2021-04-24T10:43:23Z")

</div>

Just want to mention I had good experience using [TarsosDSP](https://github.com/JorenSix/TarsosDSP) library for Pitch Estimation (tho it does other sound stuff too).

I downloaded the TarsosDSP-2.4.jar from [releases](https://0110.be/releases/TarsosDSP/TarsosDSP-2.4/) and put it in src/lib.

The you need to add it to dependencies in build.gradle.kts

```auto
    dependencies {
      …
      implementation(files("$projectDir/src/lib/TarsosDSP-2.4.jar"))
      …
    }

```

I didn’t get it to work until I also added `java-library` to plugins section:

```auto
    plugins {
        …
        `java-library`
        …
    }

```

Finally here’s a simple example using pitch estimation

> **Imports**
>
> ```kotlin
> import org.openrndr.application
> import be.tarsos.dsp.io.jvm.AudioDispatcherFactory
> import be.tarsos.dsp.pitch.PitchProcessor
> import be.tarsos.dsp.pitch.PitchProcessor.PitchEstimationAlgorithm
> import kotlin.concurrent.thread
> 
> ```

```kotlin
    fun main() = application {
        program {
            var pitch = -1f

            // Based on https://stackoverflow.com/questions/31231813/tarsosdsp-pitch-analysis-for-dummies

            // On separate thread to not block drawing process
            // Use daemon thread to allow the app to quit when main thread finishes
            thread(isDaemon = true) {
                val bufferSize = 4080
                val audioDispatcher = AudioDispatcherFactory.fromDefaultMicrophone(bufferSize, 0)
                val pitchProcessor = PitchProcessor(PitchEstimationAlgorithm.YIN, 44100f, bufferSize) {
                    detectionResult, _ -> pitch = detectionResult.pitch
                }
                audioDispatcher.addAudioProcessor(pitchProcessor)
                audioDispatcher.run()
            }

            extend {
                // -1f means no tone detected
                if (pitch != -1f) {
                    val x = (pitch / 1000.0) * width
                    drawer.rectangle(x, 0.0, 5.0, height.toDouble())
                }
            }
        }
    }

```

I gotta say that you can just easily use Java libraries (of which there are a lot!) yet essentially write nice Kotlin is a really nice feature of OPENRNDR.

It would maybe it’d be nice to have something about how to add JARs in the guide? I used way more time simply trying to get the JAR to work than I did actually using the library.

PS: TarsosDSP is GPL which might affect your project license.

---

<div class="post-metadata">

**Author:** ![edwin](https://avatars.discourse-cdn.com/v4/letter/e/73ab20/32.png) [@edwin](https://openrndr.discourse.group/u/edwin)\
**Post date:** [April 24, 2021, 11:00am UTC](https://openrndr.discourse.group/t/openrndr-and-sound-minim-beads-and-tarsosdsp-libraries/35/6 "2021-04-24T11:00:33Z")

</div>

Cool and thank you for sharing your findings!

The guide tapers off quickly at the point where other documentation exist. I see your point though, I think at least pointing towards other texts to read when dealing with these kind of problems would definitely help.

---

<div class="post-metadata">

**Author:** ![abe](https://yyz2.discourse-cdn.com/free1/user_avatar/openrndr.discourse.group/abe/32/700_2.png) [@abe](https://openrndr.discourse.group/u/abe)\
**Post date:** [April 24, 2021, 1:17pm UTC](https://openrndr.discourse.group/t/openrndr-and-sound-minim-beads-and-tarsosdsp-libraries/35/7 "2021-04-24T13:17:22Z")

</div>

Thanks for sharing! I think it might be nice to convert your reply into an independent post, maybe titled “how to use .jar libraries in OPENRNDR”. Later we could even do a sticky post listing useful posts like that one, a bit like a “user made, easy to update and discuss” guide.