We simply revealed a brand new ScyllaDB pattern utility, a video streaming app. The undertaking is to be had on GitHub. This weblog covers the video streaming utility’s options and tech stack and breaks down the knowledge modeling procedure. 

Video Streaming App Options

The app has a minimum design with essentially the most crucial video streaming utility options:

• Record all movies, looked after through introduction date (house web page)
• Record movies that you simply began looking at
• Watch video
• Proceed looking at a video the place you left off
• Show a growth bar beneath each and every video thumbnail

Generation Stack

The use of ScyllaDB for Low-Latency Video Streaming Programs

ScyllaDB is a low-latency and high-performance NoSQL database suitable with Apache Cassandra and DynamoDB. It’s well-suited to take care of the large-scale knowledge garage and retrieval necessities of video streaming packages. ScyllaDB has drivers in the entire standard programming languages, and, as this pattern utility demonstrates, it integrates effectively with trendy internet construction frameworks like NextJS.

Low latency within the context of video streaming products and services is an important for turning in a continuing person enjoy. To put the groundwork for prime functionality, you wish to have to design a knowledge type that matches your wishes. Let’s proceed with an instance knowledge modeling procedure to peer what that appears like.

Video Streaming App Knowledge Modeling

Within the ScyllaDB College Knowledge Modeling direction, we educate that NoSQL knowledge modeling must at all times get started together with your utility and queries first. Then, you’re employed backward and create the schema in line with the queries you wish to have to run to your app. This procedure guarantees that you simply create a knowledge type that matches your queries and meets your necessities.

With that during thoughts, let’s move over the queries that our video streaming app must run on each and every web page load!

Web page: Proceed Observing

In this web page, you’ll record the entire movies that they’ve began to look at. This view contains the video thumbnails and the growth bar beneath the thumbnail.

Question: Get Watch Development

SELECT video_id, growth FROM watch_history WHERE user_id = ? LIMIT 9;

Schema: Watch Historical past Desk

CREATE TABLE watch_history (
   user_id textual content,
   video_id textual content,
   growth int,
   watched_at timestamp,
   PRIMARY KEY (user_id)
);

For this question, it is sensible to outline user_id because the partition key as a result of that’s the clear out we use to question the watch historical past desk. Remember that this schema would possibly want to be up to date later if there’s a question that calls for filtering on different columns past the user_id. For now, regardless that, this schema is right kind for the explained question.

But even so the growth price, the app must also fetch the real metadata of each and every video (as an example, the identify and the thumbnail symbol). For this, the `video` desk needs to be queried.

Question: Get Video Metadata

SELECT * FROM video WHERE identity IN ?;

Realize how we use the “IN” operator and no longer “=” as a result of we want to fetch a listing of movies, no longer only a unmarried video.

Schema: Video Desk

CREATE TABLE video (
   identity textual content,
   content_type textual content,
   identify textual content,
   url textual content,
   thumbnail textual content,
   created_at timestamp,
   period int,
   PRIMARY KEY (identity)
);

For the video desk, let’s outline the identity because the partition key as a result of that’s the one clear out we use within the question.

Web page: Watch Video

If you happen to click on on any of the “Watch” buttons, they’re going to be redirected to a web page with a video participant the place they may be able to get started and pause the video.

Question: Get Video Content material

SELECT * FROM video WHERE identity = ?;

It is a very identical question to the person who runs at the Proceed Observing web page. Thus, the similar schema will paintings simply superb for this question as effectively.

Schema: Video Desk

CREATE TABLE video (
   identity textual content,
   content_type textual content,
   identify textual content,
   url textual content,
   thumbnail textual content,
   created_at timestamp,
   period int,
   PRIMARY KEY (identity)
);

Web page: Maximum Fresh Movies

In the end, let’s ruin down the Maximum Fresh Movies web page, which is the house web page of the applying. We analyze this web page ultimate as a result of it’s the most complicated one from a knowledge modeling standpoint. This web page lists ten of essentially the most just lately uploaded movies which might be to be had within the database, ordered through the video introduction date.

We can must fetch those movies in two steps: first, get the timestamps, then get the real video content material.

Question: Get the Maximum Fresh Ten Movies’ Timestamp

SELECT identity, top10(created_at) AS date FROM recent_videos;

You could realize that we use a customized serve as referred to as top10(). This isn’t a typical serve as in ScyllaDB. It’s a UDF (user-defined serve as) that we created to unravel this information modeling downside. This serve as returns an array of the latest created_at timestamps within the desk. Developing a brand new UDF in ScyllaDB can also be an effective way to unravel your distinctive knowledge modeling demanding situations.

Those timestamp values can then be used to question the real video content material that we wish to display at the web page.

Question: Get Metadata for The ones Movies

SELECT * FROM recent_videos WHERE created_at IN ? LIMIT 10;

Schema: Fresh Movies

CREATE MATERIALIZED VIEW recent_videos_view AS
   SELECT * FROM streaming.video
   WHERE created_at IS NOT NULL
   PRIMARY KEY (created_at, identity);

Within the fresh movies’ materialized view, the created_at column is the main key as a result of we clear out through that column in our first question to get the latest timestamp values. Bear in mind that, in some circumstances, this will reason a sizzling partition.

Moreover, the UI additionally displays a small growth bar beneath each and every video’s thumbnail which signifies the growth you made looking at that video. To fetch this price for each and every video, the app has to question the watch historical past desk.

Question: Get Watch Development for Every Video

SELECT growth FROM watch_history WHERE user_id = ? AND video_id = ?;

Schema: Watch Historical past

CREATE TABLE watch_history (
   user_id textual content,
   video_id textual content,
   growth int,
   watched_at timestamp,
   PRIMARY KEY (user_id, video_id)
);

You could have spotted that the watch historical past desk used to be already utilized in a prior question to fetch knowledge. Now this time, the schema needs to be changed rather to suit this question. Let’s upload video_id as a clustering key. This manner, the question to fetch watch growth will paintings appropriately.

That’s it. Now, let’s see the overall database schema!

Ultimate Database Schema

CREATE KEYSPACE IF NOT EXISTS streaming WITH replication = { 'magnificence': 'NetworkTopologyStrategy', 'replication_factor': '3' };


CREATE TABLE streaming.video (
   identity textual content,
   content_type textual content,
   identify textual content,
   url textual content,
   thumbnail textual content,
   created_at timestamp,
   period int,
   PRIMARY KEY (identity)
);


CREATE TABLE streaming.watch_history (
   user_id textual content,
   video_id textual content,
   growth int,
   watched_at timestamp,
   PRIMARY KEY (user_id, video_id)
);


CREATE TABLE streaming.recent_videos (
   identity textual content,
   content_type textual content,
   identify textual content,
   url textual content,
   thumbnail textual content,
   created_at timestamp,
   period int,
   PRIMARY KEY (created_at)
);

Person-Outlined Serve as for the Maximum Fresh Movies Web page

-- Create a UDF for fresh movies
CREATE OR REPLACE FUNCTION state_f(acc record<timestamp>, val timestamp)
CALLED ON NULL INPUT
RETURNS record<timestamp>
LANGUAGE lua
AS $$
   if val == nil then
       go back acc
   finish
   if acc == nil then
       acc = {}
   finish


   desk.insert(acc, val)
   desk.kind(acc, serve as(a, b) go back a > b finish)
   if #acc > 10 then
       desk.take away(acc, 11)
   finish
   go back acc
$$;




CREATE OR REPLACE FUNCTION reduce_f(acc1 record<timestamp>, acc2 record<timestamp>)
CALLED ON NULL INPUT
RETURNS record<timestamp>
LANGUAGE lua
AS $$
   outcome = {}
   i = 1
   j = 1
  
   whilst #outcome < 10 do
       if acc1[i] > acc2[j] then
           desk.insert(outcome, acc1[i])
           i = i + 1
       else
           desk.insert(outcome, acc2[j])
           j = j + 1
       finish
   finish
   go back outcome
$$;




CREATE OR REPLACE AGGREGATE top10(timestamp)
SFUNC state_f
STYPE record<timestamp>
REDUCEFUNC reduce_f;

This UDF makes use of Lua, however you must additionally use Wasm to create UDFs in ScyllaDB. Developing the serve as be sure to allow UDFs within the scylla.yaml configuration report (location: /and so forth/scylla/scylla.yaml):

Clone the Repo and Get Began!

To get began…

Clone the repository:
git clone https://github.com/scylladb/video-streaming

Set up the dependencies:
npm set up

Adjust the configuration report:

APP_BASE_URL="http://localhost:8000"
SCYLLA_HOSTS="172.17.0.2"
SCYLLA_USER="scylla"
SCYLLA_PASSWD="xxxxx"
SCYLLA_KEYSPACE="streaming"
SCYLLA_DATACENTER="datacenter1"

Migrate the database and insert pattern knowledge:
npm run migrate

Run the server:
npm run dev

Wrapping Up

We are hoping you revel in our video streaming app, and it is helping you construct low-latency and high-performance packages with ScyllaDB. If you wish to stay on studying, take a look at ScyllaDB College, the place we have now loose classes on knowledge modeling, ScyllaDB drivers, and a lot more! When you have questions in regards to the video streaming pattern app or ScyllaDB, move to our discussion board, and let’s talk about!

Extra ScyllaDB pattern packages:

Related sources: