Development Log – 07/27/2020
I did a test on the AirPlay sing along, looks like the delay is at least half a second which is very noticeable. The MV has a delay from Mac and then the human voice has another delay for it being streamed to the TV. I need a way to eliminate these delays, or find out a different setup.
However, today I did made some improvement to the video search list though, I re-did the layout and made a loading spinner.
Also a great finding completely resolved the YouTube data API limitation problem. Invidios https://github.com/iv-org/invidious provides us a YouTube data replica, and the API is free to call without any limitations! I’m making a data source based on this API, this is great!
Development Log – 07/23/2020
A friend suggests that I could just send GET request to Youtube and parse HTML to get the videoIDs, he suggests I could use this instead of paying for Data API.
I think potentially this may work, but firstly, it is not scalable also not easy to maintain, secondly I took a look at the YouTube html, it is nearly impossible to figure out their HTML structure and take the videoIds, who knows how YouTube changes their site structure in the future. I think I will still go with the Data APIs for now even though they costs. If I have a good profit model, potentially I should be able to cover the expense.
Development Log – 07/22/2020
I have the basic client side functionality done, even though the UI is very preliminary. Now you could search in a text field, and select a song add it to the queue. The video player will continue to play until the queue is empty. I haven’t tried to sing along, but ideally with the microphone streamer enabled it should support basic karaoke functions.
Development Log – 07/20/2020
I figured out there’s a limit on Youtube Data API requests, roughly 100 searches per day and that is absolutely not enough for our usage. Now we should consider other mv video sources, there are other third party search API sites available but they both have their own pricing model. So I want to build the UI and the data protocol first to make it working, then could swap the data source later when we find a good one.
I made a YoutubeVideoListProvider which fetches mv list from Youtube, and a XCDYoutbeVideoDetailsProvider fetches stream url and details from another source. I was able to get the responses back with correct stream url.
Planning – The project could be separate to 3 Stages:
Stage 1: Basic features good enough for myself usage. Use YouTubeDataAPI as data layer, and a video player. Make it able to sing karaoke at home for myself.
Stage 2: Scalable and profitable app ready to publish. Swap to 3rd party YouTube API (unlimited), setup Ads or donation.
Stage 3: Remote server to speed up search. Set up my own server and route traffic, cache Youtube results, setup voting system. Minimize expense and search time.
Development Log – 07/17/2020
Today I hit the first tech blocker since I started – To mix audio input from microphone with the MV video. As we want to be able to airplay the singer audio with mv together. It is really hard because the MV is a video stream, I did not find any handy API in AVFoundation or AVAudioEngine that can achieve this.
If this does not work out, we could potentially not have this feature in the first version. People can always airplay the entire screen with the default audio output at the same time.
I also researched the YouTube data API, likely we will be using that to fetch Youtube video list. Looks like the SDK does not support MacOS but we can send HTTP request to the endpoint though.
Development Log – 07/16/2020
Added CocoaPod support for my project (I may switch to SPM after launch), and added SwiftLint to the project.
Today I also experimented the audio streaming feature, basically I setup a AvAudioEngine instance, then connect the input node with the output node. The result is good, I could now stream microphone input to speaker in realtime.
Now I can sing along with the video, but there’s a key feature I want to achieve: To airplay the MV and microphone input to the big screen as well.
Development Log – 07/15/2020
I feel like MacOS is a much powerful platform for streaming Audio in realtime and mixing video stream. Also the software could be easily distributed on Mac (don’t have to go through AppStore). I also considered AppleTV, however I don’t think in-app YouTube playback app could pass the AppStore review. So my decision is to make this app for MacOS.
I start to experiment the MV player. There are a few ways to achieve this:
The official player recommended by YouTube (https://developers.google.com/youtube/v3/guides/ios_youtube_helper). This does not support MacOS!
The Swift-Youtube-Player (https://github.com/gilesvangruisen/Swift-YouTube-Player#breaking-changes). This does not support MacOS!
Manually make a WebView and load YouTube URL. This does not provide best experience, the entire YouTube control is in the WebView, and you have to press play button to start the video.
Then finally the XCDYouTubeKit (https://github.com/0xced/XCDYouTubeKit).
I gave XCDYouTubeKit a try. This is a great repository, so far it worked the best, provided the best user experience meanwhile also supports Airplay. However, this is against YouTube’s Terms of Service (because it doesn’t play the ads I guess?), but the author indicates that the official YouTube player is slow and ugly.
I decided to use XCDYouTubeKit as initial start point.
Development Log – 07/14/2020
Today I did some research and made a preliminary design of the system.
First, I need the source of MV, it is not realistic to have a media library for all the MV videos (at least for now), however, I realized YouTube has a ton of music MV available, and they are actually pretty much the same compared to the ones we had in KTV.
What if I could Airplay the MV on the TV, and use my microphone to sing?
Wouldn’t it be nice to have an online voting system to help user select the best suitable video?
The system could break down to 4 major parts:
- The search system – User type in a name of a song, it should go to YouTube fetch a list of MV videos, and show this list attached with the votes of each of them.
- The MV player – It needs to fetch video stream from YouTube and simply play it.
- The realtime singing system – It needs to stream audio from microphone to the speaker in realtime.
- The voting system – What it does is basically record a vote count made by user, it will ask question like “is the song the one you wanted” when user finished a song and record that to a remote database.
Development Log – 07/13/2020
Anyone missing Karaoke (we call it KTV in China)?
Because of COVID-19, all social entertainment has been closed since March. I feel like I need a in-home Karaoke system.
It needs to do a few very simple things:
· It needs to play the MV.
· I can use microphone on my everyday electronic devices to sing.
· The whole system can run on a widely adopted device (likely an iPhone or Mac or AppleTV)