DenchiSoft Karlsruhe, Germany
[email protected] https://denchisoft.com/
Version 1.8.5.c - 25.05.2021
VTube Studio – Official Documentation Contact
https://discord.gg/vtubestudio https://twitter.com/VTubeStudio [email protected]
Table of Contents
1 What is VTube Studio? ...... 4 1.1 Introduction ...... 4 1.2 Where to download? ...... 4 1.3 Is there a paid version? Why should I buy it? ...... 5 1.4 Supported Devices (iPhone/Android Tracking) ...... 5 1.5 Supported Devices (Webcam Tracking) ...... 5 2 Android vs. iPhone/iPad vs. Webcam: Which is better? ...... 5 3 Getting Started ...... 7 4 Loading your own Models ...... 9 4.1 Preparing your Live2D model for VTube Studio ...... 9 4.2 Putting your Live2D Model into VTube Studio ...... 10 4.3 Auto-Setup ...... 11 5 Loading your own Backgrounds ...... 12 6 VTube Studio Settings ...... 13 6.1 General Settings (Misc.) ...... 13 6.2 General Settings (Voice-based Lipsync) ...... 14 6.3 Camera Settings (iPhone/Android) ...... 16 6.4 Camera Settings (Webcam on MacOS/Steam) ...... 18 6.5 VTS Model Setting ...... 20 6.5.1 Basic Setup ...... 20 6.5.2 Model Movement ...... 20 6.5.3 Physics Settings ...... 21 6.5.4 VTS Parameter Setup ...... 21 6.5.5 Supported INPUT parameters (face tracking, etc.) ...... 22 6.5.6 Hotkey Settings and Types ...... 23 6.5.7 Hotkey Actions ...... 24 7 Expressions (a.k.a. „Stickers“ or „Emotes“) ...... 25 8 Animations ...... 27 9 Interaction between Animations, Expressions, Face Tracking, Physics, etc...... 28 10 Streaming to Mac/PC ...... 29 10.1 Streaming per WiFi - Preparation on PC/Mac ...... 29 10.2 Streaming per WiFi - Connecting the Smartphone App ...... 30 10.3 Alternative to WiFi: Streaming per USB from iOS to PC/Mac ...... 31 10.4 Recording with OBS (transparent background) ...... 32 10.5 Using NDI for streaming to OBS with invisible UI (transparent background) ...... 33 11 Connection Issues & Troubleshooting ...... 34 11.1 Possible Causes for Connection Issues ...... 34 11.2 Network Lag Troubleshooting (general) ...... 36 11.3 Network Lag Troubleshooting (macOS and iPhone) ...... 36 12 Taking/Sharing Screenshots ...... 37 13 Item System ...... 38 14 VTube Studio Files and File Format ...... 41 15 FAQ ...... 41 15.1 Can my face be accidentally revealed on stream? ...... 41 15.2 How do I buy VTube Studio? Why should I? Can I use it commercially?...... 42 15.3 Can I try VTube Studio before buying it?...... 43 15.4 Can I use my webcam for face tracking instead of my smartphone? ...... 43 15.5 How can I use VTube Studio together with my friends? ...... 43 15.6 Can I use USB instead of WiFi to connect my smartphone? ...... 43 15.7 What is the “Virtual Webcam” feature? ...... 43 15.8 My device gets really hot! What is “Streaming Mode” in the smartphone app? ...... 44 15.9 Are there differences between the macOS and Windows version? ...... 44 15.10 Why does my stream to PC/Mac lag? ...... 44 15.11 Can I use my DSLR camera for webcam face tracking? ...... 44 15.12 Why is the app running at crazy FPS (400+)? ...... 45 15.13 Why is blinking and eye tracking not supported on Android? ...... 45 15.14 I bought the PRO version on Android. Can I transfer it to iOS? Or Steam? ...... 45 15.15 VTube Studio doesn’t work on my smartphone. What can I do? ...... 45 15.16 The face tracking is going crazy on my iPhone/iPad! What’s going on? ...... 45 15.17 Why does my iPhone/iPad show the forward-pointing camera? ...... 46 15.18 Where can I get help/support? ...... 46 15.19 How can I become a VTube Studio Partner? ...... 46 15.20 Are MacOS and Linux supported? What about M1 Macs? ...... 46 15.21 Can I use VTube Studio in my company? How about for TV ads? ...... 46 15.22 I can’t load my Live2D model in VTube Studio. What’s wrong? ...... 46 15.23 Can I use the iPhone/Android tracking at the same time with webcam tracking? ...... 46 15.24 The Webcam tracker doesn’t work! What should I do? ...... 46 15.25 I am on MacOS and the webcam tracker is broken...... 46 15.26 Does VTube Studio store/record any personal data?...... 47 15.27 Who is using VTube Studio? I want to see some examples! ...... 47 16 Privacy Policy and Licensing ...... 47 17 Further Terms ...... 48
1 What is VTube Studio? 1.1 Introduction
VTube Studio is an app available for iPhone, Android, macOS and Windows. The app uses your smartphone or webcam to track your face and animate a Live2D Cubism Model accordingly. The model is shown directly on your phone or on your PC by streaming the face-tracking data over your local network from your phone to your PC/Mac. Example video: https://youtu.be/mN0sJXCubQw
VTUBE STUDIO ONLY SUPPORTS LIVE2D MODELS. VROID/VRM IS NOT SUPPORTED.
Going purely by tracking quality, it’s iOS > Webcam > Android
1.2 Where to download?
To run VTube Studio, you need to download the smartphone app (Android or iOS) and/or the desktop app on Steam (Windows or macOS). If you only want to use your webcam, you don’t need the smartphone app.
• Desktop (Steam) o Windows/macOS: https://store.steampowered.com/app/1325860/VTube_Studio/
• Smartphone o iOS: https://apps.apple.com/us/app/vtube-studio/id1511435444 o Android: https://play.google.com/store/apps/details?id=com.denchi.vtubestudio
1.3 Is there a paid version? Why should I buy it?
Please check “How do I buy VTube Studio?” in the FAQ section of this document.
1.4 Supported Devices (iPhone/Android Tracking)
To run VTube Studio, you need a PC (Windows 7 and up) or Mac (10.14 Mojave and up) and a smartphone/tablet that supports the VTube Studio smartphone app. VTube Studio requires the augmented reality framework ARCore (from Google, on Android) or ARKit (from Apple, on iOS).
Supported devices:
• Android o Any device that supports ARCore should work. o See https://developers.google.com/ar/discover/supported-devices o Some devices may not work despite ARCore being supported. When this happens, the camera preview will show a blue rectangle. You can try manually downgrading to older versions of ARCore and see if that helps, otherwise you’ll have to wait until Google fixes support for your device in future versions. • iOS o Any iPhone or iPad that has FaceID. ▪ iPhone X, XR, XS (regular and Max) ▪ iPhone 11, 12 or newer (regular, Pro and Pro Max), ▪ iPad Pro (3rd, 4th+ generation) o Also supported: devices without FaceID but with the Apple A12 (or newer) Bionic Chip. ▪ iPhone SE 2020 ▪ iPad (8th generation)
1.5 Supported Devices (Webcam Tracking)
Any webcam will do, better picture quality means better tracking. Generally, a resolution of 1280x720 or above is recommended for good eye- and blink-tracking. Framerates between 15-30 FPS are sufficient for good tracking. Some users reported problems when using DSLR cameras.
2 Android vs. iPhone/iPad vs. Webcam: Which is better?
The smartphone face tracking is provided by Google ARCore on Android and Apple ARKit on iOS. ARKit face tracking on iOS is much more advanced when compared to ARCore on Android. This is partially due to the supported iOS devices having specialized hardware for face tracking (TrueDepth Camera). iOS devices with the A12 chip or newer without the TrueDepth camera are also supported and the tracking quality is roughly the same. Some minor differences may be observed when using the devices in environments with very bad lighting.
In short, iOS devices are much better. They provide much smoother and more detailed tracking. Newer iPads or iPhones are generally preferable over older ones, as old ones may get warm/hot during usage (see also FAQ). Furthermore, some important features are not supported on Android. If you can, I’d recommend getting a recent (newest gen) iPhone/iPad.
Webcam tracking is really good as well but has some disadvantages. Mainly, it puts additional strain on your CPU that you wouldn’t have when the tracking is performed by a connected phone. Also, blink- and eye-tracking are not as accurate as they are on iOS.
Feature iOS Android Webcam
Face Direction Tracking
Mouth Tracking Frown not supported Frown not supported Frown not supported
Brow Tracking Individual brows not
supported.
Eye Tracking Tracking quality depends on
camera quality and lighting
Blink/Wink Tracking
Tongue/Cheek Tracking
Face Tracking Delay Very small Small (~0.1s) Small/Medium (~0.2s)
To add to that, the tracking on iOS is less shaky and needs less smoothing. This results in your model reacting a bit faster and movement looking more natural in general. To summarize, if you want to take advantage of high-quality face tracking, the iOS version is the way to go but webcam tracking is also very good and may make things easier for you if your PC is powerful enough to run it.
3 Getting Started
Feel free to play around a bit by yourself inside the app! It comes with a few Live2D models, but of course you can also load your own ones (more on that later).
VTube Studio consists of the smartphone app (iOS/Android) and the PC app (Windows/macOS). They are very similar. Let’s look at the interface of the smartphone app first.
In the first image, you can see the VTube Studio loading screen. If you want to find out the version of the app you’re using, it’ll be shown right there.
The second image shows the main menu of the app. From there, you can access the main app functionality. From top to bottom:
• Show camera preview image (see also third image). o The icon next to it starts the camera calibration without showing the camera preview. You can also calibrate by clicking on the camera preview image when it’s shown. o When using the webcam version, there is no way to show your webcam image in VTube Studio for privacy/security reasons. • Show model selection bar. o The icon next to it opens the screenshot config. Using this, you can take screenshots directly from within the app, save them and share them on social media. Also supports PNG screenshots with transparent background. • Change background (shows background selection list). o The icon next to it opens the item/prop system config to import items into the scene. See the chapter “Item System” for more info. • Lock model position (will make sure you can’t accidentally move the model around when touching the screen, face tracking will still move/animate the model). o The icon next to it locks/unlocks all items in the scene. See the chapter “Item System” for more info. • Show settings, including VTube Studio model and hotkey setup. • Show logs (if you run into any issues/errors, check the log). • Hide main menu icons. Double-clicking anywhere on the screen will show the menu icons again.
Now let’s see what happens when you click the “Show model selection bar” icon.
The first item will open a guide on how to load your own models into the app (this document). In the Steam-version, it’ll open the folder that you have to put the models into in order to import them. The second button will unload the model if a model is currently loaded.
The items after that are the models currently available in the app. VTube Studio comes with a couple example models for you to play around with. After clicking a model, it’ll be loaded into the scene. If the model is not yet configured, you’ll be asked to run the “Auto-Setup” for iOS/Webcam or Android which will set the model up based on default Live2D parameter names and value ranges. This will be covered later.
Once the model is properly configured, the model will imitate your movement and facial expressions. You can move, rotate and scale the model with the usual touch gestures. When opening a model in the PC app, the controls for moving/rotating/scaling will be shown on the screen.
You can close the camera preview at any time. There is no reason to keep it open. You can click the camera preview window to calibrate the face tracking. While doing this, keep your face at a normal distance from the camera with a neutral expression on your face. This is especially important on Android/Webcam, iOS only needs calibration to reset the face direction.
4 Loading your own Models
VTube Studio comes with a few default models. Most of them belong to Live2D Inc., so please be sure to read the chapter “Privacy Policy and Licensing” if you want to use them in your streams.
This chapter covers how to load your own Live2D models into VTube Studio. It will be assumed that you already have your Live2D model finished and have a basic understanding what Live2D is and how it works (Live2D parameters, etc.)
4.1 Preparing your Live2D model for VTube Studio
There isn’t really any specific special preparation you need to do with your Live2D model so it works with VTube Studio. As explained later in the chapter “VTS Parameter Setup”, you can map any face tracking parameter to any Live2D parameter, so you could theoretically choose parameter IDs and ranges however you want.
However, it is recommended to adhere to the default IDs and parameter ranges (min/max). Please also note that only the Live2D parameter IDs matter to VTube Studio, not the Live2D parameter names.
When you’re done with your model, export it via File → Export For Runtime → Export as .moc3 file Make sure to choose the following settings:
• “Export Version” set to “SDK 3.0” or “SDK 3.3” or “SDK 4.0” (recommended). • “Export Physics settings file (physics3.json)”. If you don’t to this, your model will not have physics. Always export your physics file like this and never independently from the model, as this is the only way to make sure the physics file is “registered” with the model (in the .model3.json file) so it’s usable by VTube Studio. If no physics file is registered, VTube Studio will use the first one it finds in the model folder. • Export Types: SDK
This will create a folder with the following files (ignore the files marked red). If your model fails to load later in VTube Studio, make sure all files are there.
A short overview about what these files do:
•
Make sure you have all these files in a folder before you proceed. I usually call this folder
If you want to use Live2D Animations and Live2D Expressions, put them in this folder as well or in a subfolder (see the two folders in the screenshot above). How expressions and animations work will be explained later. You should also add a model icon into that folder (any filename, .png or .jpg, recommended 512x512 pixels).
4.2 Putting your Live2D Model into VTube Studio
Take your model folder and put it in the “Live2DModels”-Folder in VTube Studio. Where this folder is located differs depending on the platform. In the Steam-version, you can use this button to open the folder:
Platform Folder Location iOS Open VTube Studio app data folder on your iPhone/iPad in iTunes (Windows or macOS) or Finder (on macOS Catalina). You’ll find a Live2DModels-Folder.
• Copy Live2DModels -Folder to your desktop • Put your new model folder file into that folder • Copy Live2DModels-Folder back into your iPhone/iPad (overwrite the old folder)
Alternatively, you can also directly access the “VTubeStudio” app data folder on your device using your iPhone/iPad file browser and put data in there directly from your gallery, phone storage or iCloud/Dropbox/Email/etc.
Android
Now start VTube Studio. When the app is started, it automatically looks for new Live2D models and creates a VTube Studio Model (VTS Model,
You should now be able to see your model in the model selection bar. If not, make sure all files are present and there are no errors in the logs.
4.3 Auto-Setup
When loading a new Live2D model, you will be asked to run the “Auto-Setup”. The Auto-Setup can be triggered manually as well after the model is loaded (on the model tab in the settings).
When running the Auto-Setup for iOS/Webcam or Android, VTube Studio will automatically look for default Live2D parameters in your model. VTube Studio will assume that you have used default Live2D parameter IDs and parameter names and set up your VTube Studio accordingly. You can learn more about the default IDs here: https://docs.live2d.com/cubism-editor-manual/standard-parametor-list/#
If you do not use the default IDs, you will have to set your model up manually. This is explained in detail in the chapter “VTS Parameter Setup”.
The model will be optimized for use with either Android or iOS/Webcam depending on which option you choose.
Your model should now be ready to use. Of course, you may need to finetune it, but it should move already. On iOS, you might have to open the camera preview once after installing the app before the model will move.
The whole process is also explained in this video:
[VTube Studio] Loading your own Live2D Models -- https://www.youtube.com/watch?v=AssUd6EtFXw
5 Loading your own Backgrounds
VTube Studio comes with a few default backgrounds. Some of them are animated backgrounds that can’t be added or removed (for example the doggo one).
The Steam app also comes with a color-picker-background, pictured below. Using this background, you can also activate “Transparent Mode”, which will enable OBS and other capture software to record your model with a transparent background, so you don’t need to use a color key.
You can freely add backgrounds (videos or images) to VTube Studio. Be careful when using videos with a large file size in the mobile app as they could potentially crash the app due to memory limitations.
Supported formats are:
• Image o .jpg o .png • Video o .mp4 o .webm (may not be supported on some platforms)
The backgrounds-folder can be found in the following locations. You may need to restart the app after adding your own backgrounds.
Platform Folder Location iOS Open VTube Studio app data folder on your iPhone/iPad in iTunes (Windows or macOS) or Finder (on macOS Catalina). You’ll find a Backgrounds-Folder.
• Copy Backgrounds-Folder to your desktop • Put your new background file into that folder • Copy Backgrounds-Folder back into your iPhone/iPad (overwrite the old folder)
Android
After clicking the settings icon in the main menu, the settings screen will be shown. It has several tabs, a help icon (links to this document) and a language selector button that lets you change the interface language (app restart needed).
6.1 General Settings (Misc.)
From here, you can purchase VTube Studio PRO (iOS/Android only). On iOS, there is also a button to restore purchases on this screen.
Furthermore, there are some general settings for hiding the VTS logo, remembering the scene (save open background and model when restarting), etc.
Next, there’s the “Stream to PC” setup. This is for connecting your phone to your PC to stream the face tracking data to the PC app. This will be covered in the chapter “Connecting your Smartphone to your PC/Mac”. On iOS devices, you can also start an USB connection as alternative to WiFi here. This is recommended as it seems to be more reliable/stable than WiFi. On Android, this option is unfortunately not available.
Finally, there’s the “Mouse Input Config”. This allows you to use the mouse/finger position as input parameter to animate your model in addition to face tracking. You can select an X-Range and Y-Range in pixels. This range will be mapped to [-1, 1] for X and Y. With this setting, you could for example make your model follow your finger or mouse pointer with its eyes. This will also show mouse-clicks (right, left, middle) that can be used to trigger hotkeys. In the smartphone app, these mouse-clicks can be triggered by touching the screen with one finger (left-click), two fingers (right-click) and three fingers (middle-click).
6.2 General Settings (Voice-based Lipsync)
The voice-based lipsync options are only available on Windows and MacOS (except M1/Silicon Macs), you cannot use them on smartphones.
Voice-based lipsync enables you to do two things:
• Open/Close mouth (or any Live2D parameter) based on current microphone volume. • Change mouth shape (or any Live2D parameter) based on detected frequencies in voice.
If microphone audio lags behind the values shown on the UI, use the “Reload” button to reload the selected microphone. Alternatively, you can use a hotkey to reload the microphone.
When enabled (“Use microphone” switch), you will be able to use three additional parameters in your VTube Studio model:
• VoiceVolume: Parameter between 0 and 1 depending on microphone sound volume. o Volume Gain Slider controls how strongly the volume affects this parameter. o Volume Cutoff Slider controls the minimum volume to still be picked up by the microphone. You can filter out background noise with this. • VoiceVolumePlusMouthOpen: Combines VoiceVolume parameter with value from tracking. With this, you could for example make your mouth open when there’s sound and when the face tracking sees the mouth as open. This parameter is also between 0 and 1. • VoiceFrequency: This parameter is between 0 and 1 and sits at 0.5 per default. It goes up and down depending on the frequencies found in your voice. You can use this to control your mouth form. o Frequency Gain Slider controls how strongly the detected frequencies affect this parameter.
How the VoiceFrequency Parameter is calculated:
When you speak, VTube Studio analyzes your speech and extracts the phonemes (A, I, U, E, O, Other) from your speech. You will see the squares at the bottom (see image above) turn dark when a specific phoneme is detected.
You can click each phoneme to turn it red (-), green (+) or neutral. Depending on that, when this specific phoneme is detected, it will push the VoiceFrequency parameter up or down (and with that also your mouth shape, if you use this parameter).
There are no recommendations on how you should set the phonemes up. Just try around a bit and see what works for you. It may take some trial and error, but if set up correctly, this
can look really good especially when singing. ♪ ♫ ♬
6.3 Camera Settings (iPhone/Android)
This config tab will let you change settings related to rendering and the camera:
• AR Camera Config o Change face tracking camera FPS. This does not affect the app framerate. o Change whether the camera preview is shown. If you want to make sure your face is never shown on your smartphone screen, you can configure that here (see third picture). o Change whether face mask is shown over your face.
• Tracking Config o Change behavior of model when face tracking is lost. You can either have the model freeze in place when your face is lost or have it return to its default pose. o Change eye-blink linking behavior ▪ If “Always” is rotated, the EyeOpen values of your left and right eye will be always set to the average of both. ▪ If “When face is rotated” is selected, the EyeOpen value of your eyes will be linked when your head is rotated too much to the side. This will use the value of the eye that is still visible when your head is rotated to one side. o Various sensitivities for the iOS/Android face tracker. If (for example) your mouth doesn’t open enough, a quick fix might be to increase the sensitivity here.
• iOS Blink Fix o Due to a recent bug in iOS face tracking, your HeadY parameter will twitch upward very slightly when blinking. This can be counterbalanced using those sliders until it is fixed officially by Apple (hopefully soon). A value for Y of around 30-45 is recommended, but check what works well for you. X can be left at 0 usually. Depending on your Live2D model, you may not need this setting at all because the effect is very subtle.
• Quality and Effects o Turn on Anti-Aliasing (smoother model edges but costs performance, not recommended on mobile) o Change app FPS. VSync means the FPS will be synced to your display update rate. This is recommended on PC to prevent screen-tearing. o Tracking V2: Adds various experimental iOS tracking improvements. Use with caution! ▪ Better winking: When one eye is fully closed and the other one isn’t, this will force the partially open eye a bit more open. This makes winking easier/better. ▪ Better frowning: Automatically frown more by lowering your brows. This is done by detecting when the brows are down and subtracting a certain value of MouthSmile in that case.
• Record Screen o Smartphone-only. Lets you directly record your screen including microphone sound. A small red dot will be shown at the bottom-right while recording. Tap it to stop the recording at any time.
6.4 Camera Settings (Webcam on MacOS/Steam)
Starting with the Steam release of VTube Studio, you can use your webcam for face tracking. Please note that just like with iPhone/Android version, your webcam picture will never be shown within VTube Studio. There is no way to reveal the webcam video in VTube Studio and VTube Studio has no code that would even enable it to do so.
Most settings here should be self-explanatory. Generally, a resolution of 1280x720 and FPS of at least 10 are recommended for decent tracking. This is especially true for eye- and blink-tracking.
When checking “Autostart cam with VTS”, the camera will be started a few seconds after the scene has finished loading when starting VTube Studio.
For details on the different tracking quality levels, please refer to the information shown when selecting them in VTube Studio. Important: Only Level 5 allows winking
Right after turning the camera on, the camera calibrates. You can also manually calibrate afterwards by using the button or a hotkey. This resets the face position to looking forward. Calibration is important, so make sure to look straight at the camera with a neutral face when running it.
The webcam tracking settings can be used to customize the behavior of how face tracking values are calculated. Again, most are self-explanatory. Play around with different values until you find something that works for you. “Link Eye Blinking” and “Webcam Parameter Calc. Mode” will be explained in a bit more detail.
Link Eye Blinking:
Using this, you can link your eyes in case you have problems with weird blink behavior. It’s recommended to set this to “When face is rotated”. What this does is, when the face is rotated left/right to a point where one eye becomes invisible to the tracker, the other visible eye will be used to control blinking.
Webcam Parameter Calc. Mode:
“Automatic” is the old way. If this works for you, feel free to keep using it. This requires an initial ~10 second warmup phase (see instructions in VTube Studio). The new “Manual” (now default) mode does not require this and may be more accurate for most people in most situations.
For the “Manual” mode, it’s important to set the sensitivity sliders to something that works for you and always calibrate the webcam once while looking straight at it with a neutral face. These sliders only have an effect on the webcam tracking. If you want to set sensitivities for smartphone tracking, use the same sliders in the Android/iOS version.
6.5 VTS Model Setting 6.5.1 Basic Setup
Once you have loaded a model, you will be able to access the model settings in the VTube Studio model config tab.
At the top, you can enter a name for your model. This name will be shown in the model selection bar. Below that, you can see the name of the Live2D model file and the automatically created VTube Studio model file (VTS model).
You can choose an icon (.jpg or .png, click icon at top right to open file selection) and default idle animation (.motion3.json) for your model. You can choose from any file inside your model folder.
You can also run the auto-setup for your VTube Studio model. This will set up your model based on standard Live2D parameter names and values. For more information about these parameters, please see: https://docs.live2d.com/cubism-editor-manual/standard-parametor-list/#
If you change the parameter IDs or ranges, auto-setup will not be able to set up your model and you’ll have to do it manually. Even if you use auto-setup, you may have to fine-tune your VTS model to fit your Live2D model.
6.5.2 Model Movement
With the “Model Movement” settings, you can move the model left/right and closer/further away from the screen based on your head position. With the sliders, you can configure how much the model is moved.
6.5.3 Physics Settings
Here, you can boost or reduce the effect of your Live2D physics setup. You can also activate “wind”, which will randomly apply a wind-like force to the physics system. This is experimental and may not look good depending on how your model is set up.
6.5.4 VTS Parameter Setup
This is the most important part of your model settings. Here, you set up which face tracking parameters control which Live2D parameters.
The general idea is: • You can freely map any INPUT parameter (face-tracking, mouse, etc.) to any OUTPUT parameter (Live2D parameter). • How the values are mapped can be freely configured. For example, the input parameter MouthOpen is within the range [0, 1], with 0 being closed and 1 being all the way open. If your Live2D parameter (for example named ParamMouthOpen) has the range [-10, 10], you could map the ranges so (IN, 0) becomes (OUT, -10) and (IN, 1) becomes (OUT, 10). Or you map it any other way, there are no restrictions. • The red dot will show where the current value is placed within the input and output range. • By activating “Limit Range”, you can make sure the the input or output values never exceed the range you set for the input or output. This will also cause the value to stop smoothly when approaching the limits. • Generally, it is recommended to only change the output range and leave the input range as is. • More smoothing will make the movements less shaky but will introduce lag. Experiment with the smoothing values until you find something that works for each parameter. On iOS, you will most likely need very little smoothing. • Auto-breath will fade the output parameter up and down in a breath-like motion. No input parameter is required when using this option. Any input will be ignored. • Auto-blink will randomly reduce the parameter to zero. This can be combined with an input parameter, but none is necessary. • As input parameter, you can choose from a list of available parameters. iOS supports more face tracking parameters than Android (see below). • As output parameter, you can choose any Live2D parameter. Each output parameter can only be chosen once, as otherwise you would have multiple input parameters writing to the same output parameter.
If your model doesn’t move despite the Live2D parameter clearly moving in the VTS model config, the cause is most likely an expression, an animation or the physics system overwriting the value from the face tracking. This is explained in detail in the chapter:
Interaction between Animations, Expressions, Face Tracking, Physics, etc.
6.5.5 Supported INPUT parameters (face tracking, etc.)
VTube Studio currently supports the following input parameters that can be mapped to output Live2D parameters. The iOS-only parameters are also marked accordingly in the app.
Parameter Name Explanation iOS Android Webcam FacePositionX horizontal position of face ✔️ ✔️ ✔️ FacePositionY vertical position of face ✔️ ✔️ ✔️ FacePositionZ distance from camera ✔️ ✔️ ✔️ FaceAngleX face right/left rotation ✔️ ✔️ ✔️ FaceAngleY face up/down rotation ✔️ ✔️ ✔️ FaceAngleZ face lean rotation ✔️ ✔️ ✔️ MouthSmile how much you’re smiling ✔️ ✔️ ✔️ MouthOpen how open your mouth is ✔️ ✔️ ✔️ Brows up/down for both brows combined ✔️ ✔️ ✔️ x-pos. of mouse or finger within set MousePositionX range ✔️ ✔️ ✔️ y-pos. of mouse or finger within set MousePositionY range ✔️ ✔️ ✔️ TongueOut stick out your tongue ✔️ ❌ ❌ EyeOpenLeft how open your left eye is ✔️ ❌ ✔️ EyeOpenRight how open your right eye is ✔️ ❌ ✔️ EyeLeftX eye-tracking ✔️ ❌ ✔️ EyeLeftY eye-tracking ✔️ ❌ ✔️ EyeRightX eye-tracking ✔️ ❌ ✔️ EyeRightY eye-tracking ✔️ ❌ ✔️ CheekPuff detects when you puff out your cheeks ✔️ ❌ ❌ BrowLeftY up/down for left brow ✔️ ❌ ✔️ BrowRightY up/down for right brow ✔️ ❌ ✔️ VoiceFrequency* depends on detected phonemes ❌ ❌ ✔️ VoiceVolume* how loud microphone volume is ❌ ❌ ✔️ VoiceVolume MouthOpen + VoiceVolume PlusMouthOpen* ❌ ❌ ✔️ VoiceFrequency MouthSmile + VoiceFrequency PlusMouthSmile* ❌ ❌ ✔️ MouthX Mouth X position (shift mouth left/right) ✔️ ❌ ❌ detects angry face (EXPERIMENTAL, not FaceAngry recommended) ✔️ ❌ ❌
* these parameters cannot be used directly on iOS/Android but will work on PC/Mac when using a smartphone for tracking. Parameters like VoiceVolumePlusMouthOpen will default to using the value from MouthOpen when the model is loaded on the smartphone.
6.5.6 Hotkey Settings and Types
In VTube Studio, you can use hotkeys to trigger various actions. When creating a new hotkey, it is saved in the VTube Studio model configuration file. You can give hotkeys names that are also shown in the logs when the hotkey is activated.
There are two ways to trigger hotkeys:
• Keyboard-Hotkeys (PC/Mac only) o As the name implies, you can trigger hotkeys by pressing keys on your keyboard. o You can set combinations of up to 2 keys. o Keypresses are also read when the VTube Studio window is not in focus, for example when you’re playing a game. However, this does not work on macOS. o Also included: Mouse-Hotkeys (right/left/middle click).
• On-Screen Button Hotkeys (PC/Mac and smartphones) o Up to 8 hotkeys can be set as on-screen buttons. They will be shown as semi-transparent white circles on the right of the main screen (see picture 3). o Opacity can be changed. They can also be made invisible. o You can customize the color for each on-screen button.
o If you activate “Send hotkeys to PC”, pressing an on-screen hotkey on your smartphone will send a signal to your PC (if connected) that the key with that ID has been pressed. When the model you have loaded on your PC also has on-screen button hotkeys set up, the corresponding key will be triggered. o You do not need to have a model loaded on your smartphone to send hotkey presses to your PC. If no model is loaded, all 8 on-screen buttons will be shown.
6.5.7 Hotkey Actions
Various actions can be triggered by hotkeys:
• Play Animation (.motion3.json) o Will play the given animation once (overwrites face tracking) • Change Idle Animation (.motion3.json) o Will change the currently looping idle animation. Will not be saved into the VTS model configuration. • Set/Unset Expression (.exp3.json) o Toggles the given expression in the model (overwrites face tracking and animations) o If multiple expressions set the same Live2D parameter to different values, the last expression will decide the value. o More on this in the chapter “Expressions” • Remove all Expressions o Removes all currently set expressions. • Reload Model Texture o Reloads the model texture. This means you can modify your model .PNG-texture outside the app and reload it on the fly. Can be used for various cool effects during streams. • Move Model o Moves the model to the given position/rotation/size. o You can configure a time that the movement should take. o The position for the hotkey can be set using the “Record Position” button, which will save the current model position, rotation and size. The numbers shown are: ▪ x-pos/y-pos/size rotation • Change Background o Changes the background to the one you select for this hotkey. • Reload Microphone o Reloads the currently active microphone used for voice lipsync. • Calibrate Camera o Triggers camera calibration. • Change VTS Model o Changes the VTube Studio model to the one you select for this hotkey. • Take Screenshot o Triggers screenshot with the previously set screenshot settings.
For animations and expressions, there are a few extra settings you can use to customize fade times and animation playback behavior. For example, you can stop animations/expressions after a certain number of seconds or once the button is let go.
For animations specifically, you can set them to stop on the last frame. This will stop the animation on the last frame once playback has finished and hold all parameters there until you press the hotkey again. All Live2D parameters used in face-tracking for your model will not be affected by this and fade back automatically after the animation has finished playing regardless. 7 Expressions (a.k.a. „Stickers“ or „Emotes“)
Expressions are created using the Live2D Cubism Viewer. You can download it for free as part of the Live2D Cubism Editor: https://www.live2d.com/en/download/
Expressions allow you to set Live2D parameters to certain values using hotkeys. Using expressions, you can for example trigger different facial expressions or costume changes.
To create expressions, first load your model into the Live2D Cubism Viewer by dragging in the .model3.json file. Your model should now be open and ready to edit.
To create a new expression, you can now do the following:
1. Add a new expression using File → Add → Expression 2. Make sure Animation → Apply Expression is activated. 3. There should now be a folder called “Expressions” in the navigation window. Open it and click your newly created .exp3.json file. 4. Create your expression by adjusting the sliders and checkboxes for the parameters you want to be part of the expression. 5. Fade-times are ignored by VTube Studio in the current version. Instead, a default fade-time is used. Alternatively, you can set a fade-time inside of VTube Studio. 6. You can also right-click a parameter and set it to “Multiply”. This is not yet supported by VTube Studio and may result in your expressions not working. Please leave it at “Additive”. 7. To save the expression, right-click your .exp3.json file and select “Save”. Save it somewhere next to your VTS/Live2D model or in any subfolder of that folder so VTS can find it when your model is loaded.
To set up the expressions in VTube Studio:
1. Create a new hotkey inside of VTube Studio as explained in the chapters about hotkeys. 2. Use the hotkey type “Set/Unset Expression” 3. Select your newly created expression from the list. If you can’t find it there, make sure the .exp3.json file is in the same folder as your VTS model file or in a subfolder of it. 4. Your hotkey is now fully set up. You can use either your keyboard or the on-screen buttons to activate and deactivate it.
When multiple hotkeys modify the same Live2D parameter, the value of the last activated hotkey is used. When you then deactivate the “last” hotkey, the value is returned to the value of the “previous” hotkey. When all hotkeys that modify a certain Live2D parameter are deactivated, that parameter is returned to the value from the animation, the value from face tracking or the default value. More on this in the chapter:
Interaction between Animations, Expressions, Face Tracking, Physics, etc.
8 Animations
Animations can be created with the Animator in the Live2D Cubism Editor. Once your animation is done, select File → Export for Runtime → Export motion file.
This will give you .motion3.json files (Live2D animation files) that can be read by VTube Studio. Put them in the same folder as your VTS model or a subfolder.
These animations can be used as looping background idle animation or one-shot animation.
In VTube Studio, you can also use various settings for animation playback, such as: • Ending animation after X seconds. • Holding animation on last frame • …
9 Interaction between Animations, Expressions, Face Tracking, Physics, etc.
In VTube Studio, Live2D parameters can be controlled by various entities, such as Live2D Animations, face tracking or Live2D Expressions. But what happens when animations, face tracking and expressions try to write to the same Live2D parameter at the same time? That is explained in this chapter.
In VTube Studio, each Live2D parameter can be controlled by one of six “value-providers”. In order of priority (lowest to highest), they are:
Priority Value-Provider Notes P0 Default Live2D parameter value Available for every Live2D parameter P1 Value from Idle Animation P2 Value from Face Tracking P3 Value from One-Time Animation As long as animation is active P4 Value from Live2D Expression As long as expression is active P5 Value from Physics System
The final value of an individual Live2D parameter in your model will be determined by the highest- priority active value provider. For example, when a Live2D parameter called ParamA has a one-time animation (triggered by hotkey) writing to it, any values from the face tracking or idle animation are overwritten.
Once the one-time animation ends, control will be passed back to the active value-provider with the next highest priority.
Specifically, this means that the face tracking would now control ParamA if ParamA is set as an output parameter in your VTS model config. If not, ParamA will now be controlled by the idle animation. Or, if the idle animation also does not contain ParamA, it will be set to its default value, which every Live2D parameter has.
When control of a Live2D parameter is passed between value-providers, it is always faded smoothly and not just set instantly to prevent any ugly jumps.
10 Streaming to Mac/PC
The main use-case of VTube Studio is to use your smartphone for face tracking, render your model in the Mac/PC app and then use OBS (or recording similar software) to stream to YouTube or Twitch. Any Windows/Mac recording software should work with VTube Studio.
If you use an iPhone/iPad for tracking, you can use WiFi or USB to stream the face tracking data to the PC app. On Android, you can only use WiFi.
VTube Studio uses your local network (TCP/UDP) or USB to send face tracking data from your smartphone to your PC or Mac. Depending on your home network setup, there may Be connection issues, most frequently caused by your firewall configuration. For more info, check out the chapter:
Connection Issues & Troubleshooting
10.1 Streaming per WiFi - Preparation on PC/Mac
Streaming the face tracking data from your smartphone to PC/Mac is easy. First, make sure both your smartphone and PC/Mac are on the same local network, otherwise they will not be able to see each other.
Then, start the desktop app. In the settings, you can activate the server after choosing a port (try leaving the default port first, 25565).
This will start the VTube Studio server on your PC on every available network device. Since your PC may have multiple network devices, you may see multiple IPs when clicking the “Show IP list” button.
While your IPv4 shown here is typically a local, internal address and cannot be publicly accessed over the internet, your IPv6 addresses are likely to be public addresses! Please be careful to block them out if you post screenshots on the internet. Stay safe!
When starting the server for the first time, you may see a popup from the Windows Defender Firewall. Make sure you click “Allow access” and “Private networks” is checked. Otherwise, your smartphone will not be able to connect to the PC app as connections will be blocked by the firewall.
The server is now active and is listening for connections from the smartphone app. Leave it this way.
10.2 Streaming per WiFi - Connecting the Smartphone App
Next, start the app on your smartphone. In the settings, you can manually type in the IP and port from the desktop application or use the “Find Server” button to automatically scan for the server and use its IP and port. This scan should not take longer than 5 seconds.
Now, click “Connect to PC”. The app will now connect to the server. If the connection is lost for some reason, the app will try to re-establish the connection automatically.
Congratulations. You are now streaming your face tracking data to your PC. You can now open a VTS model in the desktop application and it will use the face tracking data from the smartphone. You can also have the same or a different model open in the smartphone app at the same time if you’d like.
You do not need to have a model open on your smartphone when streaming to PC. In fact, it is recommended to not have one open for performance reasons. When using “Streaming Mode”, any opened models on your smartphone will be automatically unloaded when connecting.
10.3 Alternative to WiFi: Streaming per USB from iOS to PC/Mac
If you use an iPhone/iPad for tracking, you can also use a wired USB connection to send over the face tracking data in real time. This is very reliable and more stable than WiFi but also a newer feature so it’s now as well tested. Right now, this is the recommended way to stream face tracking data.
To start a connection, activate the “Connect USB” switch on your phone first. If a face tracking WiFi connection to your PC is active, this will also deactivate it as they cannot be running at the same time. Afterwards, activate the “Connect USB” switch in the app on your PC/Mac.
If your device doesn’t show up or fails to connect, make sure all of the following are true.
• iOS device is connected to the PC per USB cable. • iOS device is unlocked and running the VTube Studio app. • The app on iOS says “USB started. Waiting for PC app to connect.” • The iPhone is “paired” with the PC (the phone must “trust” the PC). This can be checked by making sure iTunes can see the contents of the iPhone. • Important: Make sure you have installed iTunes!! iTunes starts some background services on Windows/MacOS that are required for communication with iPhones. iTunes doesn’t have to be running, but make sure it is installed on your PC/Mac and has been started at least once.
10.4 Recording with OBS (transparent background)
On your PC in VTube Studio, choose the Color Picker background. On Windows, you can use the “Transparent in capture” option here, which will make the window background transparent when the window is recorded in OBS so you don’t need to use greenscreen/chroma keying.
On macOS, this feature is not supported by OBS, so choose any color you want here. You can remove that color from the video capture in OBS using a “Chroma Key” filter. Keep in mind that you can’t use that color in your Live2D model, otherwise some parts of your model may be transparent as well.
Add a background in OBS, then add a capture for VTube Studio. To do that, select “Game Capture”. This supports transparent backgrounds but is only available on Windows. On macOS, this option is called “Syphon Client”, but does not work after macOS 10.14 Mojave (see https://github.com/zakk4223/SyphonInject), so you need to use a regular window capture and a chroma key filter.
Your capture is now active. On macOS, you need to add a filter to remove the green (or whatever color you selected) background using Right Click your capture -> Filters -> Add Filter -> Chroma Key.
You can now place your character anywhere on screen by moving/scaling/rotating it in directly in VTube Studio or in OBS.
Alternatively, you can use the Virtual Webcam feature or NDI in VTube Studio to create a webcam stream and then directly use that in apps like Zoom, Discord, etc. (see FAQ).
10.5 Using NDI for streaming to OBS with invisible UI (transparent background)
It is recommended to use OBS “Game Capture” to record the VTS window. Alternatively, you can use the Virtual Webcam or Newtek NDI (Network Device Interface) to create a video stream that can be used as input for software such as OBS.
The quality and latency of NDI streams is very good. It also supports a transparent background (no chroma/color key needed) and does not record the VTube Studio UI at all. CPU utilization of VTS may increase when using NDI. OBS plugins exist for macOS and Windows
OBS Plugin Page: https://obsproject.com/forum/resources/obs-ndi-newtek-ndi%E2%84%A2- integration-into-obs-studio.528/
OBS Plugin Download (Win/Mac): https://github.com/Palakis/obs-ndi/releases/tag/4.9.1
To use NDI, download the plugin from the GitHub-page linked above (.exe for Windows, .pkg for macOS) and install it. In VTube Studio, activate the NDI toggle in the “Camera Settings” tab. It is possible but not recommended to have NDI and the Virtual Webcam active at the same time.
In OBS, the following settings are recommended. Sometimes the NDI streams seems distorted in OBS. If that happens, resize (per drag) the VTube Studio window until the stream “snaps back” to looking normal.
11 Connection Issues & Troubleshooting
Under normal circumstances, there should be no connection issues. There also should be no delay and no lag or hiccups/spikes. The streaming should be smooth and should have at most around 10 milliseconds of delay.
However, depending on your local network setup, there might be some issues. This chapter discusses possible causes and solutions.
11.1 Possible Causes for Connection Issues
If you have trouble connecting your smartphone to the PC app (smartphone app is constantly stuck on “Connecting…” or server scan doesn’t find anything), please work through the following list before asking questions on the Discord.
• Make sure there is no firewall blocking the connection. In 90% of cases, this is what causes the connection problems. o To test this, try temporarily turning off your built-in firewall (Windows Defender) and any additional network security software you have. If this fixes the issue, turn them back on and find out how you can add exceptions for VTube Studio (“VtubeStudio.exe”). For Windows Defender, see the image below. o After downloading updates for the PC app, you may have to re-add it to the firewall exceptions. • Make sure both devices (PC/Mac and smartphone) are on the same network. o This means both should be on the same WiFi. If the PC and/or smartphone are connected to your router via cable, that is fine too. o If your router has multiple WiFi networks available, make sure it is the same for your PC and smartphone. Also, try both networks. • On Windows, make sure the used network is set to “Private” (see image below). • Try turning off your Anti-Virus software temporarily. • If “Updated” are shown as 0, you may have to restart the app once. • If you have a VPN, see if there's an option related to “Invisible to Devices” switched on. • Make sure iOS/Android and the app are up to date. • Network extending devices and access points like Google Nest may cause problems. • Check your router settings. Sometimes there are privacy settings activated that prevent devices from discovering each other. • If your smartphone can’t see your PC at all, there are issues with your network setup that you’ll need to fix before VTube Studio can work. To test this, use a network discovery / “ping” app and let them run a discovery on your network. o Just search for “Network Scanner” on the respective smartphone app store. Any of those apps should work. o If you run into specific issues, try to Google them first before asking on Discord. • Sometimes, VTube Studio reports the wrong server IP in the Steam version. Try opening a terminal on windows and type “ipconfig”. If this shows IPs not listed in VTube Studio, try out all of them in the smartphone app.
11.2 Network Lag Troubleshooting (general)
If you experience lag or short breaks/freezes when streaming to PC, work through the following list:
• First, try restarting your router. That sometimes helps. • Are you on WiFi? o In some extreme cases, your WiFi reception may cause lag or even very long (multiple seconds) delays in packets being sent over your network. Make sure you use 5GHz WiFi and not 2.4 as depending on your environment, 2.4 Ghz WiFi can have wildly unstable latency. o With WiFi, the latency still shouldn’t be higher than 10 milliseconds. • If possible, use wired LAN connection instead of WiFi. This will give you a lag-free, 1-ms latency connection. You can use wired LAN on your smartphone using an adapter such as this one for Android (USB-C) or this one for iPhone. • Try changing the WiFi-channel that your router uses. When there are routers nearby sending on the same channel, this will cause lag/packet drop issues. • If nothing helps, try out “ping” / “network analyzer” apps from your smartphone app store to analyze the network connection between your smartphone and PC. This often helps you understand when/why the freezes occur.
11.3 Network Lag Troubleshooting (macOS and iPhone)
If you are on macOS and iOS, there might be some other causes:
• Turn off location services, they will cause lag spikes periodically when your Mac scans your environment for WiFi networks. To turn that off, follow this guide. o https://osxdaily.com/2018/08/20/disable-location-services-mac/ • If you’re using WiFi, remove all unused WiFi-Networks from your list of known networks on both iPhone and Mac to stop them from scanning for them constantly. You can follow this guide. o https://9to5mac.com/2018/07/20/mac-how-to-forget-wireless-networks/ • When using iPhone and MacBook on WiFi: Very important to turn off AirDrop on the Macbook, because it WILL cause massive lag spikes when the iPhone is close to the Mac. Open a console and type: o sudo ifconfig awdl0 down • Later, when you’re done streaming, you can turn it back on by rebooting your Mac or typing this in a console: o sudo ifconfig awdl0 up • Turning off Bluetooth on your Mac and iPhone can also help. You can follow this guide to turn of Bluetooth on your Mac. o https://support.apple.com/guide/mac-help/turn-bluetooth-on-or-off-blth1008/mac
12 Taking/Sharing Screenshots
With VTube Studio, you can take screenshots directly from within the app, save them and share them on social media. Screenshots are saved as .PNG file. You can also turn the background transparent so only your model and any items in the scene are visible in the screenshot.
Furthermore, you can set up a hotkey to take screenshots using the previous screenshot settings.
.
On Windows and MacOS, screenshots are saved to your user’s “Pictures” folder. On smartphones, screenshots are saved directly to the phone gallery. The app will ask for permission to use this folder.
Screenshots with transparent background are really useful for producing PNGs of your model that you can for example use in video/stream thumbnails.
On smartphones, you can also directly share your screenshots on Twitter, Discord, etc.
13 Item System
Starting with VTube Studio version 1.5.0, you can import your own props (“items”) directly into the scene and even attach them to your Live2D model.
Items are read from the “Items” folder next to your “Live2DModels” folder. VTube Studio comes with a few items included, but you can add your own by putting PNGs into that folder.
If you put a folder into the “Items” folder, this folder will be considered to contain the frames for an Animated Item. PNGs you put into that folder will be read in alphabetical order and combined into an animation that can be loaded like a regular item in VTube Studio. Keep in mind that items with many frames may take up to a couple seconds to load.
FPS for the animated item can be set freely between 0 and 60 and once set will be remembered when you load the item again.
It’s recommended but not required to keep frames of an animated item at the same size. The “hitbox” (clickable part of the item) will be based on the first frame of the animated item.
.
First, choose the item you’d like to add. Then choose an order. Items with a higher order are in front of items with a lower order. The Live2D model itself is at order 0, so you can also place items behind it.
If you want the item to be able to attach to the Live2D model, make sure to select “Pin item to Live2D model”
There is an experimental “Censor” option that applies a mosaic censor effect to the item. There is also a “Mirror” option that reflects the item.
If the item is animated (must be multiple PNGs in a folder in the “Items” folder), you can set the FPS here. It’s generally recommended to keep item FPS under 45. You can put any floating-point number here, so even something like 3,7 FPS is valid.
Once the item is in the scene, you can move/scale/rotate it with the same controls as for the model on PC/Mac and iOS/Android. To attach an item to the model, just drop it on there.
The item will be attached to the uppermost ArtMesh in your model at the position where you drop the item. The point of attachment will be the CENTER OF THE ITEM PNG. This can make it a bit tricky sometimes to attach the item to the correct ArtMesh, so you may need to play around a little bit to make it work properly. Keep in mind that this feature is in beta.
Once your item is at the correct position, you can double-tap (double-click) it to lock the position. That way, you can no longer move, scale or rotate the item until it is unlocked again. If the item is attached to your model, it will still move according to the model movement.
If you move, scale or rotate the Live2D model itself, any locked items attached to the model will also move, scale and rotate with it as you would expect.
To unlock locked items, choose the option from the main menu (see arrow in second image). By pressing that button once more, all items will be locked again.
To delete items from the scene, simply drag them over the trash can icon in the bottom right. It will only appear if you drag an item near it.
Most of the free included items/props have been provided by @catboymech and @7MDigital on Twitter. They have many other great VTuber assets as well, so please make sure to follow and thank them!!
14 VTube Studio Files and File Format
All files VTube Studio generates to save your model setup and meta data are fully human-readable, not encrypted and use a JSON format.
Currently, VTube Studio saves all model-related configuration data in a file right next to your Live2D model. The file is called:
There is no documentation available for the file format at this point, but if you’re interested, just open them up in your favorite text editor. Most things should be relatively self-explanatory. But please be advised that manipulating your model fines manually may result in broken files so please be careful.
15 FAQ
15.1 Can my face be accidentally revealed on stream?
No, that can’t happen. VTube Studio uses your smartphone camera just for face tracking and does not store or transmit your face or any raw face tracking data to your PC. Only the calculated parameters are transmitted, so at no point will any video be available on your PC. The webcam version also has no functionality whatsoever to reveal the webcam video.
If you also want to make sure your face is never shown on your smartphone screen, make sure to set “Camera Preview” to “Never Show” (see chapter “Camera Settings”).
Don’t worry so much that people could see your face. You are beautiful! ♥
15.2 How do I buy VTube Studio? Why should I? Can I use it commercially?
There are two ways to purchase VTube Studio depending on how you want to use it.
• I want to use iPhone/Android tracking only. o First, download the VTube Studio app from the iPhone or Android App Store. The download itself is free. Then go into the app settings. There’s a big button to buy the PRO version. o The PRO version gives you access to some features not available in the free version, such as being able to stream to your PC for longer than five minutes and being able to remove the VTube Studio logo in the app. o You do not need to purchase anything in the Steam version now, just download it for free and use it. No watermark is shown while using iPhone/Android tracking.
• I want to use webcam tracking only. o Download on Steam for free and try out the webcam tracking. You will notice a watermark being shown only when the webcam tracking is active. o Purchase the “remove watermark” DLC to remove the watermark.
If you want to use both webcam- and iOS/Android-tracking, you’ll have to buy VTS on iPhone/Android and Steam.
I know the watermark is super cute, but please buy VTube Studio regardless! Due to popular demand, you can now turn her back on by unchecking “Hide VTS Logo” in the settings. You can also add her to your scene as an item/asset using the item system.
By the way, the animation was made by Walfie (@walfieee)
As a private person/streamer, you are allowed to use VTube Studio commercially even without purchasing any PRO versions or DLCs. This includes any kind of commercial use, including (but not limited to) monetized YouTube, Twitch, …, streams, superchats, Live2D model commissions, etc. If you can afford it and like VTube Studio, please support future development by purchasing it! <3
This only applies to individual creators, smaller VTuber agencies and smaller companies in general. As a bigger company, you will need additional licensing. Please refer to the VTube Studio EULA for more information and/or contact us directly.
• EULA: https://denchisoft.com/license/ • Contact: [email protected]
15.3 Can I try VTube Studio before buying it?
You can and should!
The app on Android and iOS itself is free and has a PRO version. The free version contains all features (including loading your own models), so please try VTube Studio first before purchasing.
On Steam, the app is free as well including all features. You only need to purchase the Steam DLC if you want to get rid of the watermark otherwise shown while using webcam tracking.
15.4 Can I use my webcam for face tracking instead of my smartphone?
Yes, the Steam version supports webcam tracking.
15.5 How can I use VTube Studio together with my friends?
VTube Studio does not support loading multiple models. However, if you want to do a collaboration stream with your friends, that is possible by using Zoom/Discord/Skype/etc. group video chat.
When using Discord for example, just have everyone use the Virtual Webcam feature of VTube Studio and then select that webcam as your main video chat camera in Discord. Alternatively, record using OBS and then use the OBS Virtual Webcam plugin to create a webcam stream for Discord. The person “hosting” the stream/collab can then record their Discord window using OBS, cut out everyone’s streams and use chroma key filters to remove the backgrounds. This works quite well, but you might need the high-quality 1080p video chat upgrade in Discord to get good picture quality.
It is also possible to use OBS over the internet using LAN-Over-Internet tools like Zero Tier One or Hamachi (free up to 5 participants). You could start multiple instances of VTube Studio on the “host” PC and then have people connect to them. Just remember to use different ports for each instance! Disclaimer: I have not personally tested this method, but users have reported that it works. Of course, there may be network delay.
15.6 Can I use USB instead of WiFi to connect my smartphone?
You can if you use an iPhone/iPad for face tracking. Direct face tracking data USB connection is not supported on Android. USB is very stable so this is the recommended way to send face tracking data.
On Android, as an alternative you could connect your smartphone via USB and the use USB tethering to put them in the same network even without WiFi.
15.7 What is the “Virtual Webcam” feature?
“Virtual Webcam” (available on Windows only) allows you to record the VTube Studio window (without the UI) and directly make it available as a webcam stream, making it easy to use it in Discord, Zoom, etc.
First, you need to install the virtual camera by pressing the “Install” button. This should show a windows console popup and may ask you for administrator privileges, which you’ll have to grant. After successful installation, you can check the “Activate virtual webcam” option to start the webcam stream.
You can now select the stream as normal webcam in Discord, OBS, Zoom, etc. The name of the webcam is “VTubeStudioCam”. The virtual webcam also supports streaming with transparent background. Please note that the video quality with the virtual webcam is lower than what you would get when directly recording the window with OBS.
If the installation did not work, make sure to start VTube Studio as admin. If the webcam still does not show up (for example in OBS), make sure OBS is also started as admin.
15.8 My device gets really hot! What is “Streaming Mode” in the smartphone app?
This is unfortunately normal. Your device may get really hot, but with the correct settings, it should not lead to any serious issues such as overheating or crashes.
To reduce heating, there are three things you should do when streaming to PC:
• Unload your Live2D model on the smartphone. You do not need to have a model loaded when streaming to PC and having a model open on your smartphone requires a lot of performance, which can be too much for some smartphones. • Remove the camera preview. Rendering the camera preview in the app can also cause some performance issues when done for a long time. Try turning it off. • Turn the screen brightness all the way down. This will significantly reduce the heat produced by your phone.
When using these settings, streaming for 15+ hours should be no problem (keep the phone charged). Tested on iPhone X and iPad.
“Streaming Mode” automatically removes the model, removes the camera preview and lowers the screen brightness when connecting to your PC. It is recommended to always use streaming mode when streaming to your PC.
15.9 Are there differences between the macOS and Windows version?
Windows supports all features. MacOS does not support transparent backgrounds with OBS and background hotkeys. This means hotkeys will only work on macOS if the VTube Studio window is focused. When for example playing a game at the same time, the hotkeys will not activate.
Also, webcam tracking for macOS is highly experimental and will most likely stay that way. Please keep that in mind when using VTube Studio for macOS.
15.10 Why does my stream to PC/Mac lag?
This is almost always caused by network issues. As a first thing to try, restart your router/switches, PC and smartphone. If the issues persist, check the chapter “Connection Issues & Troubleshooting”.
Sometimes, “gaming performance improvement” tools like Razer Game Boost can also cause performance issues, so try deactivating them and see if that improves things. that in mind when using VTube Studio for macOS.
15.11 Can I use my DSLR camera for webcam face tracking?
Should generally work but people have reported some issues. You may have to turn off “USB selective suspend” for some DSLRs. If the camera still doesn’t work in VTube Studio, try tunneling it through OBS by using the OBS virtual webcam functionality and reading that virtual webcam with VTube Studio. This may cause performance issues on some PCs but it should work.
15.12 Why is the app running at crazy FPS (400+)?
This is a bug caused by some GPU drivers reporting incorrect display refresh rates, causing FPS to get unlocked when VSync is active in VTube Studio so it will try to render at the fastest possible speed.
To fix this, turn on G-SYNC in the Nvidia GPU settings (or the equivalent for your GPU). If that still doesn’t work, consider setting the FPS config to 60 in VTube Studio instead of VSync.
15.13 Why is blinking and eye tracking not supported on Android?
VTube Studio uses Google ARCore on Android. Eye/Blink/Wink-Tracking is not supported by this framework yet. There is no information if/when this will become available. It is possible that it will never be supported.
If and when it is added to ARCore, it will also be supported by VTube Studio.
15.14 I bought the PRO version on Android. Can I transfer it to iOS? Or Steam?
As a general rule, no. Technically, it is not possible as Google and Apple use different app-stores. However, I may grant a grace period of one month.
If you purchased VTube Studio on Android and want to “transfer” that license to iOS, you can contact me per mail or on Discord. If you do, please first also purchase it on iOS and provide proof of purchase. I will then refund your purchase on Android. Transferring a purchase to/from Steam is also not allowed by Valve DLC guidelines. But if you use iOS/Android tracking, you do not need the Steam DLC because all it does is remove the watermark when using webcam tracking. No watermark is shown in the Steam version when using smartphone tracking.
15.15 VTube Studio doesn’t work on my smartphone. What can I do?
Please make sure you’re using a supported device as detailed in the chapter “Supported Devices”.
15.16 The face tracking is going crazy on my iPhone/iPad! What’s going on?
If the tracking mask in the camera preview is flying around or glitching out randomly, this is usually caused by magnetic interference. The iPhone/iPad uses a magnetometer (digital compass) to calculate the device facing direction. Please make sure your device does not have a case with magnets inside and is not leaning against a computer monitor. Having the device on a wireless charger may also cause similar issues.
In some instances, it can also be caused by bad lighting. Make sure you have good, non-colored lighting in your tracking setup. Some colors seem to confuse iOS/Android face tracking. Try going in a different room and see if that helps.
It has also been reported that mirrors somewhere in the camera view can confuse the tracking.
15.17 Why does my iPhone/iPad show the forward-pointing camera?
This means your device is not supported. You should also have seen a popup telling you this when the app starts. For supported devices, see the chapter “Supported Devices”.
15.18 Where can I get help/support?
Discord or mail. Check the first page of this document.
15.19 How can I become a VTube Studio Partner?
The partner program is currently closed. Once it is opened again, it will be announced on Twitter so please stay tuned!
15.20 Are MacOS and Linux supported? What about M1 Macs?
• Linux: Not officially supported. Some people managed to run VTS on Linux emulated so you could try getting in contact with them in the VTube Studio Discord. • MacOS: Supported experimentally with limitations. Download normally through Steam but please be aware that some functionality may not work, including stable webcam tracking and sound lipsync. Silicon M1 Macs should also work but there may be even more stability issues.
15.21 Can I use VTube Studio in my company? How about for TV ads?
Depending on the size of your company, you may need additional licensing. Please see the VTube Studio EULA or contact me for more information.
15.22 I can’t load my Live2D model in VTube Studio. What’s wrong?
First, check the logs if there are any errors. If there are errors in the logs but no error popup was shown, there may be a bug in the app. Please contact me on Discord so I can take care of it.
There have also been cases of Antivirus software blocking the app from reading models. Try disabling them or adding an exception for VTube Studio. One such example is “IObit Malware Fighter”.
15.23 Can I use the iPhone/Android tracking at the same time with webcam tracking?
Not sure why you would ever want to, but yes. If you have both active, webcam tracking will be used as long as the webcam can track you. Once it loses tracking, VTS will automatically use the tracking data received over the network instead.
15.24 The Webcam tracker doesn’t work! What should I do?
Are you on Windows 8/10 N Edition? If so, you’ll have to install the Media Feature Pack first: https://www.microsoft.com/en-us/software-download/mediafeaturepack
Otherwise, check the logs and post them in the VTube Studio Discord. Make sure no other app is currently using the camera. Try plugging the camera out and back in. If nothing helps, try rebooting.
15.25 I am on MacOS and the webcam tracker is broken.
Webcam tracking on MacOS is experimental and may not work at all depending on your device. Before purchasing any DLCs, please make sure you are happy with the tracking on MacOS as I cannot guarantee it will work in a stable way. In the worst case, if VTS or any other app can’t access your webcam anymore, you may have to reset your SMC like this:
System Management Controller (SMC) Reset Guide: https://support.apple.com/en-gb/HT201295
That said, there are quite a few users who use VTube Studio on MacOS and do not have any problems.
15.26 Does VTube Studio store/record any personal data?
No.
Specifically, no personal data is stored on the device or sent to any external servers by VTube Studio. For more information, see the privacy policy on https://denchisoft.com/privacy/
15.27 Who is using VTube Studio? I want to see some examples!
VTube Studio is used by many indie and bigger fulltime VTubers as well as VTuber agencies on a regular basis. Here are some random notable examples in no particular order:
Agencies • Nyanners --- [Twitter] from VShojo • Froot --- [Twitter] from VShojo • Zentreya --- [Twitter] from VShojo • Hime Hajime --- [Twitter] from VShojo • IronMouse --- [Twitter] from VShojo • Liliana Vampaia --- [Twitter] from MyHoloTV • Hoshino Char --- [Twitter] from Tsunderia • Kamiko Kana --- [Twitter] from Tsunderia • Inukai Purin --- [Twitter] from Tsunderia
Indie VTubers • Asagiri Yua --- [Twitter] • Savi --- [Twitter] • Yue/Chai --- [Twitter] • Rosuuri --- [Twitter] • Bao --- [Twitter] • Selphy --- [Twitter] • Artemis of the Blue --- [Twitter] • … and thousands more! Just search VTube Studio on Twitter!
16 Privacy Policy and Licensing
For up-to-date information about licenses and the VTube Studio privacy policy, please check the VTube Studio website: https://denchisoft.com/license/
Live2D avatars (except the “Akari” model) are © Live2D Inc. Use of Live2D avatars is governed by the terms listed on the Live2D samples website at https://www.live2d.com/en/download/sample-data/
17 Further Terms
This project is being developed and maintained by Denchi (https://twitter.com/DenchiSoft).
Some of the graphics in this document have been taken from https://www.irasutoya.com/ in compliance with their usage terms.
Disclaimer: This app is an indie production and is not developed or published by Live2D Inc., but is officially licensed to use the Live2D Cubism SDK. Do not contact Live2D Inc. about any questions regarding VTube Studio.
For business and support inquiries, you may contact DenchiSoft at the following address: [email protected]
The Akari chibi animation was drawn by Walfie (@walfieee)