Puppetstring VTuber Tracking
A downloadable tool for Windows, macOS, and Linux
Perform as a VTuber with Inochi2D models!
Webcam Face Tracking
Mic Audio Tracking
Display Color Tracking
Keyboard Toggles
Installing via the itch.io desktop app is strongly recommended for Windows users, to minimize download size and check/run the update automatically (for Linux, it's recommended to just download the ZIP archive).
Alternatively, you can also get Puppetstring on Steam.
At the moment, Puppetstring cannot transmit Bone transformation data to Inochi Session, however we have added new blendshapes as a replacement:
psHeadPitch (replacement for Head -> Pitch) psHeadRoll (replacement for Head -> Roll) psHeadYaw (replacement for Head -> Yaw) | psHeadLeftRight (replacement for Head -> X) psHeadUpDown (replacement for Head -> Y) psHeadFrontBack (replacement for Head -> Z) |
All of these blendshapes range from -1 to 1. This specific problem and workaround most likely will be resolved in the next Inochi Session update (0.8.x or 0.9).
If you like this app, don't tip here! Consider contributing to the Inochi2D Project instead:
Inochi Creator | Inochi Session | Inochi2D on GitHub | Inochi2D Discord | Inochi2D Patreon
Download
Development log
- Puppetstring is Now Available on Steam!79 days ago
- 0.0.4 Update: Default Expression AdjustmentJan 08, 2024
- 0.0.3 Update: Minor Feature Additions & BugfixesDec 14, 2023
- 0.0.2 Update: Minor Feature Additions & BugfixesOct 13, 2023
Comments
Log in with itch.io to leave a comment.
How do i simulate breathing with this? The Inochi sample model Midori and the bundled Clementine model do not breathe.
You have to rig the breathing first, you don't need this to simulate breathing as it can be done using Session only.
thanks for a quick reply, i didn't know at the time how to write an expression as opposed to a ratio in the tracking window or even what those words meant.
For those finding this thread later, how to breathe is on this page of the documentation (was hard to find as the homepage just says WIP) https://github.com/Inochi2D/inochi-session/wiki/Expressions
not sure where to put an error report, but puppetstring will capture the camera video but not display the wireframe that shows that it is identifying my face. I am on garuda linux (Arch based distro) if that helps.
Delete the `puppetstring` folder in .config if it's present. I've had this problem before and I *think* that's how I got it working again.
Thanks for the info! I ended up changing distros in the end because i wanted something a bit more stable for streaming. I'll keep that in mind for next time though.v
good news: I was able to get multi-instance puppetstring v21 working on both of my webcams.
bad news: my CPU did not appreciate the extra workload haha ORZ
comment: okay, I did some testing. note that not all bugs found were restricted to the use case or test case below.
use case: using keyboard controls to toggle emote blendshapes between 0 and 1 (key1..key9 toggling emote1..emote9) , accompanied by a separate group of keyboard controls mapped to the same key to set all emote blendshapes to 0 (a group of keybinds setting emote1..emote9 with key0). this would allow toggling independent emotes with key1..key9, and pressing key0 to turn off all emotes.
test case: keyboard control toggle; keybind (Alt)(1), blendshape "testShape", values {0,1}. keyboard control set; keybind (Alt)(0), blendshape "testShape", initial value {0}, set value {0}.
comment: Puppetstring 0.0.4 does not like it if you set multiple keyboard controls to the same blendshape. (that is, when several keyboard controls have their blendshape targets set to the same string.)
bug0: when a toggle keybind and a set keybind both target the same blendshape, the displayed values for the toggle keybind may appear to change correctly in Puppetstring, but the output blendshape communicated to session remains fixed regardless of keypresses. replicated successfully.
bug1: when a toggle keybind and a set keybind both target the same blendshape, clicking the edit button for the set keybind may instead result in opening the edit view for the toggle keybind, or opening the edit view for the set keybind as if it were an increment/decrement keybind. replication has been difficult; my conjecture is that the result of clicking the edit button for the set keyboard control may depend on variables or values that are persistent to a given instance of Puppetstring.
comment: as I was doing testing for this bug, additional bugs were discovered outside of the test case. keyboard set controls are hecked up, friendo. UI always goes like this lol
bug2: after editing a set keybind, future set keybinds may have an additional "Value 2" underneath "Key 1". for example, after three calls to edit any set keybinds, editing another set keybind might have a "Key 1" with the fields "Value 1", "Value 2", "Value 2", and "Value 2". partially replicated successfully; did not test whether or not the process is initiated by editing a set keybind which has multiple keys to set a blendshape to different values, or if it's initiated with any set keybind regardless of how many keys or values it has.
bug3: for a set keybind with multiple keys, the second key always sets the targeted blendshape to 0, regardless of the value entered for "Key 2" in its "Value 2" field. however, entering a value for "Key 1" "Value 2" will result in "Key 2" setting the blendshape to "Key 1"s "Value 2".
bug3 cont'd: "Key 1" "Value 1" {0.1}, "Key 1" "Value 2" {0.2}, "Key 2" "Value 2" {0.3}. this setup will cause "Key 1" to set the blendshape to 0.1 and "Key 2" to set the blendshape to 0.2. Replicated successfully.
bug4: editing a set keybind may result in all "Value 2" fields being initiated with the value from "Value 1", and the "Value 1" field being initiated as {0}.
comment: these are all the bugs I recalled and found in this sitting. there may be others I can't recall right now. and unrelatedto the bugs reported here, I have a suspicion there may be other bblendshapes like psHeadFrontBack where the values sent to Session may not match the values Puppetstring has, but I haven't gotten to around to actually testing this so I'm not confident whether this is the case nor which blendshapes may be affected.
final comment: best of luck, AR14. don't forget to hydrate, get up and move around, and find a bit of time to get some serotonin in.
okay, feature requests.
(or maybe more like feature suggestions? idk, ignore these if you want, I'm not your mom)
(also sorry about the messy formatting, the comment field is deeply disagreeing with me)
keyboard controls
* toggles with transition times
* timed sets/toggles (automatically reverts to 0 after a user-defined amount of time passes)
* "held" keystroke type (sets a blendshape only while key is held)
* access to numpad keys
widget controls
* panel of tileable(?) widgets whose outputs can be mapped to custom blendshapes
* buttons (essentially keyboard controls but mapped to a UI object)
* 1D sliders/dials
* 2D touch pads (basically just a square you click in)
misc. blendshape management
* blendshape aliases
> if blendshape has arg1 name, send its value as arg2 name (maybe averaging values if multiple are aliased to the same name? or just let Session handle it)
* custom blendshape expressions
> custom blendshapes that are calculated from other blendshapes, using expressions (Javascript?) provided by the user
* tracking-activated toggles
> custom blendshapes that are set/toggled/incremented when a set of user-defined conditions are met (various blendshapes equal/greater/less than user-defined values or other blendshapes).
* tracking loss blendshape
> a blendshape that toggles to 1 if tracking is lost, or a "tracking lost" condition for custom blendshape expressions
I have use cases in mind for all of these, but also I'm not going to perish if 0 of them appeal to you lol
where would be an appropriate place for feature requests/bug reports for Puppetstring?
You can just write it here or in Inochi2D's Discord server if you're already in it.