-
Notifications
You must be signed in to change notification settings - Fork 1
/
todo.txt
247 lines (199 loc) · 12.5 KB
/
todo.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
bugs:
figure out why sometimes there are no stats being shown
testing:
test interpolation things
test music song number
test interpolation_observations code
tj:
does settings overwrite work for items that are lists?
we have option to show first prompt in youtube description
maybe eventually put in the changing aspect of prompt
maybe make it a function that takes in the prompt and the it maybe just blanks, or something like 1-100 or w/e
Figure out why difuse has to be seperated and have video settings between itself
learn how to make a gist branch and use it
list all features
get a settings.txt that is for quick testing with really low settings and use that for testing
make a settings.txt that has core settings for face animations so that i can just leave that in
and not have to change everything every time
get noise injector hooked up
used_series in try_to_get_inbetween needs work, it should check the original series and see if they are something other than their default
that does nothing
change Box Zoom/OD to Guided Zoom/AI Guided Zoom
is prompts hooked up to frame multiplier? it should be
special consideration for antarctic prompts
make instructions on getting youtube jsons set up for other used_series
figure out why new users have to use DD before PD works for them
get noise injector hooked up
used_series in try_to_get_inbetween needs work, it should check the original series and see if they are something other than their default
that does nothing
bug : sometimes when i stop and start generating again, it generates from an earlier frame than i am actually at,
figure out why this happens and how to stop it. there is probably something counting completed frames and setting the start
point from there, this should be in prepare diffuse
realESGAN:
with how it is right now it will mess things up when it stops in the middle of the run,
it will move all the images out of the main folder and run esgan on them and put the esgan scaled images in the main folder
really, we want to just put the esgan images in a new folder and leave the originals where they are.
if esgan is being used, then change where we get the images from. the base folder should always have the originals
guided zoom:
make instructions on how to use guided zoom next to guided zoom
change Box Zoom/OD to Guided Zoom/AI Guided Zoom
bug?: doesnt seem to be zooming in as much as it should
Eventually:
hook up OD zoom
possible to update position/size of box as zoom happens?
external TPU
get it set up on a the free account(so long as the free account can connect to main gdrive)
make it so multiple requests can be sent from multiple other gpus
just send a random id number with a request image and then delete the file with that id number
and send the id number backon the response file, then have the original sender look for that id number when selecting a file and when deleting
antarctic text prompts not deleting the .txt after prompts are used, and the start_time is not being reset after it is used
in video animation:
instead of using the video frames, after the first frame use the interpolation of the previous generated frame and the next video frame.
also, make a way to sometimes use exactly the video frame(in these cases don't do the above thing)
when i stop and restart a print with antarctic prompts it gives me the original prompt again instead of having
the most recent antarctic prompt
make instructions to use rife exe in features
hook up toggle to turn off rife interpolation
it should only work if doing video animation
make a setting series for turning it on and off for certain frames
aud 0 - no interp
aud 2 - interp at first woman (but less frames_scale/frames_skip_steps as 0)
aud 3 - interp whole time (lower specs b/c worse gpu)
aud 4 - exactly the same as 0, but with interp the whole time
aud improvements:
around 398 the white clock lines up perfect as a big daisy behind her ear
make her start out as barely vaguely alien/cyborg
he then starts out as a normal human
add keyframes to basic settings and make a pull request of it
turn ccmixter stuff into a seperate pip that takes in a search term and eventually other restrictions
and automatically downloads the song and returns a path to it and its credits
video frame interp helper:
this needs a better name
make its settingsc
onsolidate its notes
maybe taking a 1/4 interp would make it more cohesive and less flickery
put ffmbeg and new upscaling into youtube description
eventually make a warning for when an interpolation is attempted with settings that have failed in the past
this isnt totally straight forward because i think it also depends on if another gpu is runnnig and how intensely
SUPER Eventually, once we get a custom interpolation set up we can use that if the settings as is would crash.
custom film interpolation (maybe this could eventually be a fork of FILM instead of code that is in Pop Diffusion)
input is animation style "preceding frame #: (interp #),..."
create a series from whatever is input
current_interp_frame_num
loop through all the frames:
check to see if the next frame is already in interp folder,
if it isn't:
copy frame to interp folder and rename it based on current_interp_frame_num
update current_interp_frame_num
calculate how many frames should be in there for this interpolation
frames_from_interpolation = 2 ** (interpolation_number) - 1
check to see if at least this many frames exist in interp folder
if they do:
update current_interp_frame_num
continue loop
if they don't:
do this interpolation
rename the newly interpolated frames based on current_interp_frame_num
update the current_interp_frame_num
copy preceding fame to interp folder
-------------------------------------------------------
ability to use a user provided mp3 instead of ccmixter
try adding text in the middle of animation creation just to see the effect
make a ccmixter 'never again' mp3 list and check for it when getting music, if we get one in the list then loop
back and get another one
-this should be in github AI/Pop_Diffusion/ccmixter_never_again_mp3_list.txt
-take the url from the music credits (ex: http://dig.ccmixter.org/files/speck/52473) and put it into a list in the txt file
-when getting a random mp3, check the list and if it is in the list, then get another one
how to start with all cells closed by default?
maybe this is not possible, ive checked some
can dynamically morph from x to y using weights in for loops
example: adorable: 10, creep: -10 ----> adorable: -10, creep: 10
maybe we need some sort of built in check to make sure that we do not sum to 0
i think this could be done in some way that uses the current 'try_to_get_inbetweens" function
it can use a list of strings instead of just a string though,
this way we can have multiple going in tandem
dynamic way to fade through random artists to keep the fade smooth
example: by 0:a,b,c ; 1:b,c,d ; 2:c,d,e... always just one artists coming and one going
In settings overwrite there should be an option to use the same antarctic prompts again
3D Color stuff:
test a long generation with color stuff to see if we can keep color
if zippy 3d color fx works well,
get PD set up for 3d and move color fx stuff over
once this is done, maybe make the first frame become the color pallete for the rest with an option to change it to a later frame
OD
figure out what point we are at with OD and get it working with specific items
predefined objects to zoom into
beyond:
it asks what to zoom into while the print is happening
this could even be done without OD and with a click and drag
OD Eventually:
gpt3 can be used to combine object detected and current prompt into a new prompt
OD Look at
make an info print of the object that is being zoomed in on (this means send it over from the TPU)
make a way for more settings to be input, number of frames to reach zoomed object, object to zoom in on if available
pick either:
number of frames to reach goal
speed to head towards goal
--------for both of these we probably want to pace it so that we reach both at the same time
maybe sometimes we pick a target based on how much it has to do with current or next prompt, not sure
maybe sometimes we use the label of the target to help create the next prompt with gpt3
Rotatable guided zoom box:
would need to alter the current box selector library
want to kep track of the top of the square and the way that it is rotated
how to import my own external library?
can i make my own pip? somehting else?
test:
test info and prompt videos without antarctic prompts
test new antarctic text promp formating
AP Eventually:
hook up another ai that determines category of antarctic phrases and choose artists/modifiers from list
how can antarctic prompts happen with one TPU and multiple main programs?
a ranadom antarctic and is in the request image and resulting txt
move youtube, gist,to Pop Directory
3D for color:
bug:
if it crashes and resumes then i lose all the antarctica prompts
is this still an issue? if so then it can be checked by starting a run with some antarctic prompts and t
hen turning it off and back 9on
try this without another machine running:
https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/GPT-J-6B/Inference_with_GPT_J_6B.ipynb#scrollTo=9vnZfO1HGSS7
=======================================================
Settings expirements:
play with cut_pow less 1.0
try antarctic with just an init image and either skip all steps on first prompt or get it to make even the first prompt
so this way it is completely the ai deciding what to do
Live Interaction Stuff:
a way to add prompts while generation is happening
a button to cancel current frame/image and delete it
New Animaiton mode:
make cutn scheduling have new animation mode so it can be different from frame to frame
look into making models be done with the "new" system
try training with my own art dataset
Misc:
when i do a series of images in a batch, i should be able to get the seed of each one (when it is on random)
figure out about gpt finetunning and see if can fine tune on zippys and discord channels to make a helpful DD bot
a way to load settings from settings file
this should override the settings in the markdowns
possible to use new init images throughout the print? maybe something involving reverse or something to make it gradually go from one init to another through a prompt? not really thought out yet..
a way to make comparisons of two different changing settings or a control next to each other in one video. Ex: show two slideshows in the same video, in one of them sat_scale is changing, in the other one it is not, and everything else is the same.
a grid comparison of changing settings and their combinations, this could even be a 3d grid or a many dimension grid that is viewed by changing settings(between pre-defined, pre-printed settings) and seeing result in real time
an estimated completion time would be cool, but probably difficult
Meta:
figure out a way to test code from in here
maybe replace google drive with a local folder
there is something in some versions about setting it up to work with cpu instead of gpu, but maybe there
would be a way to use it with alocal gpu too
Research:
find out how many notebooks other people are running at same time in pro+
learn about skip_augs
bbox jupyter:
run this in an ipynb
https://github.com/gereleth/jupyter-bbox-widget/
figure out how to modify its code so that you can change things (for example make the p key do the same thing as the w key - move the box up)
in order to do this you will need to pull the repo instead of just installing it from pip and importing it
you will need to have the code locally and then you can import it
maybe before doing this you will want to learn how to make a simple python program and then import it as an external library
once you can do this then maybe you can modify the bbox widget code and import your modified code in the same way
the part of the code that assigns stuff to the keypresses is in here
https://github.com/gereleth/jupyter-bbox-widget/blob/main/src/BBoxWidget.svelte
eventually we want to modify this code in such a way that allows us to rotate the box so that it can be a diamond and everywhere in between