usr/ data dir #818
usr/ data dir #818
Conversation
frdel
left a comment
frdel
left a comment
There was a problem hiding this comment.
Thanks, adding some notes.
Some of these are my mistakes from the original ticket, sorry for that, but they are easy to fix.
Please test the migration on some of your dev environments and mapped docker folders to be sure.
Thank you.
python/helpers/memory.py
Outdated
| # load custom instruments descriptions | ||
| index = knowledge_import.load_knowledge( | ||
| log_item, | ||
| files.get_abs_path("usr/instruments/custom"), |
There was a problem hiding this comment.
should be just usr/instruments
python/helpers/memory.py
Outdated
|
|
||
| # Get subdirectories from memory folder | ||
| subdirs = files.get_subdirectories("memory", exclude="embeddings") | ||
| subdirs = files.get_subdirectories("usr/memory", exclude="embeddings") |
There was a problem hiding this comment.
no more need to exclude embeddings as they are now in tmp/
python/helpers/migration.py
Outdated
| _move_dir("tmp/upload", "usr/upload") | ||
| _move_dir("tmp/downloads", "usr/downloads") | ||
| _move_dir("tmp/email", "usr/email") | ||
| _move_dir("knowledge/custom", "usr/knowledge/custom") |
There was a problem hiding this comment.
should be just usr/knowledge (my bad, I wrote it in the ticket)
python/helpers/migration.py
Outdated
| _move_dir("tmp/downloads", "usr/downloads") | ||
| _move_dir("tmp/email", "usr/email") | ||
| _move_dir("knowledge/custom", "usr/knowledge/custom") | ||
| _move_dir("instruments/custom", "usr/instruments/custom") |
There was a problem hiding this comment.
should be just usr/instruments (my bad, I wrote it in the ticket)
python/helpers/migration.py
Outdated
|
|
||
| _move_file("tmp/settings.json", "usr/settings.json") | ||
| _move_file("tmp/secrets.env", "usr/secrets.env") | ||
| _move_file("tmp/default.env", "usr/default.env") |
There was a problem hiding this comment.
We don't use default.env, just .env (i forgot to add space between default and .env in the original ticket).
The .env is loaded in dotenv.py in helpers using get_abs_path(.env), this has to be adjusted for the new path.
|
|
||
| # Handle memory directory | ||
| memory/** | ||
| !memory/**/ |
There was a problem hiding this comment.
Let's keep all of these original folders here for safety, we don't want our devs to accidentaly upload their memory or files...
|
|
||
| # Handle knowledge directory | ||
| knowledge/** | ||
| !knowledge/**/ |
There was a problem hiding this comment.
Let's ignore knowledge/custom for safety.
Knowledge itself should not be ignored.
linuztx
force-pushed
the
usr-migration
branch
from
ea572a2 to
2facfbe
Compare
linuztx
force-pushed
the
usr-migration
branch
from
3fac135 to
2facfbe
Compare
2 participants
/usrdirectory: Moved all persistent user data (chats, uploads, downloads, tasks, secrets, settings) from/tmpto/usrto centralize storage and simplify backups./usr/knowledge/customand/usr/instruments/custom./usr/memory.defaultsubfolder)./tmp/memory/embeddings(transient data).python/helpers/migration.pyto automatically move existing data from/tmpto/usron startup.memory.py,persist_chat.py,task_scheduler.py,settings.py, and other helpers to reflect the new directory structure.attachmentsStore.jsand API documentation to point to/usr/uploadsinstead of/tmp/uploads..gitignorein favor of a single/usrexclusion rule