How to move psql database from 13 to 15 for lemmy 0.18.4 (monero.town)

submitted 1 year ago by [email protected] to c/[email protected]

132 comments fedilink hide all child comments

Backstory: I had a debian 11 VPS. I installed the postgresql program from the debian 11 apt repo, back a few months ago. Back then, it was postgres 13 on the debian stable repos.

Fast forward couple months: debian 12 comes out. I do a "apt dist-upgrade" and in-place upgrade my VPS from debian 11 to debian 12. Along with this upgrade, comes postgresql 15 installed.

Now, fast forward couple more months: lemmy 0.18.3 comes out. I do not upgrade (I am on lemmy 0.18.2---afaik).

Fast forward some time, too: lemmy 0.18.4 comes out. I decide to upgrade to 0.18.4 from my existing 0.18.2.

I pull the git repo. Compile it locally. It goes well, no errors in the compilation process. I stop the lemmy systemd service, then I "mv" the compiled "lemmy_server" to /usr/bin dir.

I try to restart the now-upgraded lemmy systemd service. However, the systemd service fails.

I check the sudo journalctl -fu lemmy and I see the following error message:

lemmy_server[17631]: thread 'main' panicked at 'Couldn't run DB Migrations: Failed to run 2023-07-08-101154_fix_soft_delete_aggregates with: syntax error at or near "trigger"', crates/db_schema/src/utils.rs:221:25

I report this issue here: https://github.com/LemmyNet/lemmy/issues/3756#issuecomment-1686439103

However, after a few back and forths and internet search, I conclude that somewhere between lemmy 0.18.3 and 0.18.4, lemmy stops supporting psql <15. So, my existing DB is not compatible.

Upon my investigation on my VPS setup, I concluded that psql 15 is running, however, lemmy is using the psql 13 tables (I do not know if this is the correct term).

Now my question: is there a way to import the lemmy data I had in the psql 13 tables to a new psql 15 table (or database, I don't know the term).

To make things hairier: I also run a dendrite server on the same VPS, and the dendrite server is using the psql 15 with psql 13 tables on the same database as the lemmy one.

The dendrite database is controlled by a psql user named "dendrite" and the lemmy database is controlled by a psql user named "lemmy" . I hope this makes differentiation between two databases possible. And so I do not harm my existing dendrite database.

Any recommendations about my options here?

you are viewing a single comment's thread
view the rest of the comments

[-] [email protected] 1 points 1 year ago

Can you comment with your /etc/lemmy/lemmy.hjson file?

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

Here it is:

$ cat /etc/lemmy/lemmy.hjson
{
  database: {
    # put your db-passwd from above
    password: "REDACTED"
  }
  # replace with your domain
  hostname: hostname.tld
  bind: "127.0.0.1"
  federation: {
    enabled: true
  }
  # remove this block if you don't require image hosting
  pictrs: {
    url: "http://localhost:8080/"
  }
}

I redacted the password, and the hostname entries.

[-] [email protected] 1 points 1 year ago* (last edited 1 year ago)

you don't have a port number specified in lemmy.json it is either using the default value or you have LEMMY_DATABASE_URL set?

Do you have an environment variable: LEMMY_DATABASE_URL in your lemmy_server service config or otherwise?

Lookibg at the Rust source code, there is a comment that says "// The env var should override anything in the settings config"

[-] [email protected] 2 points 1 year ago

I think my setup is using the default value.

I do not see me specifying the LEMMY_DATABASE_URL in my systemd file:

$ cat /etc/systemd/system/lemmy.service
[Unit]
Description=Lemmy - A link aggregator for the fediverse
After=network.target

[Service]
User=lemmy
ExecStart=/usr/bin/lemmy_server
Environment=LEMMY_CONFIG_LOCATION=/etc/lemmy/lemmy.hjson
# remove these two lines if you don't need pict-rs
Environment=PICTRS__SERVER__ADDR=127.0.0.1:8080
Environment=PICTRS__STORE__PATH=/var/lib/pictrs/files
Environment=PICTRS__REPO__PATH=/var/lib/pictrs/repo
Restart=on-failure

# Hardening
ProtectSystem=yes
PrivateTmp=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

[-] [email protected] 1 points 1 year ago

I do not see me specifying the LEMMY_DATABASE_URL in my systemd file:

ok, so you have a choice... we have to figure out the syntax of the port number in config file, or you introduce a LEMMY_DATABASE_URL in your lemmy.service file.

[-] [email protected] 2 points 1 year ago

I would go with introducing LEMMY_DATABASE_URL in my lemmy.service file. However, is doing that going to expose my lemmy database password to the lemmy.service file?

[-] [email protected] 1 points 1 year ago

yes, the password will need to be in the LEMMY_DATABASE_URL... but a lemmy.service should be secure... ? I mean it's kind of thing the system using during boot.

[-] [email protected] 2 points 1 year ago

but a lemmy.service should be secure…

yeah it is secure, as the root user password is required to do anything with that file. Alright, let's do this.

[-] [email protected] 2 points 1 year ago

So the next step is the backup, did you do the backup of lemmy data from 13? And spot-check that it looks like your data?

[-] [email protected] 2 points 1 year ago

user@server:~$ sudo -iu postgres /usr/lib/postgresql/15/bin/pg_dump --port=5432 lemmy > /home/user/lemmy_databackup.sql
user@server:~$ ls -la lemmy_databackup.sql
-rw-r--r-- 1 user user 56554817 Aug 23 00:59 lemmy_databackup.sql
user@server:~$ ls -lah lemmy_databackup.sql
-rw-r--r-- 1 user user 54M Aug 23 00:59 lemmy_databackup.sql

[-] [email protected] 1 points 1 year ago

it's plain text, I suggest grep against it for something you know is in there... that you created? Just to be sure when we do a restore against port 5433 we aren't overwriting good data ;)

[-] [email protected] 2 points 1 year ago

I grepped the exported sql file. I identified a long blog post of mine. So others should be there, too.

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

Ok, so now we restore to 15... double-check we have the port right for this...

Ok, so there is one issue we might run into. the lemmy user probably doesn't exist on the PostgreSQL 15.3 install - so try these from the from-scratch install documentation:

sudo -iu postgres /usr/lib/postgresql/15/bin/psql --port=5433 -c "CREATE USER lemmy WITH PASSWORD 'db-passwd';"
# maybe not needed with restore? sudo -iu postgres /usr/lib/postgresql/15/bin/psql --port=5433 -c "CREATE DATABASE lemmy WITH OWNER lemmy;"
sudo -iu postgres /usr/lib/postgresql/15/bin/psql --port=5433 -c "ALTER USER lemmy WITH SUPERUSER;"

Then to do the restore:

sudo -iu postgres /usr/lib/postgresql/15/bin/psql --port=5433 --file lemmy_databackup.sql

The restore may end up creating the lemmy user and a password for it, I'm not sure... so if you used different passwords for 13 and 15 it might have gone back to the 13 password for that user. Haven't tested.

[-] [email protected] 2 points 1 year ago

user@server:~$ sudo -iu postgres /usr/lib/postgresql/15/bin/psql --port=5433  -c "CREATE USER lemmy WITH PASSWORD 'REDACTED';"
[sudo] password for user:
CREATE ROLE
user@server:~$ sudo -iu postgres /usr/lib/postgresql/15/bin/psql --port=5433 -c "CREATE DATABASE lemmy WITH OWNER lemmy;"
CREATE DATABASE
user@server:~$ sudo -iu postgres /usr/lib/postgresql/15/bin/psql --port=5433 -c "ALTER USER lemmy WITH SUPERUSER;"
ALTER ROLE
user@server:~$ sudo -iu postgres /usr/lib/postgresql/15/bin/psql --port=5433 --file lemmy_databackup.sql
psql: error: lemmy_databackup.sql: No such file or directory
user@server:~$ ls -la lemmy_databackup.sql
-rw-r--r-- 1 user user 56554817 Aug 23 00:59 lemmy_databackup.sql
user@server:~$ sudo -iu postgres /usr/lib/postgresql/15/bin/psql --port=5433 --file /home/user/lemmy_databackup.sql
SET
SET
[...]
[many many rows of output]
[...]
ALTER TABLE
ALTER TABLE
REVOKE
GRANT

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

ok, so at this point your service file needs to have the correct LEMMY_DATABASE_URL environment variable, set to the PostgreSQL 15 install... 5433 port and correct password.... and you should be able to start lemmy service and the migration should work this time.... and you be up and running.

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

service file needs to have the correct LEMMY_DATABASE_URL environment variable, set to the PostgreSQL 15 install… 5433 port and correct password…

Can you give me the example for that directive? I can then modify it to my exact situation (password, etc.)

EDIT: Is it something like this? Environment="LEMMY_DATABASE_URL=postgres://username:password:5432/lemmy"

I am not sure on the username:password part btw

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

https://join-lemmy.org/docs/administration/configuration.html

Going by your previous service config example....

Environment=LEMMY_DATABASE_URL=postgres://lemmy:password@lemmy_db:5433/lemmy

With 5433 on there instead of what most people use, 5432.

now the "lemmy_db" is a hostname, which I pulled from those docs, but probably just set to 127.0.0.1 (or "localhost") instead of adding that to your hosts file?

[-] [email protected] 1 points 1 year ago

But the port should be 5433, right?

[-] [email protected] 2 points 1 year ago

yha, I edited the comment already. And I suggest you replace lemmy_db with IP address or localhost.... maybe we go with localhost

postgres://lemmy:password@localhost:5433/lemmy

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

I edited my reply...

now the "lemmy_db" is a hostname, which I pulled from those docs, but probably just set to 127.0.0.1 (or "localhost") instead of adding that to your hosts file?

[-] [email protected] 2 points 1 year ago

but probably just set to 127.0.0.1 instead of adding that to your hosts file?

You can see the lemmy.hjson file I posted in this thread, I think it is just set to 127.0.0.1 in there

[-] [email protected] 2 points 1 year ago

ok, then 127.0.0.1 should work...

[-] [email protected] 2 points 1 year ago

To verify, here is what I have added to my lemmy.service file:

Environment=LEMMY_DATABASE_URL=postgres://lemmy:[email protected]:5433/lemmy

[-] [email protected] 2 points 1 year ago

looks good

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

So, I save the edit and quit the lemmy service file. And then do sudo systemctl enable --now lemmy.service ?

EDIT: OK. Systemctl reports that lemmy.service is active and runnning.

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

I assume it is already enabled from when you ran 0.18.2...

on Ubuntu 22.04 if I try to start it after editing the service config it tells me to run a command to rebuild some cache. But I don't know if Debian is using different service control architecture... hard to keep up with the debates ;)

I don't see any harm to just issue a start

[-] [email protected] 2 points 1 year ago

I did start my lemmy.service, and then I started my lemmy-ui.service (I had it disabled, too, since the lemmy.service wasn't working for the last week).

Now I navigate to the frontpage of my lemmy instance (hostname.tld) and lemmy-ui is asking me to setup the instance.

Is something wrong? I was expecting my old posts with the old sub-lemmy's show up?

[-] [email protected] 2 points 1 year ago

lemmy-ui is asking me to setup the instance.

yha, that's no good... it thinks it is talking to an empty database.

Is something wrong?

yes.

[-] [email protected] 2 points 1 year ago

When I click "Linked Instances" in the lemmy-ui, it starts populating the old lemmy instances that my instance was connected to. Maybe if I wait sometime, my posts would return?

[-] [email protected] 2 points 1 year ago

I started a new trunk branch comment

this post was submitted on 22 Aug 2023

4 points (100.0% liked)

Lemmy PostgreSQL

33 readers

2 users here now

founded 1 year ago

MODERATORS

[email protected]