4
How to move psql database from 13 to 15 for lemmy 0.18.4 (monero.town)
submitted 1 year ago by [email protected] to c/[email protected]
132 comments fedilink hide all child comments

Backstory: I had a debian 11 VPS. I installed the postgresql program from the debian 11 apt repo, back a few months ago. Back then, it was postgres 13 on the debian stable repos.

Fast forward couple months: debian 12 comes out. I do a "apt dist-upgrade" and in-place upgrade my VPS from debian 11 to debian 12. Along with this upgrade, comes postgresql 15 installed.

Now, fast forward couple more months: lemmy 0.18.3 comes out. I do not upgrade (I am on lemmy 0.18.2---afaik).

Fast forward some time, too: lemmy 0.18.4 comes out. I decide to upgrade to 0.18.4 from my existing 0.18.2.

I pull the git repo. Compile it locally. It goes well, no errors in the compilation process. I stop the lemmy systemd service, then I "mv" the compiled "lemmy_server" to /usr/bin dir.

I try to restart the now-upgraded lemmy systemd service. However, the systemd service fails.

I check the sudo journalctl -fu lemmy and I see the following error message:

lemmy_server[17631]: thread 'main' panicked at 'Couldn't run DB Migrations: Failed to run 2023-07-08-101154_fix_soft_delete_aggregates with: syntax error at or near "trigger"', crates/db_schema/src/utils.rs:221:25

I report this issue here: https://github.com/LemmyNet/lemmy/issues/3756#issuecomment-1686439103

However, after a few back and forths and internet search, I conclude that somewhere between lemmy 0.18.3 and 0.18.4, lemmy stops supporting psql <15. So, my existing DB is not compatible.

Upon my investigation on my VPS setup, I concluded that psql 15 is running, however, lemmy is using the psql 13 tables (I do not know if this is the correct term).

Now my question: is there a way to import the lemmy data I had in the psql 13 tables to a new psql 15 table (or database, I don't know the term).

To make things hairier: I also run a dendrite server on the same VPS, and the dendrite server is using the psql 15 with psql 13 tables on the same database as the lemmy one.

The dendrite database is controlled by a psql user named "dendrite" and the lemmy database is controlled by a psql user named "lemmy" . I hope this makes differentiation between two databases possible. And so I do not harm my existing dendrite database.

Any recommendations about my options here?

you are viewing a single comment's thread
view the rest of the comments
[-] [email protected] 2 points 1 year ago

I just arrived. I will be here for the next hour or hour-and-a-half.

Let's go if you are ready.

Here are some of the suggestions I have:

  1. Let's try putting the postgresql URL to the lemmy.hjson file and try again (it was in the systemd service file lemmy.service )

  2. or, anything you suggest

[-] [email protected] 2 points 1 year ago

ok, I'm here now

I don't think the URL gets read from the file, we need to find out if there is a port= setting in there

[-] [email protected] 1 points 1 year ago

I don’t think the URL gets read from the file

By "file" you mean the lemmy.service systemd file?

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

no, the lemmy.hjson...

Good news is I found the docs: https://github.com/LemmyNet/lemmy/blob/main/config/defaults.hjson

EDIT: make sure 5433, was 5432 before edit

 # Port where postgres can be accessed
 port: 5433

By “file” you mean the lemmy.service systemd file?

Speaking of, I think remove the environment variable from there, as it will take precedence.

[-] [email protected] 2 points 1 year ago

Speaking of, I think remove the environment variable from there, as it will take precedence.

I will remove the following line we have added yesterday:

Environment=LEMMY_DATABASE_URL=postgres://lemmy:[email protected]:5433/lemmy

How do we transform this line so that it fits into the lemmy.hjson file?

[-] [email protected] 2 points 1 year ago

And need to correct the lemmy.hjson to 5433 - right?

[-] [email protected] 2 points 1 year ago

/etc/lemmy/lemmy.hjson file doesn't have a port directive in it yet. We are going to add one now. And yes, the expected port number should be 5433, which is for postgresql 15.

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

So to recap, should be easy for our next attempt:

  1. put the port in lemmy.json
  2. remove the environment variable from service file
  3. start attempt, see if it has your old data available
  4. If no old data, stop immediately

I suppose for future, maybe worth some effort to block federation in the NGINX or something to make sure federation doesn't kick off on wrong data if it isn't what we expect

[-] [email protected] 2 points 1 year ago
  1. done
  2. done
  3. started lemmy.service

no errors, status shows active. I also started lemmy-ui.service

again, no errors, status shows active 4. my domain.tld on my tor browser (my usual browser) still shows a Setup Screen for a new Lemmy instance.

My suggestion: Although I have updated lemmy backend to the v0.18.4, I have not yet upgraded the lemmy-ui (frontend) to its latest version. Should I compile lemmy-ui again, so that we have its latest version, too, and then try again?

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

yha, it is (lemmy-ui update) probably ok to do it since you are not in an urgent attempt to get online.

This isn't what I expected.... it isn't talking to your PG13 that 0.18.2 had (or it would error) and it seems to be talking to the empty new database again.

Make sure you stop server. I think we need to pg_dumpall and try to make sense of this

[-] [email protected] 2 points 1 year ago

I stopped the lemmy backend and its frontend.

Should I re-compile the lemmy-ui and try again? Or do you want to examine the pg_dumpall first?

[-] [email protected] 2 points 1 year ago

i'm in agreement with you on your idea to update lemmy-ui (yarn and all that)....

As maybe it's talking to the wrong data, but I doubt it. Anyway go ahead on that first.

[-] [email protected] 2 points 1 year ago

Anyway go ahead on that first.

OK. You will hear from me in 10 minutes.

[-] [email protected] 2 points 1 year ago

OK. I have updated the lemmy-ui to 0.18.4

Then I have restarted the lemmy.service and lemmy-ui.service

Upon visiting my domain.tld, I am still being greeted by "Setup New Instance" UI. So, bad news it is.

How would you like to proceed? Do we look at pg_dumpall now?

[-] [email protected] 2 points 1 year ago

Do we look at pg_dumpall now?

yes... you have the command from last time, this time we do pg_dumpall

[-] [email protected] 2 points 1 year ago

I am here now. Are you there?

[-] [email protected] 1 points 1 year ago

yep, here

[-] [email protected] 2 points 1 year ago

yes, let's go

[-] [email protected] 1 points 1 year ago

sorry about yesteday crash, I purged some lemmy server performance data I had and hopefully won't happen again.

Did you get a pg_dumpall?

[-] [email protected] 2 points 1 year ago

Did you get a pg_dumpall?

we still have the one from the first time we exported the database from psql 13. Do you want to get a new one from the one we (tried to) import to the psql 15? If so, how was I doing it again?

[-] [email protected] 2 points 1 year ago

So, the command I do is the following?

sudo -iu postgres /usr/lib/postgresql/15/bin/pg_dump --port=5433 lemmy > /someplace_with_space/lemmy_databackup_2.sql

Observe that I have modified the --port to 5433 (which is the psql 15) and I have appended a "_2" to the exported sql filename.

[-] [email protected] 1 points 1 year ago

on, sorry, and REMOVE the "lemmy" after the 5433 port

[-] [email protected] 1 points 1 year ago* (last edited 1 year ago)

this time we want to change the /bin/pg_dump to /bin/pg_dumpall

EDIT: and remove "lemmy" after port 5433

[-] [email protected] 1 points 1 year ago 
user@server:~$ ls -lh
total 54M
-rw-r--r-- 1 user user 5.8K Jul 18 18:08 cargo_build_output.txt
drwxr-xr-x 3 user user 4.0K May 12 14:09 go
-rw-r--r-- 1 user user  54M Aug 23 00:59 lemmy_databackup.sql
-rw-r--r-- 1 user user 1.3K Jul 20 09:59 lemmy_matrix_Caddyfile_example.txt
-rw-r--r-- 1 user user  547 Jul 18 16:46 yarn-error.log
user@server:~$ sudo -iu postgres /usr/lib/postgresql/15/bin/pg_dumpall --port=5433 > /home/user/lemmy_databackup_2.sql
[sudo] password for user:
user@server:~$ ls -lh
total 109M
-rw-r--r-- 1 user user 5.8K Jul 18 18:08 cargo_build_output.txt
drwxr-xr-x 3 user user 4.0K May 12 14:09 go
-rw-r--r-- 1 user user  55M Aug 25 00:55 lemmy_databackup_2.sql
-rw-r--r-- 1 user user  54M Aug 23 00:59 lemmy_databackup.sql
-rw-r--r-- 1 user user 1.3K Jul 20 09:59 lemmy_matrix_Caddyfile_example.txt
-rw-r--r-- 1 user user  547 Jul 18 16:46 yarn-error.log
user@server:~$
[-] [email protected] 1 points 1 year ago

well the size is about the same, 55M vs 54M, so the restore worked.

We need to study what is in this file and try to make sense of how there are two Lemmy databases in there. What you are looking for is sections like this:

--
-- Database "lemmy" dump
--

--
-- PostgreSQL database dump
--

We are looking for different names or something, because somehow you have one with only a few communities and messages, another with far more.

[-] [email protected] 2 points 1 year ago

So, I am cat -ing this file into less pager and inspect by my eye and do keyword searches?

[-] [email protected] 1 points 1 year ago

pretty much, unless you have other ideas. you could download the file and edit locally too...

[-] [email protected] 2 points 1 year ago

OK. I am examining the file. I see the sections you mention. What else should I be looking for?

[-] [email protected] 1 points 1 year ago

the name, and another one of those sections.

[-] [email protected] 1 points 1 year ago

ok, this grep might be of use:

grep --after-context=6 "Data for Name: local_site; Type: TABLE DATA;" lemmy_databackup_2.sql

I'm not sure if there is anything sensitive in that output... but I think it will be revealing how many matches you get.

[-] [email protected] 2 points 1 year ago

grep --after-context=6 "Data for Name: local_site; Type: TABLE DATA;" lemmy_databackup_2.sql

I think I got two matches on that.

[-] [email protected] 1 points 1 year ago

yha, I think that's the problem... you have two installs, but we need to figure out their names

we could grep for "-- Database " and see if it gives you two names...

[-] [email protected] 1 points 1 year ago

we want fresh. this time pg_dumpall, and point to psql 15 port... do NOT use the same filename ;) Make sure we export to a new file.

[-] [email protected] 2 points 1 year ago

you there? my server crashed right after your message to upgrade, and I'm back now.

[-] [email protected] 2 points 1 year ago

I'm back, my server crashed, sorry

[-] [email protected] 1 points 1 year ago

I have to leave and catch some sleep. This was my latest message: https://monero.town/comment/1266295

If you have some suggestions for me to try, just type it here and I will try during the day tomorrow and write back.

I will be around the same hours tomorrow again.

[-] [email protected] 1 points 1 year ago

How do we transform this line so that it fits into the lemmy.hjson file?

We don't... your lemmy.hjson you were using for 0.18.2 - if you used the same password, the only thing we need to do is add the port to 5433... the URL doesn't need to go into that file

[-] [email protected] 2 points 1 year ago

if you used the same password

Yes, I used the same psql password.

OK. Then, I am going to remove the

Environment=LEMMY_DATABASE_URL=postgres://lemmy:[email protected]:5433/lemmy

line from the lemmy.service file. And then, I will add the port directive.

The updated /etc/lemmy/lemmy.hjson file is as follows now:

{
  database: {
    # put your db-passwd from above
    password: "REDACTED"
    port: 5433
  }
  # replace with your domain
  hostname: domain.tld
  bind: "127.0.0.1"
  federation: {
    enabled: true
  }
  # remove this block if you don't require image hosting
  pictrs: {
    url: "http://localhost:8080/"
  }
}

observe that I have added the port: 5433 to the database: clause.

[-] [email protected] 2 points 1 year ago

yep, looks good. We will find out if the two formats talk to different data ("DATABASE" within PostgreSQL 15)

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

lemmy.hjson

Good news, found the docs:

# Port where postgres can be accessed
port: 123
[-] [email protected] 2 points 1 year ago

soo, what should I add to the lemmy.hjson file now?

[-] [email protected] 2 points 1 year ago

replied to other msg

this post was submitted on 22 Aug 2023
4 points (100.0% liked)

Lemmy PostgreSQL

33 readers
2 users here now

founded 1 year ago
MODERATORS
[email protected]
[email protected]