Node stops generating new files in the store folder after backup

[augustov]

Hello,

I have a node running previous 1.4.18 that has some rewards still to claim. After 1.4.19 this node stopped generating files in the store folder and I realized this several days after the node had been running.

I backed up what I had and tried to restore the node in a fresh install which had 350Q accumulated with all the .config folder contents and the node started running and kept my peerid but the reward started from 0. Meaning I had lost the Quil I had accumulated. Then, once I restored what I had the store folder kept the same way not generating new files but “accruing” rewards

So, knowing I had lost my Q I reinstalled again with only the keys and config file to see if the store folder problem will persist but after I restarted the node with the old keys and config file, the node didn’t keep generating files in the store folder.

The main issue is that once I back up in a fresh install, it stops generating files in the store folder.

So my question is, what is the best way to restore the .config folder and have a working node?. I don’t care that I lost the rewards accrued post 1.4.19. But since I still have some rewards previously to 1.4.19. I need to keep my old peer ID functional.

These are the files I have but I don’t know if I need to back them all up except the store folder, or only the keys and config(which I did and didn’t work)? What is the best way to solve this issue?

Any suggestions would be appreciated

[abo]

Hi,

Welcome to the forum! I edited the title of your post a bit to make it easier to find for others. Hope you don’t mind.

As I understand the issue is that the node stops generating new files in the store folder after backing it up. This is probably an issue with your backup procedure.

You can find generic information about backups/node migration here:

How to migrate or configure a new node

This is asked many times on Telegram and I think it would be helpful to share the steps required. Migrate an existing node to a new server You want to migrate an existing node to a new server. Do these steps on the old server: Stop the node. Make a backup of the .config/ directory. Do these steps on the new server: Restore your backup of the .config/ directory. Start node. Configure a new node You want to start a new node on a new server and make a config change such as enabling gRPC. O…

And there are a few guides out there with detailed instructions on how to create backups:

Node Runner Guides

The purpose of this topic is to gather links to node runner guides with an eye toward consolidating information that should be the same for all guides in one place (e.g. what files should be backed up). https://quilibrium.guide/ https://docs.quilibrium.one/ https://docs.quilibrium.space/ Kingcaster | Quil Node Running Guide - Mac M2 Mini Please comment to add any I might have missed.

Let me know if this doesn’t resolve your problem please!

[augustov]

Hi abo,

So I checked the links you provided me. The backup is being done correctly.

I am backing up the entire .config folder But as I’ve already stated, the store folder doesn’t have any files and the backup is done of everything except the store folder, that since 1.14.19 update, didn’t generate new files.

To be more specific. I don’t have store folder contents, once I reinstall the node and let it run for the first time it generates some files in the store folder. then, I proceed to back up the old things namely the keys, config.yml etc., and restart the node to retrieve my old peer-id. the store folder stops creating new files.

We used to restore only the keys and config file prior to 1.14.19 and this worked to keep the peer-id and the node will generate new store files. This time it’s not working for me

[abo]

I’m sorry I don’t follow. Could you write down the problem in this format:

• Steps to reproduce (eg. 1. delete the .config directory, 2. start a node with version X, 3. use X command to create a backup, 4. use X command to restore to Y location)

• Expected result (this is what you expect to happen after performing the steps above, eg. there are files in location X)

• Actual result (this is what happened after performing the steps above, eg. there are no files in location X)

[augustov]

1. Deleted the full ceremonyclient folder.
2. Git cloned the ceremonyclient folder again
3. Started node and let it run for a few minutes to build .config folder.
4. Stopped node
5. Replace files from .config folder from my backed up files on storj. Except store folder. Since as explained doesn’t have anything it’s empty. (To see what files I am replacing see image in the initial post)
6. Restarted node.

Expected result: Node runs as expected with my peer id from the backup and store files start generating.

Actual Result: Node runs as normal, shows correct peer id but store folder does not generate more files

Hope it is clearer now. Thanks!

[littleblackcloud]

Much clearer thank you!

You say “node runs as normal”, but that seems odd as it isn’t creating the store.

Do you see any errors in the logs?

Try deleting the MIGRATIONS and REPAIR files and restart the node.

[augustov]

Thanks for the follow-up. So the logs look good:

I did delete MIGRATIONS and REPAIR and restarted the node.

After leaving it running for some time this is what I see:

It recreates an empty store folder and doesn’t record any logs inside after being running for 20 minutes

[littleblackcloud]

The logs are showing that your node is calculating proofs (look at increment increasing).

Are you sure there’s nothing in the store folder?

Can you open a terminal and run ls -laR /path/to/ceremonyclient/node/.config. Replace /path/to/ with the actual path of your ceremonyclient directory.

One more thing to try and then I’m out of ideas without having access to the system myself.
Open .config/config.yml and look at the path to your store, e.g.

db:
  path: .config/store

What’s your path?

[augustov]

Ok man, you saved my life.

So basically for some reason the path was wrong. This is what I had:

db:
  path: .config.store

I corrected it to:

db:
  path: .config/store

and store folder started creating new logs right away.

Don’t ask me how it got messed up. But thankfully after your help and 500+ $QUIL lost node is working good now.

Thank you. I owe you a beer.

[demipoet]

Hi @augustov Just circling back to your original post here. The issue reported is that when you upgraded your node to 1.4.19 your rewards went to 0 QUIL.

However, this is not an issue at all, imo. Before 1.4.19 (1.4.18-p2, 1.4.18-p1, 1.4.18, and 1.4.17), all rewards are manually managed centrally and is posted on the website. At those points, node rewards management is not decentralized and hence not managed within your node. Only starting from 1.4.19 did that change. From 1.4.19, rewards management has become decentralized and each node rewards are managed internally on each node (whose same info are then propagated to other peers on the network).

Do note that rewards <1.4.19 are separate from the rewards starting from 1.4.19, and therefore everyone’s nodes starting from 1.4.19 will start their rewards counter from 0. The previous rewards are still claimable.

[augustov]

Thanks demi but no the issue is basically that I lost the ones I had accumulated from 1.14.19 on. Because the store folder was not logging anything. I understand the ones prior are not accumulated locally and will be claimable later. Thanks