I have a simple wish, with a probably not so simple solution.
I recently started with linux (Arch kde), I’m loving it, I quickly realized that this OS and almost all apps, are highly customizable, I’m laving that as well. My problem is the unavoidable reinstalls and that I have a laptop.
Is there any way that I can save all my configs, apps and my apps’ configs, and transfer them over to my laptop, while almost having a very quick back-up. I realize that I could turn it into an ISO somehow, but that wouldn’t work (I think) because my laptop has vastly different hardware. I also realize the partitioning problem. So in my idealistic world, there should be a solution that requires a clean install (from scripts or manual) and some .sh file, that installs all my apps, pastes all my configs and reboots.
So is this possible? and if yes, how should I go about this? did someone make a tool for this already? Or(!) can I burn it to a flash and the drivers will correct themselves/I’ll deal with them later?
For final words I’d like to say that I’m far from finished configurating, but I’d like to know the proccess, to not shoot myself in the foot somewhere along the way of configing, thanks!
Reinstall? Why?
Create a separate partition for /home and don’t format it when reinstalling, so you will keep all your stuff.
This is the way, I’ve kept a 7 year old install going this way, through 3 laptops and even LUKS encryption.
I just copied my whole root partition to a new Laptop over netcat. It still has close hardware (Intel CPU, no extra GPU, etc.), but some differences in interfaces etc.
Things one might have to consider:
- /etc/fstab will need to be redone
- All interfaces changed, so network configs may need to be updated
- Other programs relying on hardware or paths that don’t exist anymore need to be updated (eg. conky did not work due to i8k being not supported, other interface ids etc.)
But literally nothing that would break anything. Because Arch is usually installed manually, one knows what needs to be cared for, what could break or could cause certain issues.
@lemmy_user_838586 @bizdelnick That is “a” way not “the” way.
deleted by creator
Analyzing your comment in a different light. What your saying is if I copy my /home (someone said /etc too) over to my laptop, and back it up as well, I’m golden?
would different hostnames and usernames make a problem? As far as my knowledge goes it won’t as long as I also bring /etc over, but I have no Idea if /etc is connected to something deeper or not.
And also also, might seem like a dumb question but I had to edit a file to automount my other disks at startup, won’t it like break everything if my system only gets /home after boot or something? Caz I have enought free space to copy over my existing /home, delete it, partition, and mount it back. What’d the benefits and dangers be?
if I copy my /home (someone said /etc too) over to my laptop, and back it up as well, I’m golden?
/home yes., but ideally only files and dirs starting with a dot (so called “dotfiles” under your home dir.
tar cvfa homedots.tar.gz /home/username/.??*should take care of it.Please note it will include some large stuff that’s probably not needed, like .cache, or some individual caches for other apps that don’t use .cache, like the browsers.
Don’t copy /etc, it’s usually machine-specific.
would different hostnames and usernames make a problem?
Hostname no (if you don’t bring etc). Username technically yes, you may want to rename the home dir. The user id and group id are important too but usually off it’s the first user on the same distro it will receive the same ids (typically 1000 nowadays). If not, you can change that manually and recursively
chown 1000:1000 -r /home/username.I’d love to upvote you multiple times, you cleared it up for me. Thank you.
You’re welcome.
To clarify, /etc can have things that are relevant for the machine so you may want to back it up, but it’s not usually transferrable directly to another machine because it probably doesn’t play the exact same role. It has things like service configs, network configs etc.
Even if you’re trying to migrate a machine to new hardware and the machine will play the same role it’s best to pick and choose files from /etc/ on a case by case basis. What I do is grab a tarball of /etc and set it aside, then if I need to redo something the same way it was on the old machine I can dig through the tarball and only use the relevant files.
Like I said it’s extremely specific. For example if I want to reconfigure the SSH daemon that’s usually a couple of lines which I know by heart (turn root login and password logins off) which I can do by hand; if I want to reconfigure CUPS printing it’s best to use the CUPS admin interface to autodetect the printer, you don’t usually want to mess with its config files; for some things like /etc/fstab or NFS or RAID I may want to copy some stuff but edit the disk UUIDs; for some things like Samba I could in theory copy the config straight over. It varies.
The list of installed packages may also be relevant when you migrate to a new machine. Different distros have different commands for obtaining a list of installed packages, and different ways of using that on the new machine to restore the same package selection. This is useful and typically can get you started much faster on the new machine.
You dont even need a separate partition, just dont format and dont delete the /home folder. You can even keep the /etc folder as well to keep system wide settings.
There’s a distro called NixOS that is created for this purpose. It also has a tool called home manager that will manage your dot files for you. Once you back up like two or three configuration files you can recreate your system (minus any actual data)
When you do this in Arch there’s no guarantee you get the same package versions and there’s no guarantee everything works
To add to this, another viable path is using Nix, the package manager, on its own. That way you can get Home Manager to manage your applications and dotfiles independently of your base system, as long as you are able to install Nix.
It’s my general workflow, run Determinate Nix Installer, install Home Manager, clone my config and I’m off to the races. Been sharing that config between Debian, Ubuntu on WSL and Bazzite for a while and it’s served me well so far.
For the OS side a few ways.
- Clone & then rename+change drivers
- Ansible/chef
- NixOS
For home folder side of things a dotfile manager, cloud services, and file sync tool will take care of most things. I use chezmoi for dotfiles & nextcloud for file syncing. Firefox is only cloud synced service I still use for now. I have yet to find any decent sources of information on dotfiles so gonna be stuck going through those stupid things to figure out what you want to sync.
mkstage4 is exactly for this. It’s basically just a wrapper for tar which excludes unneeded directories for you already.
I’ll look into it, thanks!
heading in a completely different direction that what you were aiming for, but the declarative distros (currently a subset of immutable distros) like NixOS and Guix are trying to solve just this sort of issue – their main focus is on dealing with development environments but a lot of people have been enjoying them on desktop environments as well
ex. with NixOS, your entire system configuration is stored in one master config file
/etc/nixos/configuration.nix(that you can optionally keep synced with git) – the main config can be modularized (ie. break out the hardware definitions into its own include so you can still use the master config on both desktop and laptop) – and Nix has been making big strides with Home Manager, their own way of being able to collect and define all of your home directory config files and themingcurrently, NixOS is not for the faint-of-heart, documentation (both quality and lack of) regularly gets critiqued – NixOS and Nix package manager are all configured in the Nix language, a functional language used nowhere else
Guix comes out of the GNU project so dealing with proprietary drivers is harder than it needs to be – Guix is configured in Guile Scheme
You’re not the only one who mentioned NixOS. But you warned me about the quality. Thank you.
Also, I’d really not want to switch of my current system. I already have data, configs and everything. I probably could re-do it in days, but seems like a lot of struggle to use a worse distro in the other 99% of the time when I’m not thinking about moving configs to my laptop.
It’s a better distro, not a worse one. This is because it has easy rollbacks, upgrades, etc. You never get stuck with a broken system since the previous state is in another entry when you boot. You only need to hit down arrow and enter to go back to the previous configuration even if you can’t boot right now
There is a pacman command that prints the list of all packages installed by users. I don’t remenber the command sorry but you’ll find that here:
https://wiki.archlinux.org/title/Pacman/Tips_and_tricks
Its probably “pacman -Qe”.
Then it should be easy to create a script that install all that automaticcally. If your are cautious you should have a backup of your home anyway on some storage device .
Huh, does everything store it’s configs in ~/.config ? Is it some unwritten rule? That’d make life so much easier.
Unfortunately not every app follows that convention. A lot of them just dump config files into your home directory, including Firefox.
There’s a script called XDG-Ninja that can list some of the apps that do that. You can probably get it from the AUR
At least it’s somewhat narrowed down, I’ll try the app, thanks!
Sorry I edited my post, I was wrong.
.config stores many apps settings. But unfortunately some apps stores that directly in ~ as hidden files and directories. Personnally I make a backup of my whole home.
oh, no problem, at least I know now. Thanks!
I think you could install your system using a generic kernel, package it up as ISO and just boot it on basically any other machine with the same architecture. Proprietary bits like NVidia driver and firmware could pose a problem.
That’s basically what a live USB is.Reinstalling is Windows-user logic. Also why start with Arch Linux of all distros.
friends used it, tryied it, stuck with it. But wdym Windows user logic? What do I do when I brick something?
It’s really hard to [accidentally] permanently break Linux to the point of requiring a reinstall.
Here’s a really good tip: Keep a live distro (I use Mint) on a USB drive. If something real bad happens, you can boot into the live distro, and chroot into your OS and do the repairs you need. While also having a live distro with web access and a browser to help.
I broke my GRUB once (or twice) and fixed it again this way.
Keep a backup of your /boot folder, GRUB (or equivalent) configs, etc, also check documentation on Arch wiki for boot process. 99.9% of the time you should be able to fix things to at least get to a TTY after boot.
That takes stress off of my back. But does not solve the problem, that I’d like an identically configured portable system (my laptop). :/
I do have live USBs laying around, thanks for the tip.
You could likely use
ddor clonezilla to create a duplicate of your boot drive and boot your laptop right from that, but that’s not quite what you’re after.There are some distros lately that use a declarative config file to set the whole thing up that I think is much more what you have in mind. The big ones that come up a lot are nixOS and Fedora Silverblue. Maybe one of those systems would be to your liking.
What your saying is that I can dd / ?
that’s very alien to me.
dd duplicates directories. It’s a terminal app. Built into all Linux distros. For more details, do a
man ddin a terminal session. Clonezilla is a distro that runs a live system from USB or DVD which lets you backup and restore entire systems. Both are powerful, but have a learning curve.
Usually with most Linux distributions you can just make a tarball of the entire system (don’t forget the p to preserve ownership,…) and unpack it to a new partition, install the boot loader again and it should work on a new system, as long as the kernel does work with the hardware on the new system. Alternatively you can reinstall and keep your home directory to keep all your user config.
I don’t know by heart if it’s able to do your bidding, but perhaps it’s worth checking out penguins-eggs. I guess the following would be its elevator pitch:
"penguins-eggs is a console tool, under continuous development, that allows you to remaster your system and redistribute it as live images on usb sticks or via PXE.
The default behavior is total removal of the system’s data and users, but it is also possible to remaster the system including the data and accounts of present users, using flag --clone. It is also possible to keep the users and files present under an encrypted LUKS file within the same resulting iso file, flag --cryptedclone.
You can easily install the resulting live system with the calamares installer or the internal TUI krill installer."
This sounds scary, but an option! I’ll put it at the end of the list, thanks!
So is this possible? and if yes, how should I go about this? did someone make a tool for this already? Or(!) can I burn it to a flash and the drivers will correct themselves/I’ll deal with them later?
I think this is where a few respondents got the impression you are looking at this like a Windows install. It is not. All of the drivers, minus proprietary (also called non-free) drivers (i.e., Nvidia, file format support, etc), are already included in the installation. On laptops, this can get weird with some of the laptop-specific hardware, but most of it works out of the box most of the time. Exceptions are old WinTel-era wireless and networking cards which needed a terrible driver wrapper, but has long since fallen out of favor. Thankfully!
Wait, so… I can copy everything as-is except /boot ?
~/boot is at the root of the drive. Your home folder should be in ~/home/username. THAT you can copy wholesale. I believe. Don’t take my word for it. Deja Dup can do it for you, as well, or the entire system.
Bit of pedantry, but
~/bootexpands to something like/home/username/boot./bootis a folder at the root of your filesystem, while~/bootis a directory in your home folder.Oh yes! Thanks for reminding me. The ~ is a shortcut to the active users home folder? Thanks!
And if I make a user with the same username on the other system, it’ll just… connect?
If you don’t encrypt the drive, yes. Some things you will have to reauthenticate, however, like your online accounts, but when those are reconnected everything should work as intended. That you should confirm, however. I don’t encrypt, though I should ;)
I don’t even know how-to, or what it truly means to encrypt, so I don’t have to worry about that. And I just love hearing the other parts. Thanks
I’m not an expert in Linux, but I’ve been using it for more than 20 years. I used to be plagued by this issue, but since online services have matured, I’ve got most of my stuff synced up. That, and my NAS and an external drive for backups. I do have a few thoughts, though.
One, I believe you can simply copy your /home directory and restore your OS settings by restoring it to a new install. This strikes me as a limiting option, as it doesn’t allow you to distro hop, at least not seamlessly. Also, get an external drive for backups. I use Deja Dup for simple, easy backups and restorations.
Two, I would suggest you investigate either Fedora (https://getfedora.org) or Pop_OS! (https://pop.system76.com/) as an alternative. Fedora is based on Red Hat, which is very mature, but strikes a nice balance between the latest software and reliability. Pop_OS! is Debian-based, which is also a very well matured OS, though System76 has made some major improvements. I use their Pop Shell extension for GNOME on Fedora 39 for window tiling, easily the best I’ve used on any Linux distro. Regardless, almost any other distro should be easier to get going over Arch. Sorry, Arch users ;)
Three, if you really don’t want to leave Arch, check out Manjaro. It’s Arch-based, but it’s quite a bit easier to set up.
Four, if you’d still like to try borking things, but without facing consequences, I’d set up a local VM using Boxes for GNOME or VirtualBox (https://www.virtualbox.org/). That way you can test stuff without risking your functional system. Boxes is better, IMO, since it can install distros from the app itself. The list has at least 100 distros of all types to choose from, including Haiku and FreeBSD. It would be good, however, if you have at least 16GBs of RAM, though I generally run VMs with 4GBs of RAM, Linux can run fine with 2GBs.
I hope that helps :)
tbh I bearly have experience in any distro, but Arch didn’t pose that much of a challange. I might switch, but I really don’t see the advantage I’d get. Maybe to Debian, I used it’s terminal. But, great Idea to mess around in VMs first!
Can you explain this step:
and restore your OS settings by restoring it to a new install.
I’m utterly useless with base arch 🤣 If it works for you, who’m I to complain 👍
I guess I should have made that clear. Your /home directory is where everything user-related is stored in invisible folders. All your settings for the OS and applications are kept in there. So, if you copy that directory and restore it to a fresh install of the same distro, all of your settings will be restored. It’s been years, but I’ve done it a few times.
The only thing you’ll really need to do after that is re-install all of the apps you installed. Once you have, however, every apps settings are restored.
wonderful, thank you for the info. I’ll have to sleep on this tho.
Definitely take your time and soak up more info from other sources :) I hope all of this turns out to be at least marginally helpful :D
Thank you. You wrote so much that I checked out your profile, I just wanna say: Enjoy your time on Lemmy! And that I am honored to have more then half of your all time comments be answers to me.
I have done something similar following this post - loads of others have created similar scripted installers for Arch for their specific use cases and this guide takes it one step further with custom arch meta packages that hold deps and system wide config.
You can also do similar things with tools like ansible or saltstack or similar tools. Though these all take the approach of define your configs and system to automate the setting up of a system approach rather than the backup or clone an existing system. So are more effort initially but are able to keep multiple system in sync with system configs with far less effort then trying to create a backup/restore system for organically created configs.
that wouldn’t work (I think) because my laptop has vastly different hardware
Should not matter, you can install all the packages all your system need - such as both nvidia, amd and intel graphics drivers and the kernel will only load the ones for the hardware you have booted with. Or if you really need different configs or packages for different systems the various approaches have ways to do that.
Do you know if there is a way to generate a ansible playbook based on your current deployment state?
Or do you need to painstakingly manually code every unique facet of your system into the playbook?
Most done with the latter. But the nice thing is once you have done it once it is much easier to keep things up to date and in sync from then on words. You can also peace meal it - setup one application at a time and migrate things one by one over to it.
painstakingly manually code every unique facet
That makes it sound a lot worst then it actually it. It is only a bit more effort then setting something up for the first time manually. And pays its self back many times over when you next need to reinstall or install a new system. Assuming you keep up with making changes to the code and not directly to your system each time.
This is basically the reason why I wanted to ask early. Two problems.
Already kinda late… And, idk how to configurate Firefox addons from the terminal. Even if I did, there’re a bunch of other apps too. I’d need to do so much research.
Supposedly you can configure Firefox add-ons via Nix’s home manager and NUR but you’ll probably still have to do a lot of research :P
Let’s do the Time-shift again!!!
