Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ borg-import converts backups made with other backup software into the format use

See ``borg-import -h`` for more information.

Potential advantages over manually doing it
Potential advantages over doing it manually
===========================================

Note: we have different importers and some importers may not support all the features.
Note: There are different importers, and some may not support all features.

- automation: less manual work, import lots of backups into a borg repo with one command
- automatically makes up borg archive name from what you give + discovered timestamp
- sets borg archive creation timestamp to the historically correct date/time
- temporarily moves the source directory so the borg files cache will speed up borg create
- Automation: less manual work; import many backups into a Borg repository with one command.
- Automatically constructs the Borg archive name from the provided input and the discovered timestamp.
- Sets the Borg archive creation timestamp to the correct historical date and time.
- Temporarily moves the source directory so Borg's files cache can speed up ``borg create``.

Currently supported import formats
==================================
Expand All @@ -22,8 +22,8 @@ Currently supported import formats
--------------------------------------------------

Imports archives from an existing Borg repository into a new one.
This is useful when a Borg repository needs to be rebuilt (e.g. if
your borg key and passphrase was compromised).
This is useful when a Borg repository needs to be rebuilt (e.g., if
your Borg key and passphrase were compromised).

Usage: ``borg-import borg SOURCE_REPOSITORY DESTINATION_REPOSITORY``

Expand All @@ -48,7 +48,7 @@ See ``borg-import rsynchl -h`` for help.
`rsync-time-backup <https://github.com/laurent22/rsync-time-backup>`_
---------------------------------------------------------------------

Similar to `rsynchl`, except with timestamp extraction optimized for `rsync-time-backup` folder names.
Similar to ``rsynchl``, except with timestamp extraction optimized for ``rsync-time-backup`` folder names.

Usage: ``borg-import rsync_tmbackup --prefix=foo- RSYNC_ROOT BORG_REPOSITORY``

Expand All @@ -58,7 +58,7 @@ Backup tools based on rsync with hard links
-------------------------------------------

borg-import should, in principle, be able to import backups from any backup tool that is
based on rsync with hard links. This requires that the tool matches the assumptions listed above for simple
based on rsync with hard links, provided the tool matches the assumptions listed above for simple
rsync.

* `backintime <https://github.com/bit-team/backintime>`_
Expand Down
6 changes: 3 additions & 3 deletions docs/authors.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. include:: global.rst.inc

=================
Authors & License
=================
===================
Authors and License
===================

.. include:: ../AUTHORS

Expand Down
4 changes: 2 additions & 2 deletions docs/changes.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Changelog
=========

Version 0.1 (not released yet)
------------------------------
Version 0.1 (unreleased)
------------------------

Bug fixes:

Expand Down
14 changes: 7 additions & 7 deletions docs/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@
.. highlight:: none
.. _faq:

Frequently asked questions
Frequently Asked Questions
==========================

My old backup program isn't officially supported. Can I still use borg-import?
------------------------------------------------------------------------------

Chances are good that your old backup program is using some
kind of rsync+hardlinks mechanisms that will work with
Borg-Import's ``rsynchl`` mode. If you're unsure, try to find
out or ask how the program is organizing its backups.
Support for the software `Back In Time`_ was discovered by
simply trying it out.
It is likely that your old backup program uses an
rsync-with-hard-links mechanism that works with
borg-import's ``rsynchl`` mode. If you're unsure, try to find
out or ask how the program organizes its backups.
Support for `Back In Time`_ was discovered simply by
trying it.
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.. include:: global.rst.inc

Borg-Import Documentation
borg-import Documentation
=========================

.. include:: ../README.rst
Expand Down
15 changes: 7 additions & 8 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,28 +5,27 @@
Installation
============

To install Borg-Import, you need `Python 3`_ and `pip`_ installed. You can then either `download borg-import directly
To install borg-import, you need `Python 3`_ and `pip`_ installed. You can then either `download borg-import directly
<https://github.com/borgbackup/borg-import/archive/master.zip>`_
or clone it from the `githubrepo`_.
or clone it from GitHub_.
Extract the downloaded .zip if necessary.
Open a terminal in the borg-import directory and execute the following to install the program via pip:
:code:`pip install --user -e .`
Open a terminal in the borg-import directory and execute the following to install the program via pip: ``pip install --user -e .``

If you have */home/user/.local/bin/* in your ``PATH`` variable, you can then start using Borg-Import.
If you have */home/user/.local/bin/* in your ``PATH`` variable, you can then start using borg-import.
Otherwise, you will need to add *.local/bin/* to your ``PATH``.

For Developers
--------------

If you're planning to contribute to Borg-Import, you should set up the development environment:
If you're planning to contribute to borg-import, you should set up the development environment:

1. Install development dependencies:

:code:`pip install -r requirements.d/development.txt`
``pip install -r requirements.d/development.txt``

2. Set up pre-commit hooks:

:code:`pre-commit install`
``pre-commit install``

This will automatically run code formatting (black) and linting (flake8) checks before each commit.
The pre-commit hooks will ensure your code follows the project's style guidelines.
6 changes: 3 additions & 3 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
.. highlight:: bash
.. _quickstart:

Quick Start
===========
Quickstart
==========

This chapter will get you started with Borg-Import.
This guide will get you started with borg-import.
18 changes: 8 additions & 10 deletions docs/support.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,35 +4,33 @@
Support
=======

Please first read the docs, the existing issue tracker issues and mailing
list posts -- a lot of stuff is already documented / explained / discussed /
filed there.
Please read the documentation, existing issue-tracker issues, and mailing list posts first—much is already documented, explained, discussed, or filed there.

Issue Tracker
-------------

If you've found a bug or have a concrete feature request, please create a new
ticket on the project's `issue tracker`_.

For more general questions or discussions, IRC or mailing list are preferred.
For more general questions or discussions, IRC or the mailing list is preferred.

Chat (IRC)
----------
Join us on channel #borgbackup on chat.freenode.net.

As usual on IRC, just ask or tell directly and then patiently wait for replies.
As is usual on IRC, ask your question and then wait patiently for replies.
Stay connected.

You could use the following link (after connecting, you can change the random
nickname you get by typing "/nick mydesirednickname"):
You can use the following link (after connecting, you can change the random
nickname you get by typing ``/nick mydesirednickname``):

http://webchat.freenode.net/?randomnick=1&channels=%23borgbackup&uio=MTY9dHJ1ZSY5PXRydWUa8


Mailing list
Mailing List
------------

To find out about the mailing list, its topic, how to subscribe, how to
unsubscribe and where you can find the archives of the list, see the
To learn about the mailing list, its topic, how to subscribe, how to
unsubscribe, and where to find the list archives, see the
`mailing list homepage
<https://mail.python.org/mailman/listinfo/borgbackup>`_.
13 changes: 7 additions & 6 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,17 @@ Usage

``borg-import`` consists of a number of commands, one for each
backup system supported. Each accepts a number of arguments and
options. The following sections will describe each in detail.
options. The following sections describe each in detail.

General
-------

Note that Borg-Import will ask you for your repo's passphrase
which blocks the import until you enter it. To let Borg-Import
continue automatically, you can pass the environment variable
*BORG_PASSPHRASE*:
:code:`BORG_PASSPHRASE=xxxxxx borg-import ....`
Note that borg-import will prompt for your repository passphrase,
which pauses the import until you enter it. To let borg-import
continue automatically, set the environment variable
BORG_PASSPHRASE:

``BORG_PASSPHRASE=xxxxxx borg-import ...``

.. _rsnapshot:

Expand Down
4 changes: 2 additions & 2 deletions src/borg_import/borg.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@


def get_borg_archives(repository):
"""Get all archive metadata discovered in the Borg repository."""
# Get list of archives with their timestamps
"""Return metadata for all archives discovered in the Borg repository."""
# Get a list of archives with their timestamps
borg_cmdline = ["borg", "list", "--format", "{name}{TAB}{time}{NL}", repository]
output = subprocess.check_output(borg_cmdline).decode()

Expand Down
3 changes: 2 additions & 1 deletion src/borg_import/helpers/discover.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

def discover(root, depth):
"""
recurse starting from <root> path and yield relative dir paths with wanted <depth>.
Recurse from the given root path and yield relative directory paths at the specified depth.
"""

def _discover(root, current_dir, current_depth, wanted_depth):
Expand All @@ -23,6 +23,7 @@ def _discover(root, current_dir, current_depth, wanted_depth):


def parser(rel_path, regex):
"""Parse rel_path with regex and return a dict of named groups, or None if no match."""
m = re.match(regex, rel_path)
if m:
return m.groupdict()
14 changes: 7 additions & 7 deletions src/borg_import/helpers/names.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,17 @@

def make_name(*args, dt_format="%Y-%m-%dT%H:%M:%S"):
"""
assemble borg archive names from components.
Assemble Borg archive names from components.

args can be anything that converts to str, plus:
Arguments can be anything that converts to str, plus:

- bytes objects, which are safely decoded
- datetime objects, which are formatted using dt_format

invalid chars are replaced or removed, so the resulting archive name
Invalid characters are replaced or removed, so the resulting archive name
should be valid.

E.g.:
Examples:
make_name(b'hostname', b'üser', 1)
-> 'hostname-üser-1'

Expand All @@ -28,12 +28,12 @@ def make_name(*args, dt_format="%Y-%m-%dT%H:%M:%S"):
s = arg.strftime(dt_format)
else:
s = str(arg)
# we don't want to have blanks for practical shell-usage reasons:
# Avoid spaces for practical shell-usage reasons:
s = s.replace(" ", "_")
# the slash is not allowed in archive names
# A slash is not allowed in archive names
# (archive name = FUSE directory name)
s = s.replace("/", "!")
# :: is repo::archive separator, not allowed in archive names
# "::" is the repo::archive separator, not allowed in archive names
s = s.replace("::", ":")
components.append(s)
return "-".join(components)
2 changes: 1 addition & 1 deletion src/borg_import/helpers/testsuite/test_names.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@


def test_make_name():
# str (some with invalid/unwanted chars)
# str (some with invalid/unwanted characters)
assert make_name("backup name") == "backup_name"
assert make_name("backup/name") == "backup!name"
assert make_name("backup::name") == "backup:name"
Expand Down
6 changes: 3 additions & 3 deletions src/borg_import/helpers/testsuite/test_timestamps.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ def test_datetime_from_string():
dfs = datetime_from_string("1999-12-31T23:59:59")
dt_trg = datetime(1999, 12, 31, 23, 59, 59).astimezone(tz=timezone.utc)
assert dfs == dt_trg
# Of course, two datetimes can be equal in different timezones. Make
# sure the timezone info matches UTC, which borg itself expects.
# Two datetimes can be equal in different time zones. Make
# sure the time zone info matches UTC, which Borg itself expects.
assert dfs.tzinfo == dt_trg.tzinfo == timezone.utc

# FIXME: When this format is passed to datetime_from_string, the internal
Expand All @@ -34,7 +34,7 @@ def test_datetime_from_string():
assert dfs == dt_trg
assert dfs.tzinfo == dt_trg.tzinfo == timezone.utc

# rsync-time-backup format.
# rsync-time-backup format:
dfs = datetime_from_string("2022-12-21-063019")
dt_trg = datetime(2022, 12, 21, 6, 30, 19).astimezone(tz=timezone.utc)
assert dfs == dt_trg
Expand Down
31 changes: 15 additions & 16 deletions src/borg_import/helpers/timestamps.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,24 @@

def datetime_from_mtime(path):
"""
discover backup timestamp from a (single) filesystem object.
Discover the backup timestamp from a (single) filesystem object.

e.g. from a root/TIMESTAMP file created by "touch" at backup time,
For example, from a root/TIMESTAMP file created by "touch" at backup time,
or from root/ (assuming that the directory timestamp was modified
at backup time).
"""
t = path.stat().st_mtime
# Borg needs tz-aware timestamps in UTC timezone.
# Borg needs timezone-aware timestamps in UTC.
return datetime.fromtimestamp(t, tz=timezone.utc)


def datetime_from_string(s):
"""
parse datetime from a string
Parse a datetime from a string.

returns a tz-aware datetime object in UTC timezone if the format could be
parsed.
Return a timezone-aware datetime object in UTC if the format could be parsed.

raises ValueError if not.
Raise ValueError otherwise.
"""
s = s.strip()
for ts_format in [
Expand All @@ -40,36 +39,36 @@ def datetime_from_string(s):
]:
try:
if ts_format in ("%a %b %d %H:%M:%S %Z %Y",) and "UTC" in s:
# %Z returns a naive datetime, despite a timezone being specified.
# %Z returns a naive datetime, despite a time zone being specified.
# However, strptime %Z only tends to work on local times and
# UTC.
#
# Per astimezone docs:
# If self is naive, it is presumed to represent time in the
# system timezone.
# system time zone.
#
# If we had a UTC timezone, prevent conversion to aware
# datetime from assuming a local timezone before conversion
# If we had a UTC time zone, prevent conversion to an aware
# datetime from assuming a local time zone before conversion
# to UTC.
return datetime.strptime(s, ts_format).replace(tzinfo=timezone.utc)
else:
# If "UTC" wasn't specified using the above ts_format, assume
# the timezone specified was local and hope for the best.
# the time zone specified was local and hope for the best.
# This handles all other ts_formats as well, which are assumed
# to be local since they don't carry timezone.
# to be local since they don't carry a time zone.
return datetime.strptime(s, ts_format).astimezone(tz=timezone.utc)
except ValueError:
# didn't work with this format, try next
# Didn't work with this format; try the next.
pass
else:
raise ValueError("could not parse %r" % s)


def datetime_from_file(path):
"""
discover backup timestamp from contents of a file.
Discover the backup timestamp from the contents of a file.

e.g. root/TIMESTAMP contains: Mon Oct 31 23:35:50 CET 2016
For example, root/TIMESTAMP contains: Mon Oct 31 23:35:50 CET 2016
"""
with path.open() as f:
ts = f.readline().strip()
Expand Down
Loading