Skip to content

Add examples for BackupService and RestoreService #617

New issue
New issue
Open
Open
Add examples for BackupService and RestoreService#617
Assignees
trevorbonas
Labels
good first issueGood for newcomers

Description

@trevorbonas
trevorbonas
opened on Nov 27, 2023

Proposal:
Add examples demonstrating how to use the BackupService and RestoreService as they are both currently undocumented.

Current behavior:
Examples are missing apart from brief comments in the source code.

Desired behavior:
It would be nice to have examples demonstrating how to use both services, possibly in the examples directory or in the README. Personally, I would like to use BackupService and RestoreService to perform migrations but am unsure how to use these services.

Alternatives considered:
Other alternatives to the BackupService and RestoreService is using the Influx CLI with Python subprocess calls, but this is less testable and not preferred.

Use case:
Performing backups or restores using the InfluxDB Python client would be much easier with documentation.

Activity

  1. bednar commented on Nov 28, 2023

    @bednar
    bednar
    on Nov 28, 2023
    Contributor

    Hi @trevorbonas,

    Thanks for using our client.

    Is this something you would be willing to help with? All PR is welcome and we will be happy to review your submission.

    Regards

  2. bednar
    assigned
    trevorbonas
    on Nov 28, 2023
  3. bednar
    added
    good first issueGood for newcomers
    on Nov 28, 2023

  4. trevorbonas commented on Nov 28, 2023

    @trevorbonas
    trevorbonas
    on Nov 28, 2023
    Author

    @bednar I would love to but:

    • I don't know how to use these services correctly.

    • I get errors with BackupService:
      Platform: macOS Sonoma 14.1.1 Apple M1 Max.
      Python version: 3.11.6.
      influxdb-client-python version: 1.38.0.
      InfluxDB version: OSS 2.7.4.
      Reproducible code:

      from influxdb_client import InfluxDBClient
from influxdb_client.service import BackupService

# Using the Docker container setup with scripts/influxdb-restart.sh
# and imported data from the bird migration sample set
client = InfluxDBClient(url="http://localhost:8086", token="my-token",
    org="my-org")
metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()

      Error:

      Traceback (most recent call last):
File "/Users/trevor.bonas/reproduce.py", line 6, in <module>
metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/service/backup_service.py", line 182, in get_backup_metadata_with_http_info
return self.api_client.call_api(
       ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
return self.__call_api(resource_path, method,
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
response_data = self.request(
                ^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 365, in request
return self.rest_client.GET(url,
       ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 268, in GET
return self.request("GET", url,
       ^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 250, in request
r.data = r.data.decode('utf8')
         ^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 166: invalid continuation byte

      When the hardcoded value in influxdb_client/_sync/rest.py is changed from utf8 to latin1 then this error occurs:

      Traceback (most recent call last):
File "/Users/trevor.bonas/reproduce.py", line 6, in <module>
metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/service/backup_service.py", line 182, in get_backup_metadata_with_http_info
return self.api_client.call_api(
       ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 343, in call_api
return self.__call_api(resource_path, method,
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 185, in __call_api
return_data = self.deserialize(response_data, response_type)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 257, in deserialize
return self.__deserialize(data, response_type)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 296, in __deserialize
return self.__deserialize_model(data, klass)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 647, in __deserialize_model
instance = klass(**kwargs)
           ^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/domain/metadata_backup.py", line 51, in __init__
self.kv = kv
^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/domain/metadata_backup.py", line 72, in kv
raise ValueError("Invalid value for `kv`, must not be `None`")  # noqa: E501
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Invalid value for `kv`, must not be `None`

      Argument values for get_backup_metadata_with_http_info have no effect re: the errors.

    Let me know whether my usage is correct, whether you have any tips for getting BackupService working, or whether I should make a separate issue for these errors.

  5. lenoqt commented on Mar 13, 2024

    @lenoqt
    lenoqt
    on Mar 13, 2024

    @bednar I would love to but:

    • I don't know how to use these services correctly.

    • I get errors with BackupService:
      Platform: macOS Sonoma 14.1.1 Apple M1 Max.
      Python version: 3.11.6.
      influxdb-client-python version: 1.38.0.
      InfluxDB version: OSS 2.7.4.
      Reproducible code:

      from influxdb_client import InfluxDBClient
from influxdb_client.service import BackupService

# Using the Docker container setup with scripts/influxdb-restart.sh
# and imported data from the bird migration sample set
client = InfluxDBClient(url="http://localhost:8086", token="my-token",
    org="my-org")
metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()

      Error:

      Traceback (most recent call last):
File "/Users/trevor.bonas/reproduce.py", line 6, in <module>
metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/service/backup_service.py", line 182, in get_backup_metadata_with_http_info
return self.api_client.call_api(
       ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
return self.__call_api(resource_path, method,
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
response_data = self.request(
                ^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py", line 365, in request
return self.rest_client.GET(url,
       ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 268, in GET
return self.request("GET", url,
       ^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/influxdb_client/_sync/rest.py", line 250, in request
r.data = r.data.decode('utf8')
         ^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 166: invalid continuation byte

      When the hardcoded value in influxdb_client/_sync/rest.py is changed from utf8 to latin1 then this error occurs:

      Traceback (most recent call last):
File "/Users/trevor.bonas/reproduce.py", line 6, in <module>
metadata = BackupService(client.api_client).get_backup_metadata_with_http_info()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/service/backup_service.py", line 182, in get_backup_metadata_with_http_info
return self.api_client.call_api(
       ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 343, in call_api
return self.__call_api(resource_path, method,
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 185, in __call_api
return_data = self.deserialize(response_data, response_type)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 257, in deserialize
return self.__deserialize(data, response_type)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 296, in __deserialize
return self.__deserialize_model(data, klass)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/_sync/api_client.py", line 647, in __deserialize_model
instance = klass(**kwargs)
           ^^^^^^^^^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/domain/metadata_backup.py", line 51, in __init__
self.kv = kv
^^^^^^^
File "/Users/trevor.bonas/influxdb-client-python/influxdb_client/domain/metadata_backup.py", line 72, in kv
raise ValueError("Invalid value for `kv`, must not be `None`")  # noqa: E501
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Invalid value for `kv`, must not be `None`

      Argument values for get_backup_metadata_with_http_info have no effect re: the errors.

    Let me know whether my usage is correct, whether you have any tips for getting BackupService working, or whether I should make a separate issue for these errors.

    The library is badly outdated it seems, I am trying to restore a bucket and using RestoreService.post_restore_bucket_id returns the following error, and then further checking on my server, the endpoint that is pointing to this method, doesn't exists. The other methods for restoring or taking backups also don't work, I switched to the golang version and it looks that one is up to date.

    ---------------------------------------------------------------------------
ApiException                              Traceback (most recent call last)
Cell In[6], line 1
----> 1 restore_srv.post_restore_bucket_id("927b185ee3412511", "../../backup-bucket/")

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/service/restore_service.py:53, in RestoreService.post_restore_bucket_id(self, bucket_id, body, **kwargs)
     51     return self.post_restore_bucket_id_with_http_info(bucket_id, body, **kwargs)  # noqa: E501
     52 else:
---> 53     (data) = self.post_restore_bucket_id_with_http_info(bucket_id, body, **kwargs)  # noqa: E501
     54     return data

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/service/restore_service.py:76, in RestoreService.post_restore_bucket_id_with_http_info(self, bucket_id, body, **kwargs)
     57 """Overwrite storage metadata for a bucket with shard info from a backup..
     58 
     59 This method makes a synchronous HTTP request by default. To make an
   (...)
     71          returns the request thread.
     72 """  # noqa: E501
     73 local_var_params, path_params, query_params, header_params, body_params = \
     74     self._post_restore_bucket_id_prepare(bucket_id, body, **kwargs)  # noqa: E501
---> 76 return self.api_client.call_api(
     77     '/api/v2/restore/bucket/{bucketID}', 'POST',
     78     path_params,
     79     query_params,
     80     header_params,
     81     body=body_params,
     82     post_params=[],
     83     files={},
     84     response_type='str',  # noqa: E501
     85     auth_settings=[],
     86     async_req=local_var_params.get('async_req'),
     87     _return_http_data_only=local_var_params.get('_return_http_data_only'),  # noqa: E501
     88     _preload_content=local_var_params.get('_preload_content', True),
     89     _request_timeout=local_var_params.get('_request_timeout'),
     90     collection_formats={},
     91     urlopen_kw=kwargs.get('urlopen_kw', None))

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py:343, in ApiClient.call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, async_req, _return_http_data_only, collection_formats, _preload_content, _request_timeout, urlopen_kw)
    304 """Make the HTTP request (synchronous) and Return deserialized data.
    305 
    306 To make an async_req request, set the async_req parameter.
   (...)
    340     then the method will return the response directly.
    341 """
    342 if not async_req:
--> 343     return self.__call_api(resource_path, method,
    344                            path_params, query_params, header_params,
    345                            body, post_params, files,
    346                            response_type, auth_settings,
    347                            _return_http_data_only, collection_formats,
    348                            _preload_content, _request_timeout, urlopen_kw)
    349 else:
    350     thread = self.pool.apply_async(self.__call_api, (resource_path,
    351                                    method, path_params, query_params,
    352                                    header_params, body,
   (...)
    356                                    collection_formats,
    357                                    _preload_content, _request_timeout, urlopen_kw))

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py:173, in ApiClient.__call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, _return_http_data_only, collection_formats, _preload_content, _request_timeout, urlopen_kw)
    170 urlopen_kw = urlopen_kw or {}
    172 # perform request and return response
--> 173 response_data = self.request(
    174     method, url, query_params=query_params, headers=header_params,
    175     post_params=post_params, body=body,
    176     _preload_content=_preload_content,
    177     _request_timeout=_request_timeout, **urlopen_kw)
    179 self.last_response = response_data
    181 return_data = response_data

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/api_client.py:388, in ApiClient.request(self, method, url, query_params, headers, post_params, body, _preload_content, _request_timeout, **urlopen_kw)
    379     return self.rest_client.OPTIONS(url,
    380                                     query_params=query_params,
    381                                     headers=headers,
   (...)
    385                                     body=body,
    386                                     **urlopen_kw)
    387 elif method == "POST":
--> 388     return self.rest_client.POST(url,
    389                                  query_params=query_params,
    390                                  headers=headers,
    391                                  post_params=post_params,
    392                                  _preload_content=_preload_content,
    393                                  _request_timeout=_request_timeout,
    394                                  body=body,
    395                                  **urlopen_kw)
    396 elif method == "PUT":
    397     return self.rest_client.PUT(url,
    398                                 query_params=query_params,
    399                                 headers=headers,
   (...)
    403                                 body=body,
    404                                 **urlopen_kw)

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/rest.py:311, in RESTClientObject.POST(self, url, headers, query_params, post_params, body, _preload_content, _request_timeout, **urlopen_kw)
    308 def POST(self, url, headers=None, query_params=None, post_params=None,
    309          body=None, _preload_content=True, _request_timeout=None, **urlopen_kw):
    310     """Perform POST HTTP request."""
--> 311     return self.request("POST", url,
    312                         headers=headers,
    313                         query_params=query_params,
    314                         post_params=post_params,
    315                         _preload_content=_preload_content,
    316                         _request_timeout=_request_timeout,
    317                         body=body,
    318                         **urlopen_kw)

File ~/virtualenvs/kpi/env/lib/python3.11/site-packages/influxdb_client/_sync/rest.py:261, in RESTClientObject.request(self, method, url, query_params, headers, body, post_params, _preload_content, _request_timeout, **urlopen_kw)
    258     _BaseRESTClient.log_body(r.data, '<<<')
    260 if not 200 <= r.status <= 299:
--> 261     raise ApiException(http_resp=r)
    263 return r

ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json; charset=utf-8', 'X-Influxdb-Build': 'OSS', 'X-Influxdb-Version': 'v2.7.5', 'X-Platform-Error-Code': 'not found', 'Date': 'Wed, 13 Mar 2024 12:35:42 GMT', 'Content-Length': '47'})
HTTP response body: {"code":"not found","message":"path not found"}

    image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

good first issueGood for newcomers

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions