[Confluence] confluence.remove_page() method may not remove all child pages.

[hograthm]

Sometimes methond confluence.remove_page() with parameter recursive=True can't delete all child pages.
Instead it will delete only fixed number of child pages and delete a root page. All remaining child pages will be moved to the root of the space.
This is possible because inside confluence.remove_page() we use confluence.get_page_child_by_type()

atlassian-python-api/atlassian/confluence.py

Lines 60 to 69 in 523057c

	def get_page_child_by_type(self, page_id, type="page", start=None, limit=None, expand=None):
	"""
	Provide content by type (page, blog, comment)
	:param page_id: A string containing the id of the type content container.
	:param type:
	:param start: OPTIONAL: The start point of the collection to return. Default: None (0).
	:param limit: OPTIONAL: how many items should be returned after the start index. Default: Site limit 200.
	:param expand: OPTIONAL: expand e.g. history
	:return:
	"""

By default limit=None, but In my case it was 25 by default. This led to an incorrect amount of executions here:

atlassian-python-api/atlassian/confluence.py

Lines 579 to 582 in 523057c

	if recursive:
	children_pages = self.get_page_child_by_type(page_id)
	for children_page in children_pages:
	self.remove_page(children_page.get("id"), status, recursive)

At the moment, I still have not trace what forces limit to take default value 25. Maybe this is a bug.

[Spacetown]

There is also a limit on the server side. The user can't increase this.

[hograthm]

@Spacetown I was able to increase it manually from 25 to 200 without any error messages from server side. That's why I think 25 is hard coded somewhere. As I see this - we need some function to get this server-side limit and apply it as default one, but I'm not sure if Confluence have something like this. Will check it and then return with my research here.

[Spacetown]

In my opinion it was wrong to add the arguments start=None, limit=None to the function The paged APIS need to be called in a special way and this should be handled by the module and not by the user.

See also #935 where a normal getwas replaced by a _get_paged which is doing the paging stuff and yields each item to the user.

atlassian-python-api/atlassian/bitbucket/__init__.py

Lines 2333 to 2343 in 523057c

	def all_branches_permissions(self, project_key, permission_id, repository_slug=None):
	"""
	Get branches permissions from a given repo
	:param project_key:
	:param permission_id
	:param repository_slug:
	:return:
	"""
	url = self._url_branches_permissions(project_key, permission_id, repository_slug)
	return self._get_paged(url)

atlassian-python-api/atlassian/bitbucket/base.py

Lines 41 to 81 in 523057c

	def _get_paged(self, url, params=None, data=None, flags=None, trailing=None, absolute=False):
	"""
	Used to get the paged data
	:param url: string: The url to retrieve
	:param params: dict (default is None): The parameters
	:param data: dict (default is None): The data
	:param flags: string[] (default is None): The flags
	:param trailing: bool (default is None): If True, a trailing slash is added to the url
	:param absolute: bool (default is False): If True, the url is used absolute and not relative to the root
	:return: A generator object for the data elements
	"""
	if params is None:
	params = {}
	while True:
	response = self.get(url, trailing=trailing, params=params, data=data, flags=flags, absolute=absolute)
	if "values" not in response:
	return
	for value in response.get("values", []):
	yield value
	if self.cloud:
	url = response.get("next")
	if url is None:
	break
	# From now on we have absolute URLs with parameters
	absolute = True
	# Params are now provided by the url
	params = {}
	# Trailing should not be added as it is already part of the url
	trailing = False
	else:
	if response.get("nextPageStart") is None:
	break
	params["start"] = response.get("nextPageStart")
	return

[BlueFireTV]

@hograthm

I was able to increase it manually from 25 to 200 without any error messages from server side. That's why I think 25 is hard coded somewhere. As I see this - we need some function to get this server-side limit and apply it as default one, but I'm not sure if Confluence have something like this. Will check it and then return with my research here.

How did u increase it?

I tried it with a overriding the Method:

# delete old pages recursively
def remove_page_recursive(conf, page_id, status=None, recursive=True):
        """
        This method removes a page, if it has recursive flag, method removes including child pages
        :param page_id:
        :param status: OPTIONAL: type of page
        :param recursive: OPTIONAL: if True - will recursively delete all children pages too
        :return: 
        """
        url = "rest/api/content/{page_id}".format(page_id=page_id)
        if recursive:
            children_pages = conf.get_page_child_by_type(page_id=page_id,  type="page", start=None, limit=200, expand=None)
            for children_page in children_pages:
                conf.remove_page(children_page.get("id"), status, recursive)
        params = {}
        if status:
            params["status"] = status

        try:
            response = conf.delete(url, params=params)
        except HTTPError as e:
            if e.response.status_code == 404:
                # Raise ApiError as the documented reason is ambiguous
                raise ApiError(
                    "There is no content with the given id, or the calling "
                    "user does not have permission to trash or purge the content",
                    reason=e,
                )
            if e.response.status_code == 409:
                raise ApiConflictError(
                    "There is a stale data object conflict when trying to delete a draft",
                    reason=e,
                )

            raise

        return response

print("Page exists?")
if conf.page_exists(page_space, page_title) == True:
    # resolve page ID
    print("Get PageID")
    page_id = conf.get_page_id(page_space, page_title)

    # This method removes a page, if it has recursive flag, method removes including child pages
    print("Delete old Page")
    remove_page_recursive(conf, page_id, status=None, recursive=True)

But i didn't work :(

[Spacetown]

@Spacetown I was able to increase it manually from 25 to 200 without any error messages from server side. That's why I think 25 is hard coded somewhere. As I see this - we need some function to get this server-side limit and apply it as default one, but I'm not sure if Confluence have something like this. Will check it and then return with my research here.

This doesn‘t work because also the server has a limit. You need to do paging depending on response and there we can live with the default of 25.

[BlueFireTV]

I got it to work 😊

# delete old pages recursively
def remove_page_recursive(conf, page_id, status=None, recursive=True):
        """
        This method removes a page, if it has recursive flag, method removes including child pages
        :param page_id:
        :param status: OPTIONAL: type of page
        :param recursive: OPTIONAL: if True - will recursively delete all children pages too
        :return: 
        """
        url = "rest/api/content/{page_id}".format(page_id=page_id)
        if recursive:
            children_pages = conf.get_page_child_by_type(page_id=page_id,  type="page", start=None, limit=200, expand=None)
            for children_page in children_pages:
                remove_page_recursive(conf, children_page.get("id"), status, recursive)
        params = {}
        if status:
            params["status"] = status

        try:
            response = conf.delete(url, params=params)
        except HTTPError as e:
            if e.response.status_code == 404:
                # Raise ApiError as the documented reason is ambiguous
                raise ApiError(
                    "There is no content with the given id, or the calling "
                    "user does not have permission to trash or purge the content",
                    reason=e,
                )
            if e.response.status_code == 409:
                raise ApiConflictError(
                    "There is a stale data object conflict when trying to delete a draft",
                    reason=e,
                )

            raise

        return response

[Spacetown]

But not with more than 200 child pages. You need to implement paging instead of increasing the size. BTW: The max size can be less, it depends on the server configuration.

[BlueFireTV]

Possible but it worked for me / is working for me for now. 😊

but thanks

[gonchik]

Added like this and document it
d0edd6b