BED-9472: correlate organization teams with SCIM groups

README.md

@@ -48,10 +48,23 @@ A token with enterprise SCIM access is used to collect both `/scim/v2/enterprise
 
 Enterprise roles, including the built-in `members` role, are emitted through `GH_HasRole` and granular enterprise capability relationships. Only capability relationships with a confirmed privilege path are traversable; descriptive permissions such as `GH_WriteEnterpriseSso` remain non-traversable.
 
+### Optional privileged collection
+
+The collector runs successfully with the documented read-only permission set. Some higher-fidelity relationships require optional non-read-only permissions because GitHub exposes the supporting read APIs behind privileged permission names:
+
+- Organization `Members: write` enables external group mapping collection for IdP-synced teams, which allows the converter to emit `SCIM_Provisioned` edges from `SCIM_Group` to `GH_Team`. This is useful when the organization uses team synchronization with an external identity provider; if no GitHub teams are linked to external groups, this permission does not add graph data.
+- Classic PAT scope `manage_runners:enterprise` enables enterprise self-hosted runner group and runner collection. This is useful when the enterprise has enterprise-scoped runner groups or runners, especially runner groups shared into organizations; if all runners are organization- or repository-scoped, this scope does not add graph data.
+
+If these optional permissions are not granted, OpenHound skips the affected resources and continues collecting the rest of the GitHub environment.
+
 [![Python Version](https://img.shields.io/badge/Python-3.13-brightgreen.svg)](#about)
 
 ## Getting Started
 
-Follow the OpenHound docs to get started:
+Follow the OpenHound GitHub collector docs to get started:
 
-- [OpenHound Documentation](https://bloodhound.specterops.io/openhound/overview)
+- [GitHub collector overview](https://bloodhound.specterops.io/openhound/collectors/github/overview)
+- [Configure the collector](https://bloodhound.specterops.io/openhound/collectors/github/collect-data)
+- [Configure an organization GitHub App](https://bloodhound.specterops.io/openhound/collectors/github/configure-app)
+- [Configure an enterprise GitHub App](https://bloodhound.specterops.io/openhound/collectors/github/configure-enterprise-app)
+- [Configure a personal access token](https://bloodhound.specterops.io/openhound/collectors/github/configure-pat)

descriptions/edges/GH_MemberOf.md

@@ -1,3 +1,3 @@
 ## General Information
 
-The traversable GH_MemberOf edge represents team membership, linking a team role to its parent team or a child team to a parent team in nested team hierarchies. This edge is traversable because team membership extends access transitively -- a user who holds a role in a child team inherits the repository permissions of all ancestor teams in the nesting hierarchy, making it a key component of attack path analysis.
+The traversable GH_MemberOf edge represents team membership and projection relationships, linking a team role to its parent team, a child team to a parent team in nested team hierarchies, or a GH_EnterpriseTeam to its projected GH_Team in an organization. This edge is traversable because these relationships carry effective team membership context through the graph: a user who holds a role in a child team inherits the repository permissions of ancestor teams, and enterprise-managed team membership flows into the projected organization team.

extension/schema.json

@@ -467,7 +467,7 @@
     },
     {
       "name": "GH_MemberOf",
-      "description": "Team role is a member of a team, or team is a nested member of a parent team",
+      "description": "Team role is a member of a team, team is nested under a parent team, or enterprise team maps to its projected organization team",
       "is_traversable": true
     },
     {

src/openhound_github/github_rest_client.py

@@ -0,0 +1,3 @@
+from openhound_github.github_retry import should_retry_github_response
+
+__all__ = ["should_retry_github_response"]

src/openhound_github/github_retry.py

@@ -0,0 +1,48 @@
+from typing import Optional
+
+from requests import Response
+
+
+def _response_message(response: Response) -> str:
+    try:
+        response_data = response.json()
+    except ValueError:
+        return getattr(response, "text", "")
+    if isinstance(response_data, dict):
+        return str(response_data.get("message", ""))
+    return ""
+
+
+def is_primary_rate_limit_response(response: Response) -> bool:
+    if response.status_code not in (403, 429):
+        return False
+
+    message = _response_message(response).lower()
+    return (
+        response.headers.get("x-ratelimit-remaining") == "0"
+        or "api rate limit exceeded" in message
+    )
+
+
+def is_secondary_rate_limit_response(response: Response) -> bool:
+    if response.status_code not in (403, 429):
+        return False
+
+    message = _response_message(response).lower()
+    return "secondary rate limit" in message or "abuse detection" in message
+
+
+def should_retry_github_response(
+    response: Optional[Response], exception: Optional[BaseException]
+) -> bool:
+    if response is None:
+        return False
+
+    return (
+        is_primary_rate_limit_response(response)
+        or is_secondary_rate_limit_response(response)
+        or (
+            response.status_code in (403, 429)
+            and bool(response.headers.get("Retry-After"))
+        )
+    )

src/openhound_github/helpers.py

@@ -12,6 +12,10 @@
 from requests import Request
 
 from openhound_github.auth import GitHubAppInstallationAuth
+from openhound_github.github_retry import (
+    is_primary_rate_limit_response,
+    is_secondary_rate_limit_response,
+)
 
 logger = logging.getLogger(__name__)
 
@@ -239,18 +243,14 @@ def retry_policy(
         if response.status_code not in (403, 429):
             return False
 
-        message = _response_message(response).lower()
-        if (
-            headers.get("x-ratelimit-remaining") == "0"
-            or "api rate limit exceeded" in message
-        ):
+        if is_primary_rate_limit_response(response):
             reset_at = headers.get("x-ratelimit-reset")
             delay = int(reset_at) - now if reset_at else 0
             headers["Retry-After"] = str(delay)
             logger.warning("Primary rate limit reached, retrying in %s seconds", delay)
             return True
 
-        if "secondary rate limit" in message or "abuse detection" in message:
+        if is_secondary_rate_limit_response(response):
             logger.warning("Secondary rate limit reached, retrying in 60 seconds")
             headers["Retry-After"] = "60"
             return True

src/openhound_github/lookup.py

@@ -205,6 +205,42 @@ def projected_enterprise_team_id(self, org_login: str, slug: str) -> str | None:
             [org_login, slug],
         )
 
+    @lru_cache
+    def external_group_for_team(
+        self, org_login: str, team_database_id: int
+    ) -> tuple[int, str] | None:
+        row = self._find_single_row(
+            f"""
+            SELECT external_group_id, external_group_name
+            FROM {self.schema}.team_external_groups
+            WHERE org_login = ?
+              AND team_database_id = ?
+            """,
+            [org_login, team_database_id],
+        )
+        if row is None:
+            return None
+        return int(row[0]), str(row[1])
+
+    @lru_cache
+    def scim_group_id_for_team_external_group(
+        self, org_login: str, external_group_name: str
+    ) -> str | None:
+        rows = self._find_all_objects(
+            f"""
+            SELECT DISTINCT esg.id
+            FROM {self.schema}.enterprise_scim_groups esg
+            JOIN {self.schema}.enterprise_organizations eo
+              ON eo.enterprise_node_id = esg.enterprise_node_id
+            WHERE eo.login = ?
+              AND esg.display_name = ?
+            """,
+            [org_login, external_group_name],
+        )
+        if not rows or len(rows) != 1:
+            return None
+        return str(rows[0][0])
+
     @lru_cache
     def external_identity_id_for_guid(
         self, guid: str, environment_slug: str

src/openhound_github/main.py

@@ -61,12 +61,14 @@ def preproc(ctx: PreProcContext):
         "repo_roles": "repo_roles",
         "users": "users",
         "teams": "teams",
+        "team_external_groups": "team_external_groups",
         "team_members": "team_members",
         "saml_provider": "saml_provider",
         "external_identities": "external_identities",
         "applications": "applications",
         "enterprise": "enterprise",
         "enterprise_organizations": "enterprise_organizations",
+        "enterprise_scim_groups": "enterprise_scim_groups",
         "enterprise_runner_groups": "enterprise_runner_groups",
         "enterprise_runner_group_organizations": "enterprise_runner_group_organizations",
         "enterprise_runner_group_memberships": "enterprise_runner_group_memberships",

src/openhound_github/models/__init__.py

@@ -62,7 +62,7 @@
     ScimUser,
 )
 from .secret_scanning_alert import SecretScanningAlert
-from .team import Team
+from .team import Team, TeamExternalGroup
 from .team_member import TeamMember
 from .team_role import TeamRole
 from .user import User
@@ -78,6 +78,7 @@
     "ActionPermission",
     "User",
     "Team",
+    "TeamExternalGroup",
     "TeamRole",
     "TeamMember",
     "Repository",

src/openhound_github/models/team.py

@@ -15,14 +15,16 @@
 @dataclass
 class GHTeamProperties(GHNodeProperties):
     """Team-specific properties and accordion panel queries.
-    
+
     Attributes:
         collected: Collected/generated by OpenHound
         github_team_id: The raw GitHub team node ID.
         slug: The team's URL-safe slug identifier.
         description: The team's description.
         privacy: The team's privacy level (e.g., `visible`, `secret`).
         type: The team type, such as `enterprise` for projected enterprise teams.
+        external_group_id: GitHub's org-scoped external IdP group identifier.
+        external_group_name: The external IdP group name reported by GitHub.
         environment_name: The name of the environment (GitHub organization).
         query_first_degree_members: Query for first degree members.
         query_unrolled_members: Query for unrolled members.
@@ -40,6 +42,8 @@ class GHTeamProperties(GHNodeProperties):
     description: str | None = None
     privacy: str | None = None
     type: str | None = None
+    external_group_id: int | None = None
+    external_group_name: str | None = None
     environment_name: str | None = None
 
     query_first_degree_members: str | None = None
@@ -74,6 +78,16 @@ class ParentTeam(BaseModel):
     id: str
 
 
+class TeamExternalGroup(BaseModel):
+    """Raw mapping between an org team and its linked external IdP group."""
+
+    org_login: str
+    team_database_id: int
+    external_group_id: int
+    external_group_name: str
+    external_group_updated_at: str | None = None
+
+
 @app.asset(
     node=NodeDef(
         kind=nk.TEAM,
@@ -89,6 +103,13 @@ class ParentTeam(BaseModel):
             description="Team is a child of parent team",
             traversable=False,
         ),
+        EdgeDef(
+            start=nk.SCIM_GROUP,
+            end=nk.TEAM,
+            kind=ek.SCIM_PROVISIONED,
+            description="SCIM group is provisioned as team",
+            traversable=True,
+        ),
     ],
 )
 class Team(BaseAsset):
@@ -120,9 +141,16 @@ def node_id(self) -> str:
         """The ID from a GraphQL API response is the same as a regular node_id"""
         return self.id
 
+    @property
+    def external_group(self) -> tuple[int, str] | None:
+        if self.database_id is None:
+            return None
+        return self._lookup.external_group_for_team(self.org_login, self.database_id)
+
     @property
     def as_node(self) -> GHNode:
         tid = self.node_id
+        external_group = self.external_group
         return GHNode(
             kinds=[nk.TEAM],
             properties=GHTeamProperties(
@@ -132,6 +160,8 @@ def as_node(self) -> GHNode:
                 slug=self.slug,
                 description=self.description,
                 privacy=self.privacy,
+                external_group_id=external_group[0] if external_group else None,
+                external_group_name=external_group[1] if external_group else None,
                 environment_name=self.org_login,
                 environmentid=self.org_node_id,
                 query_first_degree_members=f"MATCH p=(:GH_User)-[:GH_HasRole]->(t:GH_TeamRole)-[:GH_MemberOf]->(:GH_Team {{node_id:'{tid}'}}) RETURN p",
@@ -176,6 +206,19 @@ def edges(self):
                 properties=EdgeProperties(traversable=True),
             )
 
+        external_group = self.external_group
+        if external_group:
+            scim_group_id = self._lookup.scim_group_id_for_team_external_group(
+                self.org_login, external_group[1]
+            )
+            if scim_group_id:
+                yield Edge(
+                    kind=ek.SCIM_PROVISIONED,
+                    start=EdgePath(value=scim_group_id, match_by="id"),
+                    end=EdgePath(value=self.node_id, match_by="id"),
+                    properties=EdgeProperties(traversable=True),
+                )
+
         # yield from self._branch_edges
         yield from self._bypass_pull_request_allowances_edges
         yield from self._bypass_push_allowances_edges