PowerBI Purview Extract/Filter/Ingest

Possible to get an example of how to extract the powerbi information from one  purview and filter it for just a list of workspaces ( i can get a list from aad hopefully to see which one's the msi has been added to as a viewer in the groups that have access to those workspaces) and then upload that into a new purview and setup relationships for the elements that are there and used by the reports?
I suppose even just extract and insert would be sufficient.  I can try to work on the filter/relationships if that would be helpful once i can see the format of the data and what it would look like.
Thanks,
Jonathon

HI, @mathjon248 - I'm a little lost in what I can do to help you out here / what you're asking for.
The recommendation from the product team for migrating between Purview accounts is to re-scan.  That guarantees that you will get all connections / relationship attributes which might be missed if you rely on search.
If you really want to go down the path of searching and extracting, you would follow these rough steps:

Determine the type at the top of the entities you want to search (e.g. use client.get_entity(guid="...") on a power bi workspace).
Search with a typeName filter of whatever the type you're looking for (see this example sample ).  Please note that the Search API is subject to change before going generally available.
Iterate over the search results and save the guids / ids returned.
Query for those guids and then collected the referred entities as well.
Make sure you traverse the relationship attributes until there are no more attributes left to discover.

That is great info. Since the PowerBI scan is only allowing scanning for an
entire tenant at the moment. I want to select certain workspaces after
extraction from a main purview where the tenant admins have access to scan.
The workspaces where a user/group have access should be the selected
workspaces to send to the purview owned by the data owners of those groups
who have access to those workspaces/data. So, each additional purview would
only contain the data elements that are owned by that group (sql dbs, data
lakes, etc) but not the entire tenant. Then we can set up the relationships
from those databases or data lake resource sets to the reports. So, it is
like a hub and spoke. The main purview has tenant access but no data access
otherwise and the spoke purviews have database access but no tenant api
access to PowerBI because of separation of responsibilities and zero trust
and least access policies.  Policies are company wide. The ultimate win is
to the get the intersection of the workspaces with access granted to the
groups having data source and owner roles for purview.  But that maybe a
big ask since we have to use AD Groups / roles to find that for any given
spoke purviews.  It would be easier if the hub purview has access to
everything and then we could filter just on workspaces. But of course that
would be easier once purview allows the user to select the workspaces to
scan.   I suppose then the in the original scenario where the hub workspace
needs a way to either delete in purview the extraneous workspaces and then
export to the spoke workspaces or a way to filter down the exported file
after export before sending to the spoke.
On Sun, Apr 18, 2021, 11:19 PM Will Johnson @.***>
wrote:

HI, @mathjon248 https://github.com/mathjon248 - I'm a little lost in
what I can do to help you out here / what you're asking for.
The recommendation from the product team for migrating between Purview
accounts is to re-scan. That guarantees that you will get all connections /
relationship attributes which might be missed if you rely on search.
If you really want to go down the path of searching and extracting, you
would follow these rough steps:

Determine the type at the top of the entities you want to search
(e.g. use client.get_entity(guid="...") on a power bi workspace).
Search with a typeName filter of whatever the type you're looking
for (see this example sample
https://github.com/wjohnson/pyapacheatlas/blob/master/samples/migratePurview/migrate_util.py#L14
). Please note that the Search API is subject to change before going
generally available.
Iterate over the search results and save the guids / ids returned.
Query for those guids and then collected the referred entities as
well.
Make sure you traverse the relationship attributes until there are
no more attributes left to discover.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wjohnson/pyapacheatlas/issues/112#issuecomment-822138467,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEMBIQJK34PZB5HXYCDMOALTJOOL5ANCNFSM43CLWCBA
.

did you have anything where it could be scanned exported imported and filtered somehow along the way by workspace?

@mathjon248 What you've described is definitely a complicated workload.
The steps I mentioned above are the process you'd have to go through but I don't have one single sample that you could modify to accomplish this.  I would encourage you to work with your microsoft reps to help prioritize the "Select Workspace" feature that you need to accomplish this.  Otherwise, you'll end up writing a LOT of code to accomplish this.

Thank you for your response and everything you have done. I really needed
all the help I can get and you have done more than I have required as I use
Databricks and needed to bulk manage more directly with the guidance and
code you provide.
On Fri, May 7, 2021, 5:12 PM Will Johnson @.***> wrote:

Ah. I've described the steps to accomplish this and I don't have the
bandwidth at this point in time to write it for you. See this comment
https://github.com/wjohnson/pyapacheatlas/issues/112#issuecomment-822138467
for the steps.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/wjohnson/pyapacheatlas/issues/112#issuecomment-834783046,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEMBIQNENODBFSZPBG5KXPTTMRJSDANCNFSM43CLWCBA
.

I'm going to close it since it's been over a month but let me know if you have any specific requirements you'd like to see in the package.