Feature request: group modules in visualizaton using project structure

Hello and thanks for a great library!
Summary
Context
Our project is using approach similiar to Feature Sliced Design, where code is divided into layers of some entities, all of which have some sort of public API
Example:
/src
 /features <-- some "layer" of the project architecture
  /feature-a
   index.ts <-- public API of the feature
   lib.ts
   utils.ts
   etc.ts <-- various internals, that are hidden behind public API
  /feature-b
   index.ts

Problem
But if i run skott against project entrypoint or any of the features index file - the generated visualization is very noisy, because all of the internal modules are exposed right away
Suggestion
Since all of these features, widgets, etc are essentialy "groups" of modules behind single entrypoint with a public API, i suggest to add an option to provide glob-like pattern, which would describe them:
const { ... } = await skott({
  entrypoint: "src/index.ts",
  groups: {
   features: "src/features/{name}/*",
   widgets: "src/widgets/{name}/*",
  }

The visualization then can use these groups, e.g. to skip the render of each and every internal module of src/features/feature-a and render it as the single features/feature-a node
This should make the visualization:

much easier to read, since internal modules are hidden
faster to render, since there would be less graph nodes to handle

Details
This feature would also allow to use skott with bigger projects.
For example our project now has about 7000 modules in it and currently skott just can't handle it in any way, since web-app graph takes forever to load and svg export shows just something like "Maximum size exceeded"
Visualization does work, if i choose an src/widgets/{name}/index.ts as an entrypoint, but result is still not really useful because of all of the internals being exposed:

At the same time our project has only about a 200-300 such "services", "features" and "widgets", which graph rendering should handle much easier

Standard questions
Please answer these questions to help us investigate your issue more quickly:

Question
Answer

Would you consider contributing a PR?
Yes

Hello @AlexandrHoroshih, thanks for opening that very detailed issue!
This reminds me #33 that was initially thought by @pigoz, that is the compact feature being more or less what you describe there.
Overall, I do agree that skott web application usefulness is quickly limited when the number of nodes grows hence it should provide ways to seemlessly group some of the nodes into namespaces, with custom rules that would fit based on the project size and structure.
Just for you own information, note that there is already a way to do some sort of grouping using the API, by providing a custom dependency resolver, the default one being EcmaScriptDependencyResolver (resolving all JS/TS modules, one node = one module). The API allows you to hook into skott module and internal graph resolution. This is a feature that I implemented for Rush.js, a popular monorepo tool. You can see an example here of a custom resolver I wrote for it there: https://github.com/antoine-coulon/krush/blob/af6d8e661deb300f93733f1e9ef1b4c839ccb309/plugins/rush-plugin-skott/src/plugins/visualize/main.ts#L27
The idea is to provide an overview of monorepo graphs where nodes are monorepo projects (app or lib), in the same spirit as Nx graph. So you could imagine doing the same, where you build your own graph based on your own rules, and skott can then deal with the rest, such as visualization, cyclic dependencies, and all other features that relying on the emitted graph (this is a old screen shot)

This is literally reducing the whole monorepo file tree around 2k files to few nodes only, enhancing a lot visualization.
Nonetheless, I grant you the fact that it is not the simplest solution and we should probably provide a better and more straightforward way to do that.
Specification
I would say given this impacts all of the visualization modes (even though web application is by far the most used one), this should be a core feature added in skott in the same spirit as you suggested where we could define some sort of namespaces and scopes under which modules should be grouped.
Let me just recap your suggestion
Given the following file tree:
  src/
    features/
        feature-a/
        feature-b/
   widgets/
       widget-a/

and the following definition:
"groups": {
   "features": "src/features/{name}/*",
   "widgets": "src/widgets/{name}/*"
}

you expect the following graph to be emitted:
{
   "features/feature-a": {...},
   "features/feature-b": {...},
   "widgets/widget-a": {...}
}

That is 3 nodes, right?
From the definition itself, do you imagine other cases where one would want src/features/{name}/* converted to something else than "group by all first children sub-folders"? Do you imagine some sort of regex pattern that could be provided to group by something else than direct children? I'd suggest starting by something simple on the definition side, with no regex pattern, but rather with parent folders than directly mean "group by all direct sub-folders starting from this one".
Implementation
At first glance I'd say that "groups" are like "views" in database in the sense that they expose a different graph shape but the internal graph stays untouched. That would allow all information such third-party, builtin modules, etc to still be collected in the background, but applying the "groups" filter on it, would emit a version of the graph in the same spirit as it's done there: https://github.com/antoine-coulon/krush/blob/af6d8e661deb300f93733f1e9ef1b4c839ccb309/plugins/rush-plugin-skott/src/plugins/visualize/graph.spec.ts#L229
Given that patching the main graph is not a good idea imho, we still need to keep the source of truth reachable so that we can still rely on it anytime.
Consequently we could expose both graphs, one "full" containing everything and a "compacted" one:
const { getStructure } = await skott(withGroupsConfig);
const { graph, compactedGraph }  = getStructure();

assert.deepEqual(graph, {
   "src/features/feature-a/index.ts": {...},
   "src/features/feature-b/index.ts": {...},
});

assert.deepEqual(compactedGraph, {
   "features/feature-a": {...},
   "features/feature-b": {...},
});

In the end, all visualization modes could detect whether there are other "views" of the graph and decide to use them or not. One benefit of having both for the web application would also be to dynamically switch modes between "compacted" and "full", if you ever want to expand whatever is inside feature/feature-a only, otherwise you lose the information of whatever is contained inside that module.
Recap:

Build the entire graph using the provided resolvers
Apply the "groups" filter on it
"groups" has for consequence of adding a "compactedGraph" to the getStructure interface
Display modes would either use the "full" or "compacted" version of the graph

What do you think of that?

Thanks for a detailed reponse!

That is 3 nodes, right?

Yep, given that tree i would expect just 3 nodes

do you imagine other cases where one would want src/features/{name}/* converted to something else than "group by all first children sub-folders"? Do you imagine some sort of regex pattern that could be provided to group by something else than direct children?

So far, not really 🤔
Approaches like Feature Sliced Design or similiar are introducing "internal" structure rules too, though
As far, as i remember, even the good-old "components and containers" approach has something similiar, e.g. if project also uses Redux - there would be internal "reducers", "actions", etc
But I think, that, given that the skott's graph is covering project as a whole, these internals, even if structured, are still not really relevant
Also, if there is a option to see a full graph (and if the size of the project allows it to be rendered), user can always see those details
I think, that it is probably makes sense to add a possibility to "unwrap" a one specific group in the graph, so internal details are visible only when needed and only for one specific group, which is being investigated 🤔

I'd suggest starting by something simple on the definition side, with no regex pattern, but rather with parent folders than directly mean "group by all direct sub-folders starting from this one"

Yep, this sound good to me 👍

do you imagine other cases where one would want src/features/{name}/* converted to something else than "group by all first children sub-folders"? Do you imagine some sort of regex pattern that could be provided to group by something else than direct children?

Actually, there is one case 😅
For some reason i forgot, that our project actually has a structure like
src/
 /team-a
   /features
    ...
   /widgets
    ...
  /team-b
   /features
     ...

which is in fact "multi-level" group, like src/{team}/widgets/{name}

But actually we cound try to group it like

"group by all direct sub-folders starting from this one"

though 🤔
Something like this:
"groups": {
   "teamAFeatures": "src/team-a/features/{name}/*",
   "teamAWidgets": "src/team-a/widgets/{name}/*",
   "teamBFeatures": "src/team-b/features/{name}/*",
   "teamBWidgets": "src/team-b/widgets/{name}/*",
}

Also, if combined with ignorePattern, looks like it is possible to construct a skott command, that will print a graph only for the code of one specific team, so this team could look at their own architecture without external (i.e. code of other teams) connections

Actually these are interesting use cases. I think we can start by providing a straightforward way of grouping things as you initially mentioned and what I refer to "group by all first children sub-folders".

Also, if combined with ignorePattern, looks like it is possible to construct a skott command, that will print a graph only for the code of one specific team, so this team could look at their own architecture without external (i.e. code of other teams) connections

Exactly, I think one should aim for the smallest subset to observe, either using ignorePattern to black list entries or using cwd to select a specific working directory to start the analysis from, an this could still be combined to "groups" for instance:
skott({
  cwd: "src/team-a",
  "groups": {
   "features": "{cwd}/features",
   "widgets": "{cwd}/widgets"
   }
})

If one still wants to have the global picture but in a grouped way, then we could still allow this kind of configuration that you mentioned:
"groups": {
   "teamAFeatures": "src/team-a/features/{name}/*",
   "teamAWidgets": "src/team-a/widgets/{name}/*",
   "teamBFeatures": "src/team-b/features/{name}/*",
   "teamBWidgets": "src/team-b/widgets/{name}/*",
}

I think that this behavior of grouping can be generalized to work with any length of record so whether using the config above alone or combining it with other API options would be fine.
But again, I think this "groups" feature should just apply a custom grouping to the "raw" graph without mutating it, so it's fine experimenting that and adding another property to the SkottStructure interface. Then we'll see how to consume that in the visualization modes to provide better visualization experience

Would you be willing to implement that? Just for you to know, there is no time constraint.
I will try to provide detailed hints tonight of what could be a first big-picture implementation of that feature

Would you be willing to implement that?

Sure!

what could be a first big-picture implementation of that feature

Given the "group by all direct sub-folders starting from this one" definition, i think, something like that should work?
groups API
In the skott API configuration there should be optional "groups" configuration
"groups": {
   "features": "src/features/{name}",
   "widgets": "src/widgets/{name}",
   "core": "src/core" // if no {name} template provided - whole folder is a group
}

There is an edge-case though:
"groups": {
   "teamFeatures": "src/team/features/{name}",
   "team": "src/team/{name}",
}

"team" group is containing all of the "teamFeatures" modules in it, so it is not clear, which group should be rendered to the compacted graph
I think, that it should be resolved in a way, that bigger group ("team" group in the example) should take priority there 🤔
And such nested groups, probably, should not be added to the compacted graph at all - at least in the first implementation, without an option to "unwrap" a single compacted group in the web-app's graph
Compacted Graph
In the skott output:
const { getStructure } = await skott(withGroupsConfig);
const { graph, compactedGraph }  = getStructure();

compactedGraph // Graph | null, because there might not be any groups defined

Web App
In the skott web-app:

If there is a compacted version of the graph - it gets to render first, since original graph might be just too large to render, so it would just block the main thread
If there is a compacted version of the graph - there is a "graph action" to switch to "source graph" and back

What do you think?

Yes, that definitely looks good to me!

Given the "group by all direct sub-folders starting from this one" definition, i think, something like that should work?

That Groups API looks ok to me. I'm just thinking about the dynamic segments i.e: src/features/{name} where {name} can be a bit counter intuitive imho. Nonetheless there is no standard way of expressing that as a glob, but we could probably just provide a wildcard instead src/features/* as usually a wildcard like this matches all files within the sub folders, we could consider that instead it matches all children sub directories.
Consequently:
"groups": {
   "features": "src/features/*", // create groups from all direct children sub folders, `src/features/team-a` will be a group
   "core": "src/core" // if no "*" wildcard provided, whole folder is a group
}

For the edge case you mentioned, I think we should provide a constraint defining that no group definition should overlap, at least in the first iteration, because it will be hard to deal correctly with multiple versions of a same set of files under different groups and still be able to represent an unified version of the graph. In other words, you would end up with 2 nodes where 1 node is a subset of the other one and at first sight this would be too tedious to deal with that in all aspects. Usually achieving that is done by clustering, but in clustering only one version of the group is there at all time, sub groups are being merged in outer groups.

Compacted Graph

const { getStructure } = await skott(withGroupsConfig);
const { graph, compactedGraph }  = getStructure();

I initially named the thing "compacted" but when rethinking it, it might be a bit misleading, because we are talking about "groups" and the "groups API", and then we end up with a "compacted" property thing when defining group 🤔 My bad actually, I think this should be named groupedGraph or graphWithGroups, to be fair with the configuration namings.

compactedGraph // Graph | null, because there might not be any groups defined

Yes, but I think for that we should favor "undefined" instead of "null", because the property won't be there when no "groups" are defined. Consequently you'd have graphWithGroups (or groupedGraph) such as:
interface SkottStructure {
// ... already existing props
graphWithGroups?: Graph;
}

For the Web App, I fully agree with your vision:

If there is a compacted version of the graph - it gets to render first, since original graph might be just too large to render, so it would just block the main thread

Yes 🙂

If there is a compacted version of the graph - there is a "graph action" to switch to "source graph" and back

And yes ✅

Note that you could split the work in two parts as skott (core) and skott-webapp are two distincts packages, they can be independently published. This one is fully up to you, just saying that in case you thought/hadn't seen that both were shipped in the same package
Also, as we're talking about the "grouping" feature, it makes me realize that some of the features around "skott-webapp" will only work with graphs containing files. This means that if we turn on the "graphWithGroups" visualization mode, sections such as "File Explorer" and some of the sections in "Summary" become irrelevant.
But this is great, because it's a subject that I needed to tackle for the new version of the web app to be compatible with monorepo visualization, with hundred/thousands of projects where "nodes" are packages and not files. I'll keep you updated

Nonetheless there is no standard way of expressing that as a glob, but we could probably just provide a wildcard instead src/features/* as usually a wildcard like this matches all files within the sub folders, we could consider that instead it matches all children sub directories

I thought about it a bit and I agree about {name}, if we consider that the user would expect the glob pattern there
But this way src/features/*  also feels a bit counter-intuitive to me, as glob like this also matches desctiption "whole folder is a group" 🤔
Maybe it could just be an explicit configuration object?
groups: {
   features: {
      parentPath: "src/features",
      // modulePath == ["feature-name", ...]
      getGroup: (modulePath) =>  modulePath[0],
   },
   core: "src/core" // whole folder is a group
}

This way groups resolving is mostly moved to the userland, which is very flexible
But probably less desirable, since it is not possible to do any kind-of-static analysis this way (as it is possible with a normal string pattern)
And since this function would be called on every module path, it would be a potential performance bottlneck, which will be out of skott's control 🤔

Hey again @AlexandrHoroshih!
Having a configuration object with functions could be nice indeed to cover more edgy cases that would not make sense with a simple string literal, this would also follow the path I initiated with dependency resolvers where skott offers some sort of IoC for graph resolution. This would for sure delegate all the responsibility to the consumers and so the grouping would become their own responsibility (dealing with overlap etc) and it's pretty much ok with that.

And since this function would be called on every module path, it would be a potential performance bottlneck, which will be out of skott's control

Well yeah but grouping is like a lazy action, internal graph resolution construction will remain the same and once the user or any of the visualization modes use getStructure#graphWithOptions this could be indeed a lazily loaded property where all the computation happen.
So if I sum up for me there would be 3 possible definitions:
const groupsConfig = {
   "features-1": "src/core" // whole "core" folder is a group, and this group is named "features-1",
   "features-2": "src/core/*" // all sub-folders (only first level such as `src/core/lol`) of "src/core" are grouped, for instance `features-2/lol` is a group,
   "features-3": {
         basePath: "src/feature",
          // in features-3 configuration, groups would become constructed such as `features-3/group0`, `features`
         group: () => (["group0", "group1/skott", "group2/is-an-awesome", "group3/library"]-
    }
}

Regarding the overlap checking we can naively check that all the static paths are not part of the same base path, in the above case, src/core and src/feature (basePath) prop are fine, but if another group defined something such as src/feature/something this would not be ok.
So

Just to clarify:
Should skott just throw an exception right away, if such groups definition is provided via config?

Yes, I would favor fail fast when some unexpected config is provided rather than silently patching things under the hood or still providing a graph that will most likely confuse and not help finding that the config is wrongly provided.

Also as configuring groups manually (specifying keys of the record statically "features-1", "features-2" etc) can become tedious on such big modules, I'd then expect the config option to be a function where one can dynamically configure keys (group names) and their matching content. This would for instance allow some automatic grouping based on Nx, Rush, Turborepo configurations, but also some other rules that could be established dynamically depending on the needs.
This is out of the scope of first iterations, I'm just thinking about it loudly 😄

Upvoting the configuration function option 🙂

Let's make it happen then!

Hello @antoine-coulon !
I finally had found some time to work on this feature - here is a small PR draft, which is still Work in Progress though, so i don't want to target it to the main skott repo yet.
But in the process of working on it, I discovered a few interesting points that I'd like to discuss:
The config API design
Current implementation
Current design is implemented as we discussed before, with an edge-case handled - in function API an option to return null should be supported, so it is possible to discard some module, if it does not belong to this group for some reason.
  groups?: {
    [groupName: string]:
      | string // <- path pattern
      | {
          /**
           * Scope of the group
           */
          basePath: string;
          /**
           *
           * @param modulePath path to the module
           * @returns group name, to which the module belongs or null if it does not belong to the group
           */
          getGroup: (modulePath: string) => string | null;
        };
  };

And generated graph looks something like that
After trying out this approach at my work project i eventually moved to using only a function API:
{
 groups: {
    app: {
      basePath: 'client',
      getGroup: (nodePath) => {
        if (nodePath.includes('client/src/entrypoint')) {
          return 'entrypoint';
        }

        if (nodePath.includes('client/src/internal')) {
          return 'internal';
        }

        if (nodePath.includes('client/src/platform')) {
          return 'platform'
        }

        if (nodePath.includes('client/src/product')) {
          const match = nodePath.match(
            /product\/(?<team>.*?)\/(?<entity>.*?)\/(?<name>.*?)\//,
          );
          if (match && match.groups) {
            const { team, entity, name } = match.groups;
            return `product/${team}/${entity}/${name}`;
          }
        }

        return null;
      },
  }
}

The "config" option just wasn't really useful to me in the end
Suggestion
I feel like that "function" option is in fact the most flexible and the least confusing option of all 🤔
Also, while working on the PR, i got better understanding of skott's internals and found out that "config" option doesn't really helps to optimize anything so far - to build a "compact" graph we need to do basically the same work on resolving module dependencies that is needed for the "full" one anyway.
I suggest to simplify the API design and only accept single function, something like that:
{
 groups: (nodePath) => {
   if (anyConditionToMatchPathToSomeGroup(nodePath)) return "groupName"

   // Module did not matched any group and is discarded from the compact version of the graph
   return null;
 }
}

☝ Also i think, that if we will decide to go with this API - it might need a better name, like "collapseRule" or "compactSomething" 🤔
Pros

Single function like (path: string) => string | null is pretty simple and flexible
It allows simpler implementation: e.g. single function API does not need any special validation about invalid or overlapping patterns

Cons

Function API, probably, cannot be used with skott-cli easily, while pattern-based config can

The latter con is only a problem because there is currently no way to run skott in web application launch mode other than via skott-cli.
While it is possible to describe groups in cli like --group="groupName: pattern/*" -- group="otherGroupName: pattern/*" - i think it may become rather uncomfortable, if project is large and there is a more than a two or three groups.
I think, that skott also needs a way to trigger it's CLI runtime in a programmatic way, for provided instance 🤔
Something like this:
// project/scripts/build-skott-graph.mjs

import { Skott, runSkottInstance } from "skott"

 const instance = new Skott({
   ...config
 })

 await runSkottInstance(instance, {
  displayMode: "webapp",
  ...options
 })

If you ok with that, i can file a separate feature-request issue (and make it happen, when i have time 😅)
What do you think about this suggestions?

Thanks for the reply!

One interesting note is that I should probably expose a util function allowing to seamlessly create that web server so that it's easy run the web app and skott API on their own, without going through the CLI.

That would be really great!
Such a feature, as I think, would simplify things a lot, because a path like import a special function -> call it with Skott instance and settings -> Skott Webapp opens feels way nicer, than custom server one 🤔

Please keep going in the nice direction you mentioned, that is using one function in the "groups" property

Got it 👍
I think i will split the work in two parts then (as you initially suggested actually) - first the group or groupBy function and then the web app part

Hi @antoine-coulon !
Here is the first part - the groupBy function in the skott package:
https://github.com/antoine-coulon/skott/pull/146
I also tried to use this API locally (by monkeypatching my node_modules/skott) to see, how groupedGraph would look in the skott-webapp and result is already much better than looking at relations between ~7000 modules directly 😄
What is interesting is that eventually i opened a few tabs with different groupedGraph renders at different levels of details:
Because project has a structure like {team}/(services|widgets)/{name}, it is possible to explore relations at both team level and specific services and widgets level.

And since the project is quite large, i was switching tabs with different views to make more sense of it.
Maybe it could lead to next iteration of this feature in skott 🤔

Hello @AlexandrHoroshih, thanks for landing the PR! I just reviewed it.
The grouped graph, looks great on the picture, it will definitely improve the visualization experience and help extracting useful information.

And since the project is quite large, i was switching tabs with different views to make more sense of it. Maybe it could lead to next iteration of this feature in skott 🤔

I'm not sure I understood that part, what do you exactly suggest? Having a set of groupedGraph and each graph would be uniquely identified (string identifier) that would follow different rules, instead to only one (as done in the PR)?
If it is what you had in mind, I think it's a great idea :) If it was not that, then it probably just gave us an interesting idea! In the web application, you could then switch between all the different graphs (views). We could even imagine provide a way of creating groupedGraphs directly from the web application itself.

Thanks!

I'm not sure I understood that part, what do you exactly suggest? Having a set of groupedGraph and each graph would be uniquely identified (string identifier) that would follow different rules, instead to only one (as done in the PR)?

Yeah, I was more or less thinking along those lines 😅
But I'm not sure that's the best way to go about it:
As my example shows that on a large project, seeing several different "layers" of the project structure makes it easier, but not radically so - there are still too many links between modules, it's still not easy to sort them out.
I think that maybe in the future, using the grouping API, it will be possible to do some more advanced analysis that could show "harmful" groups of modules.
For example, those that depend on many other modules at once and, therefore, if they are imported anywhere, they will pull half of the whole project into the main bundle 🤔

@AlexandrHoroshih thanks for contributing, the PR just got merged and the 0.33.0 got releases with the new feature!

As my example shows that on a large project, seeing several different "layers" of the project structure makes it easier, but not radically so - there are still too many links between modules, it's still not easy to sort them out.

Isn't it because you need to have a more precise definition of your groups instead of having huge top-level groups?
Also, at some point I guess it will be the job of the web application to offer a better visualization, skott's core goal is to collect all the information that it can get. From that you can imagine hundreds of use cases starting from discarding groups, filtering groups, isolating a group and its direct dependencies etc.

From that you can imagine hundreds of use cases starting from discarding groups, filtering groups, isolating a group and its direct dependencies etc.

I gave it a thought and i agree, that makes sense 🤔
I tried to do a deeper analysis and flow was kind of the same you described here:

Look at a "big picture", notice stuff that stands out, e.g. an service (or other architectural building block) that depends on others "too much"
Re-build groupedGraph with different groupBy rule, which renders problematic service and discards unrelevant information
and so on, until i figure out the problem and solution like "this service does too much and should be refactored into a few smaller services"

Yeah definitely, to me it seems hard to get the right groups at first glance. It feels like knowing what groups you want goes by visualizing first and incrementally grouping into things that make sense for the current use case.
Thing is, doing that back and forth does not really provide a great UX/DX as you always need to change the groupBy function and restart the process, that is why I was thinking of doing some of the work directly in the web app or even suggesting directly groups based on the graph (at some point in the future).
But I guess those will be done in next iterations, allowing the grouped graph to be displayed at first and allowing a switch between the "full" and "grouped" graph will already be good enough!

allowing a switch between the "full" and "grouped" graph will already be good enough!

Speaking of that 😅
Currently most of the skott-webapp source code expects to work directly with data object, which contains data.graph and related meta-information like data.cycles
Given that we are already discussing an arbitrary number of such graphs i feel like it would make sense to refactor skott-webapp internals a bit, so instead of current "single graph" type:
export interface AppState {
  data: DataState;
  ui: UiState;
}

it should expect something like that
export interface AppState {
  data: {
   source: DataState, // data for original modules graph, always available
   [key: string]?: DataState, // Any graph derived from source one - e.g. "grouped"
 };
  ui: UiState;
}

export interface UiState {
  filters: {
   // ...
  };
  network: {
   currentTarget: string; // e.g. "source" by default
   // ...
  }
}

So that it is possible to do something like that:
      subscription = appStore.store$
        .pipe(
          map(({ data, ui }) => data[ui.network.currentTarget]),
          distinctUntilChanged(isEqual)
        )
        .subscribe((data) => {
          const { graphNodes, graphEdges } = makeNodesAndEdges(
            Object.values(data.graph),
            { entrypoint: data.entrypoint }
          );

          setNodesDataset(new DataSet(graphNodes));
          setEdgesDataset(new DataSet(graphEdges));
        });

and that
<Menu.Dropdown>
          <Menu.Label>Graph View</Menu.Label>
          {Object.keys(state.data).map((name) => (
            <Menu.Item key={name}
             onClick={() => setNetworkCurrentTarget(name)}
            >
               {name}
            </Menu.Item>
          ))}
</Menu.Dropdown>

What do you think about this approach to implementation?

I'm not sure if this is the right time to generalize yet, as having custom groups created on the fly will have other implications such as having custom UI states per custom group as well (so all UI state can't be at the top-level store, there might be shared UI state but also per-graph UI state in addition to data). So I might need to handle as well the entire store design to make that fit with multiple graphs at once.
For now, I would suggest you to stick to what we had discussed in the first place, which is dealing with two statically known graphs
export interface AppState {
  data: {
   full: DataState, // data for original modules graph, always available
   grouped?: DataState, // Any graph derived from source one - e.g. "grouped"
 };
  ui: UiState;
}

And given the "grouped" graph being defined or not, just display a UI switch to display the "source"/"full" or the "grouped" one.
For now, let's keep things simple and keep the same UiState shared, that is if "circular dependencies" were checked before doing the graph switch, then it should be automatically rendered with the newly selected graph.
Another notes:

"Summary" section should be updated accordingly, information about files should be updated depending on the groups
"File Explorer" section should be hidden when the "grouped" graphs is selected, as the file explorer only makes sense in the context of a full graph where nodes are the files themselves

As a side note, I'm totally fine with having hardcoded such as "full" | "grouped" in the context of this iteration and also naively considered only these two. Let's incrementally bring small chunks of value to the users :)


Cant Start Rocket League after Bakkes Update for Rocket League Version 2.14

BakkesMod Version:  163
RocketPlugin Version:  0.7

Describe the bug
Unable to start Rocket League after the Rocket League Update to 2.14.
To Reproduce
Steps to reproduce the behavior:

Update Rocket League to 2.14
Update Bakkes Mod
Try to Start Rocket League

Expected behavior
First the Game Should start but after a little Time there will be a Error Message an it would crash
Screenshots

Additional context
I also get this in to my clipboard:
`Fatal error!
Address = 0xdba03d08 (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\plugins\serverhost.dll]
Address = 0x45496b   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x44fe86   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x45771a   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x3b09ec   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x3b10f4   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x3b0249   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x3b2713   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x39ffe6   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x44b0f3   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x43cbc6   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0x43c803   (filename not found) [in C:\Users\user\AppData\Roaming\bakkesmod\bakkesmod\dll\bakkesmod.dll]
Address = 0xf40efb31 (filename not found) [in P:\Steam\steamapps\common\rocketleague\Binaries\Win64\RocketLeague.exe]
Address = 0xf3fbf55b (filename not found) [in P:\Steam\steamapps\common\rocketleague\Binaries\Win64\RocketLeague.exe]
Address = 0xf46f2085 (filename not found) [in P:\Steam\steamapps\common\rocketleague\Binaries\Win64\RocketLeague.exe]
Address = 0xf4682b4d (filename not found) [in P:\Steam\steamapps\common\rocketleague\Binaries\Win64\RocketLeague.exe]
Address = 0xf467e24c (filename not found) [in P:\Steam\steamapps\common\rocketleague\Binaries\Win64\RocketLeague.exe]
Address = 0xf467e29a (filename not found) [in P:\Steam\steamapps\common\rocketleague\Binaries\Win64\RocketLeague.exe]
Address = 0xf4684611 (filename not found) [in P:\Steam\steamapps\common\rocketleague\Binaries\Win64\RocketLeague.exe]
Address = 0xf4e0cf76 (filename not found) [in P:\Steam\steamapps\common\rocketleague\Binaries\Win64\RocketLeague.exe]
Address = 0x88827034 (filename not found) [in C:\WINDOWS\System32\KERNEL32.DLL]
Address = 0x8a662651 (filename not found) [in C:\WINDOWS\SYSTEM32\ntdll.dll]
Address = 0x8a662651 (filename not found) [in C:\WINDOWS\SYSTEM32\ntdll.dll]
`
I dont realy know if it is a Problem with RocketPlugin but the Picture say somethin like this

This is a crash from Rocket Host which for some reason uses a file named RocketPlugin.cpp.
There was an update to Rocket Host not that long ago, so I assume it is fixed now.


plat-3434 updating action inputs in README

Description
Ticket number
PLAT-3434
Checklist

[ ] I have updated the changelog

[ ] I have tested this and added output to Jira
Comment:

[ ] Documentation added (link)
Comment:

Co-authored by

is the changelog being deleted ?

@angold5 yes, the team agreed to remove from this repo


Update SPF and DMARC insights

Does not pass equal a fail though? Just want to check that the name of the insight will still match the value, as we're not directly looking for *fail anymore.

yes sir that's the idea!


Treat tsconfig.json as JSONC

What version of oneRepo are you using?
1.1.0
Environment info
Not relevant.

Command
one tsc --use-project-references
What did you expect to happen?
Comments preserved.
What actually happened?
Comments are stripped if project references need an update, because it's serialized via JSON.stringify:
https://github.com/paularmstrong/onerepo/blob/be882795966edd469524734de8140be6fc111685/plugins/typescript/src/commands/typescript.ts#L64
Participation

[ ] I am willing to submit a pull request for this issue.

I think this is out of the scope of oneRepo's file module, unless there's a JSONC package that isn't too heavy that can handle this easily.
JSONC parsing/comment stripping is really easy. However, reading JSONC and keeping the comments intact – and rewriting changes with them requires an entire AST parser/lexer and writer (eg, babel?).


Disabled C++ test batchnorm_test.cc - TestChannelAxis

https://github.com/apache/incubator-mxnet/blame/master/tests/cpp/operator/batchnorm_test.cc#L1513
Test has been disabled as part of MKLDNN

The same comments as https://github.com/apache/incubator-mxnet/issues/10131


InfluxDb Table Structure -- Misleading Info

On the URl below the docs state:
https://v2.docs.influxdata.com/v2.0/reference/key-concepts/table-structure/

InfluxDB 2.0 uses the following columnar table structure to store data:

This is potentially misleading as the DB actually stores data in a TSM file.  Furthermore Influx does not really have "tables".
Rather we may want the docs to state that this is for annnotated csv, and is how data is presented in a response; not how it is actually stored in the DB
This is sort of nuanced but an important delineation.

interesting, @dp1140a-thank you!


Fix mistery big

in S3File._fetch, caused missing bytes at the end of a file when
reading blocks big compared to the file blocksize.

Great


Redirecting after Login

✨ Feature Request
Description of the Feature
I would like to add redirecting feature when the user logins
Problem Statement
Currently the page is not changing after user login
Are you participating in GSSoC?

[x] Yes
[ ] No

any updates

once the login is successful a new page should be shown to the user currently its not happening

any updates


support Rust 1.70 as MSRV

Change Summary
This PR sets Rust 1.70 as the minimum supported version. To go lower, we would need to replace use of OnceLock with external once_cell::sync::OnceCell both here and in jiter.
I think going below 1.65 would be hard because we use both GATs and let-else from that release.
Related issue number
Related to #1176
Checklist

[ ] Unit tests for the changes exist
[ ] Documentation reflects the changes where applicable
[ ] Pydantic tests pass with this pydantic-core (except for expected changes)
[ ] My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

please review


Confusion between deployment pipeline and a release pipeline

Hi - I've been getting a new CI/CD pipeline setup for a greenfield project, but one thing is bothering me:
I do not fully understand the difference between deployment pipelines and a release pipeline. They both seem to enable the same thing: pushing builds to environments. One is yaml based, one is UI based (from what I can tell).
I see in the documentation there is a menu item for "Deploy apps", and "Deploy apps (Classic)". Deploy apps (Classic) talks about the release pipeline stuff (stages, automations, artifacts -- this is what I've been building) - but its wording makes me think it is going to be discontinued, as if it is the "Old version".
When I try to search for any kind of help on "Release pipelines", most of the time it takes me to documentation on yaml-based pipelines, which is confusing.
Should I be building my greenfield deployment pipeline with the documentation linked in this issue (yaml based), or a release pipeline?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 77d95db6-9983-7346-d0eb-4b7443e4e252
Version Independent ID: 0a22cccc-318d-592f-d1ab-09ec01d88087
Content: Environment - Azure Pipelines
Content Source: docs/pipelines/process/environments.md
Product: devops
Technology: devops-cicd-process
GitHub Login: @juliakm
Microsoft Alias: jukullam

Slight clarification:
I understand, theoretically, the different ideas behind a build, deploy, and release (at least I think I do).
A build is an action that produces artifacts (nuget package, docker image, config ymls, etc...).
A deployment is an action that takes build artifacts and pushes them to some server.
A release is, in my mind, is what combines the build and deployment together and is responsible for coordination.
What is confusing is that it looks like Azure DevOps is moving toward a fully yaml based pipeline across the board (for building, deploying, releasing) but hasn't come out and fully said that explicitly. This is just what I've inferred.
I realize this is just a comment on documentation, but if anyone can provide some direction as to my usage of the "Release" UI pipeline feature, that would be helpful.

Slight clarification:
I understand, theoretically, the different ideas behind a build, deploy, and release (at least I think I do).
A build is an action that produces artifacts (nuget package, docker image, config ymls, etc...).
A deployment is an action that takes build artifacts and pushes them to some server.
A release is, in my mind, is what combines the build and deployment together and is responsible for coordination.
What is confusing is that it looks like Azure DevOps is moving toward a fully yaml based pipeline across the board (for building, deploying, releasing) but hasn't come out and fully said that explicitly. This is just what I've inferred.
I realize this is just a comment on documentation, but if anyone can provide some direction as to my usage of the "Release" UI pipeline feature, that would be helpful.

@jackjwilliams We're looking at your issue for future doc enhancements. In the meantime, consider these resources:

Azure DevOps Services
Azure DevOps Support Bot
Azure DevOps on Stack Overflow

@jackjwilliams YAML pipelines are multi-stage so the deploy and release stages can be in one pipeline. You can learn multi-stage pipelines here. Previously, there were separate UI-based (Classic) interfaces that broke out the build and release stages.
If you are starting from scratch, YAML pipelines might be the better fit. To learn more about the features of YAML pipelines, Classic release pipelines, and Classic build pipelines, you may find this table on feature availability helpful.


I want short unique id. UUID, CUID are too long

Problem

I make banner function in my project. Maximum number of banner is 10,000 or less. but cuid and uuid data type are too long.

Suggested solution

I want 10~12 byte of short unique id. Please make short unique id.

You can use any self-defined function that is available on your database in a default.
Prisma offers the most common ones, uuid and cuid, but we will most probably not add support for others.
Does that answer your issue?

@janpio
Could you please show me an example of self-defined  function?

One to create a nanoid with PostgreSQL: https://github.com/viascom/nanoid-postgres/blob/main/nanoid.sql

@janpio
Thank. I will make short uuid function.

Done in https://github.com/prisma/prisma-engines/pull/3556


handle utf-8 decoding errors in fortls

In langserver.py have extended the try catch block in def init_file(file path) to also include the two with statements. This handles utf-8 decoding errors in the PY3K relevant code:
`def init_file(filepath):
#
try:
if PY3K:
        with open(filepath, 'r', encoding="utf-8") as fhandle:
            contents = re.sub(r'\t', r'  ', fhandle.read())
            contents_split = contents.splitlines()

    else:
        with open(filepath, 'r') as fhandle:
            contents = re.sub(r'\t', r'  ', fhandle.read())
            contents_split = contents.splitlines()
#

    fixed_flag = detect_fixed_format(contents_split)
    ast_new = process_file(contents_split, True, filepath, fixed_flag)
except:
    return None
else:
    # Construct new file object and add to workspace
    tmp_obj = {
        "contents": contents_split,
        "ast": ast_new,
        "fixed": fixed_flag
    }
    return tmp_obj`

Thanks for the suggestion. I have merged in a commit that catches decoding errors and also reports more information on the type of issue when reading, changing, decoding, or parsing a file fails.


Not able to Attach to an individual File Share using SAS URL

Preflight Checklist

[X] I have installed the latest version of Storage Explorer.
[X] I have checked existing resources, including the troubleshooting guide and the release notes.
[X] I have searched for similar issues.

Storage Explorer Version
1.20.1
Regression From
not sure
Architecture
x64
Storage Explorer Build Number
20210720.10
Platform
Windows
OS Version
Windows 11
Bug Description
The URL is missing a path to the resource.
File Share SAS URL:
https://mystorage.file.core.windows.net/?sv=2022-11-02&ss=f&srt=s&sp=rwdlc&se=2023-08-25T21:04:42Z&st=2023-08-25T13:04:42Z&spr=https,http&sig=REDACTED
Steps to Reproduce

Launce Storage Explorer
Open Connection Dialog "What kind of Azure resource do you want to connect to"
select "File Share"  (attach to an individual File Share)
enter SAS URL
error : The URL is missing a path to the resource.

Actual Experience

Launce Storage Explorer
Open Connection Dialog "What kind of Azure resource do you want to connect to"
select "File Share"  (attach to an individual File Share)
enter SAS URL
error : The URL is missing a path to the resource.

Expected Experience
No response
Additional Context
No response

Hi @brendandelange, you need to add the name of your file share in the SAS URL, specifically right after the .net/ and before the ?.

Hi @brendandelange, you need to add the name of your file share in the SAS URL, specifically right after the .net/ and before the ?.

Hi there,
When adding the name of my file share as suggested I get he following error:
Unable to retrieve child resources.
Details:
Cannot read property 'version' of undefined

@brendandelange If you see the "Cannot read property 'version' of undefined" error immediately after adding the connection, could you look at your Credential Manager and see if it is full? Also pay attention to how many entries seem to belong to Storage Explorer. Windows Credential Manager has a per app capacity limit Credential Manager capacity doc. The error is thrown when we attempt to read the entry for the connection but cannot get a valid one.

@brendandelange If you see the "Cannot read property 'version' of undefined" error immediately after adding the connection, could you look at your Credential Manager and see if it is full? Also pay attention to how many entries seem to belong to Storage Explorer. Windows Credential Manager has a per app capacity limit Credential Manager capacity doc. The error is thrown when we attempt to read the entry for the connection but cannot get a valid one.

Good day Jason,
My Credentials manager only had 15 entries, I deleted all Storage explorer related Credentials and tried re adding the File share, but sadly get the same error.
Unable to retrieve child resources.
Details:
Cannot read property 'version' of undefined

@brendandelange I see, the problem doesn't seem to relate to the Credential Manager. Have you tried upgrading Storage Explorer or is it not an option for you?

Closing due to lack of response. If you are still encountering this problem, you can either open a support ticket via the Azure portal, or open a new issue here. This issue however will no longer be monitored.


rustfmt-nightly doesn't install cargo-fmt binary

Output from cargo install:
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading rustfmt-nightly v0.1.0
  Installing rustfmt-nightly v0.1.0
 Downloading env_logger v0.4.3
 Downloading strings v0.1.0
 Downloading toml v0.4.1
   Compiling unicode-segmentation v1.2.0
   Compiling num-traits v0.1.39
   Compiling term v0.4.5
   Compiling itoa v0.3.1
   Compiling diff v0.1.10
   Compiling void v1.0.2
   Compiling getopts v0.2.14
   Compiling quote v0.3.15
   Compiling dtoa v0.4.1
   Compiling log v0.3.8
   Compiling serde v1.0.8
   Compiling libc v0.2.23
   Compiling rustfmt-nightly v0.1.0
   Compiling unicode-xid v0.0.4
   Compiling regex-syntax v0.4.1
   Compiling utf8-ranges v1.0.0
   Compiling unreachable v0.1.1
   Compiling strings v0.1.0
   Compiling synom v0.11.3
   Compiling memchr v1.0.1
   Compiling thread-id v3.1.0
   Compiling syn v0.11.11
   Compiling aho-corasick v0.6.3
   Compiling thread_local v0.3.3
   Compiling serde_json v1.0.2
   Compiling toml v0.4.1
   Compiling regex v0.2.2
   Compiling serde_derive_internals v0.15.1
   Compiling serde_derive v1.0.8
   Compiling env_logger v0.4.3
    Finished release [optimized] target(s) in 118.6 secs
  Installing ~/.cargo/bin/rustfmt

You can see that the cargo-fmt binary wasn't installed.
I think the problem is that there's an explicit [[bin]] section now, so all the binaries now need to be listed: https://github.com/rust-lang-nursery/rustfmt/commit/65d1e950a9e03c41ee2c8cd0b6f0f2201ecc0104?diff=unified#diff-80398c5faae3c069e4e6aa2ed11b28c0R17
Quoting crates.io docs:

Do note, however, once you add a [[bin]] section (see below), Cargo will no longer automatically build files located in src/bin/*.rs. Instead you must create a [[bin]] section for each file you want to build.

Did you accidentally push the wrong thing to the libsyntax branch? This doesn't look like it has the rustfmt-nightly stuff: https://github.com/rust-lang-nursery/rustfmt/blob/libsyntax/Cargo.toml

libsyntax branch is an older version we need to keep around. Current versions are master (libsyntax) and syntex (using syntex)

Master looks like this: https://github.com/rust-lang-nursery/rustfmt/blob/master/Cargo.toml#L3
I'm confused, shouldn't it read rustfmt-nightly there?

It should, I forgot to push my local master, fixed now.

Ok, looking into raising a PR.

Thanks! cargo +nightly install rustfmt-nightly && cargo fmt works nicely now!


Crash continuously: chunkc. Too many corpses being created.

Hi,
I am unable to run chunkwm since today.
I upgraded to latest version but stille have the issue.
I see in in the console logs the messages:
par défaut	00:45:28.663061 +0200	kernel	Process[2008] crashed: chunkc. Too many corpses being created.
erreur	00:45:28.664443 +0200	ReportCrash	Failed to start process notifications for pid 2007 (16)
par défaut	00:45:28.665638 +0200	ReportCrash	Removing excessive log: <private>
par défaut	00:45:28.666790 +0200	ReportCrash	Removing excessive log: <private>
par défaut	00:45:28.667137 +0200	ReportCrash	Removing excessive log: <private>
par défaut	00:45:28.667741 +0200	ReportCrash	Removing excessive log: <private>
par défaut	00:45:28.667844 +0200	ReportCrash	Removing excessive log: <private>
par défaut	00:45:28.668151 +0200	taskgated	MacOS error: -67062

Any hint ?

This has now been fixed in v0.3.8 (brew has not been updated so you need to pull from master).

@koekeishiya i am without my laptop until tonight. If you want to release 3.8, you have write perms on the homebrew repo. Otherwise I can do it as soon as I am home.

@crisidev Hmm says write permission denied. Anyway, I opened a PR atleast so you can just merge that when you get back home.

Weird. I will check as soon as I am on the laptop. In the meantime I merged the PR. Thanks!!

Thanks a lot ! the HEAD version works !


[BUG] (Non-breaking) Design a Report in Power BI Desktop - New UI

Regarding this step:

To convert the slicer from a list to a dropdown, at the top-right of the slicer, select the down-arrow, and then select Dropdown.

The UI has recently changed, so the instructions need to be updated. Please update per the screenshot below.

Image was updated, but not instructions, in last update. Content has been updated now.
"To convert the slicer from a list to a dropdown, navigate to Visualizations > Format visual > Visual > Slicer Settings > Style, and then select Dropdown from the dropdown menu."


Warning of invalid prop type

Hi,
I'm having this warning:
Warning: Failed Context Types: prop type tree is invalid; it must be a Baobab tree. Check the render method of MyApp.
But from the log I can see:
bundle.js:143 MyApp.render()
2016-02-26 15:08:44.309 bundle.js:144   context.tree = undefined
2016-02-26 15:08:44.310 bundle.js:145   props.tree = [object Baobab]

And the way I create the root view is:
var renderApp = function() {
	var root = (<MyApp tree={AppState}/>);
	ReactDOM.render(root, document.querySelector('#myapp'));
};

Any idea of what could be the issue? It does render all the components OK but it is giving this annoying warning... when rendering the root view as well the child views..
Thanks for your help.

Did you use the root mixin or higher order function to provide the tree as context to your React app?

Ah yes, in MyApp I've:
var MyApp= React.createClass({
	mixins: [mixins.root],
...

And in the child components I've for example:
module.exports.LibraryManagerSideBar = React.createClass({
	mixins: [mixins.branch],
...

I don't think it's a problem of baobab-react module but I thought in asking here because someone else could have the same issue before.

Do you happen to tamper with context or use other library relying on context?

Hi,
I'm not using other libraries that rely on context (if I dump the context I see there only .tree).
Later I can try debug it more and see if I find the issue..
Thanks.

If your code is public somewhere I can take a look if you want.

It seems to be something related with the require/browserify flow. I noticed that inside the module where I create the Baobab state tree instanceof is true, but after the require() function return it becomes false.
I've synced a simple test case to https://github.com/jemos/baobab-testcase

I've found the problem.. I didn't knew browserify/require would look up the directory hierarchy when not found the module locally.. so I had duplicate modules in the bundle. My bad. Thanks anyway!

On a side note, maybe I should stop relying on instanceof anyway.


Adding check for nil kv response to avoid runtime panics

Fetching key value pairs would panic if the pair didn't exist, now it returns an error instead in both Watch and Get

+1

Thanks for this PR.


WIP: Move login out of models

models does far too much. In particular it handles all UserSignin.
It shouldn't be responsible for calling LDAP, SMTP or PAM for signing in.
Therefore we should move this code out of models.
This code has to belong on models - therefore it belongs in services.
There is a package in services called auth and clearly this functionality belongs in there.
Plan:

[x] Change auth.Auth to auth.Method - as they represent methods of authentication.
[x] Move models.UserSignIn into auth
[x] Move models.ExternalUserLogin
[x] Move most of the LoginVia* methods to auth or subpackages
[ ] Move the rest of the LDAP functionality in to the ldap subpackage
[ ] Re-factor the login sources to express an interface auth.Source
[ ] Allow wiring in of notify registration -  this can now easily be done
[ ] mawr refactors...

Codecov Report

Merging #16199 (5716bc1) into main (0393a57) will increase coverage by 0.31%.
The diff coverage is 47.37%.

@@            Coverage Diff             @@
##             main   #16199      +/-   ##
==========================================
+ Coverage   44.24%   44.55%   +0.31%
==========================================
  Files         695      706      +11
  Lines       82341    83226     +885
==========================================
+ Hits        36429    37079     +650
- Misses      40014    40134     +120
- Partials     5898     6013     +115

Impacted Files
Coverage Δ

cmd/generate.go
0.00% <0.00%> (ø)

models/oauth2.go
28.00% <0.00%> (-1.17%)
:arrow_down:

modules/context/repo.go
52.99% <0.00%> (-0.30%)
:arrow_down:

modules/generate/generate.go
0.00% <0.00%> (-16.00%)
:arrow_down:

modules/lfs/client.go
100.00% <ø> (ø)

modules/migrations/dump.go
0.00% <0.00%> (ø)

modules/setting/lfs.go
52.94% <0.00%> (ø)

modules/setting/repository.go
57.69% <ø> (ø)

modules/structs/user.go
100.00% <ø> (ø)

modules/templates/helper.go
49.74% <0.00%> (-1.03%)
:arrow_down:

... and 129 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6667f52...5716bc1. Read the comment docs.

OK I think that's enough for now.
Questions:

Do we think the source subdirectory is required or should things be moved back done to subdirectories of auth directly?
services/auth/source/ldap.Source has a weird wrapper in-order to keep backward compatibility. This could be dropped at the cost of a migration.
models.LoginSource.IsActived should be renamed to IsActive but this would require a migration.

services/auth/source/ldap.Source has a weird wrapper in-order to keep backward compatibility. This could be dropped at the cost of a migration.

I would prefer migration to keep code more clean

migration added.

updated to include #16465

make lgtm work


Uncaught DOMException: Failed to execute 'removeChild' on 'Node': The node to be removed is not a child of this node.

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to any page with editor
Click on browser`s feature "Translate to" and translate to any language (tested with Czech)
Click on the toolbar actions, such as img, link, table.
Pop up will open and after few click it all stops to work.

Expected behavior
Translating a web page should not lead to a editor break down
Screenshots

Desktop (please complete the following information):

OS: MAC OS
Browser: Chrome
Version 99.0.4844.83 (Official Build) (arm64)

@zdarovka
Thank you for the issue.
I think we need to find out how the translator works. However, I think it would be better to add Czech to the i18n.
Do you have any thoughts to contribute?
refs: https://github.com/nhn/tui.editor/tree/master/apps/editor/src/i18n

I could definitely contribute, but that won't solve the problems as using a translator would still break it.

I try to check it. Now I guess the translator could affect our toolbar's DOM architecture while translating.


bump SDG minimum version

SDG 0.5.0 is out, 0.21 will be based off of this, bump the version

Checklist:

[ ] Commit Message Formatting: Commit titles and messages follow guidelines in the
conventional commits.
[ ] Changelog updated with breaking and/or notable changes for the next minor release.
[ ] Documentation has been updated, if necessary.
[ ] Unit tests have been added, if necessary.
[ ] Functional tests have been added, if necessary.
[ ] E2E Workflow tests have been added, if necessary.

lets see if the dependency tests pass here

Please run the large E2E run against this as well, thanks!


SPM doesn't enumerate source files properly in different configurations

Description
When building https://github.com/thebrowsercompany/swift-composable-architecture in release mode, the generated OFM for OpenCombine is invalid, e.g.
Building for production...
<unknown>:0: error: supplementary output file map 'C:\\Users\\abdulras\\AppData\\Local\\Temp\\TemporaryDirectory.T3w0D9\\supplementaryOutputs-1' is missing an entry for 'S:\\SourceCache\\thebrowsercompany\\swift-composable-architecture\\.build\\checkouts\\OpenCombine\\Sources\\OpenCombine\\AnyCancellable.swift' (this likely indicates a compiler issue; please submit a bug report (https://swift.org/contributing/#reporting-bugs))

Note that building OpenCombine directly does not exhibit this issue.
Expected behavior
The internal generated state would be correct.
Actual behavior
The internal generated state is incorrect.
Steps to reproduce

git clone https://github.com/thebrowsercompany/swift-composable-architecture
cd swift-composable-architecture
git checkout develop
swift build -c release

Swift Package Manager version/commit hash
Swift Package Manager - Swift 5.10.0-dev
Swift & OS version (output of swift --version ; uname -a)
Swift version 5.11-dev (LLVM<PHONE_NUMBER>5130e, Swift 4bcc25e99094074)
Target: x86_64-unknown-windows-msvc

FYI, we're seeing this issue building a Swift codebase for Windows with Swift 5.9.1. We're seeing the same issue when building for Linux and Android. It is a library that depends on a stack of others: C -> B -> A -> Swift runtime. B & A alone build just fine, and one configuration of C does as well (for macOS/iOS & Wasm). All together, they have ~40k lines of Swift and a few hundred files.
Any movement on this? Anyhow I could help by testing against our codebase?

Also, looking for more info on this, I just tried building swift-composable-architecture on Windows 11 with Swift 5.9.1 and encountered the same issue as @compnerd

@compnerd we've hit this bug in a few more contexts, each time it is when the command line being passed to swift-frontend.exe is long enough that it is being passed as a file. The problem likely lies there. Is there any debug logging I can provide to help with the fix?

I hit this with a module containing ~117 files (still below the default threshold) and attached a debugger to stop when swift-frontend is being invoked. This is the supplementary output file map on disk:
"D:\\swift-winrt\\build\\debug\\Generator\\InteropTests\\WinRTComponent\\Projection\\Sources\\WinRTComponent\\Projection\\WinRTComponent\\Arrays.swift":
  dependencies: "D:\\swift-winrt\\build\\debug\\Generator\\InteropTests\\WinRTComponent\\Projection\\Build\\WinRTComponent\\Projection\\WinRTComponent.d"
  swiftsourceinfo: "D:\\swift-winrt\\build\\debug\\Generator\\InteropTests\\WinRTComponent\\Projection\\Build\\WinRTComponent\\Projection\\WinRTComponent.swiftsourceinfo"
  swiftmodule: "D:\\swift-winrt\\build\\debug\\Generator\\InteropTests\\WinRTComponent\\Projection\\Build\\WinRTComponent\\Projection\\WinRTComponent.swiftmodule"
  swiftdoc: "D:\\swift-winrt\\build\\debug\\Generator\\InteropTests\\WinRTComponent\\Projection\\Build\\WinRTComponent\\Projection\\WinRTComponent.swiftdoc"

From code inspection, it looks like it was supposed to be json, but someone ate the curly braces.
And the compiler says:
<unknown>:0: error: supplementary output file map 'C:\\Users\\tristan\\AppData\\Local\\Temp\\TemporaryDirectory.GHm1Rw\\supplementaryOutputs-1' is missing an entry for 'D:\\swift-winrt\\build\\debug\\Generator\\InteropTests\\WinRTComponent\\Projection\\Sources\\WinRTComponent\\Projection\\WinRTComponent\\Arrays.swift' (this likely indicates a compiler issue; please submit a bug report (https://swift.org/contributing/#reporting-bugs))

Curiously that Array.swift is the only one present in the supplementary output file map.

Ah never mind, it's yaml. Then I don't see the problem, but here are the files attached.
arguments-2560280406336689506.resp.txt
supplementaryOutputs-1.txt


Couldn't download error

When trying to download the images, the following error is returning me, the prompts are being generated normally.
couldn't download https://cdn.discordapp.com/attachments/link...: discord: couldn't do request https://cdn.discordapp.com/attachments/link...: Get "https://cdn.discordapp.com/attachments/link....png": context canceled

I'm getting this error today.  Why is this issue closed down?  Or what were you and am I doing wrong to get this now?

In my case, after a few hours it returned to normal.

Had it last night and today, went away mysteriously this morning... So this must be something with Midjourney.
I did notice that the URL that failed when it was failing and the URL that would open up from the Discord interface were different.  If your fail URL was
https://cdn.discordapp.com/attachments/link....png
Then the URL that MJ would generate when you "open in Browser" would be
https://cdn.discordapp.com/attachments/link....png?ex=65d64af0&is....etc.
So there's extra code at the end.I have no idea what it is or what it means, but I thought I'd report it.

Some changes have been added to deal with this.
See: #79 #80 #82


CORTX-28870: Adding the allowable list for Alex scan

Signed-off-by: Venkatesh K<EMAIL_ADDRESS>Problem Statement
Alex scan generate false positive
Design
We have added some allowable list to ignore the false positives
Coding
Checklist for Author

[x] Coding conventions are followed and code is consistent

Testing
Checklist for Author

[ ] Unit and System Tests are added
[ ] Test Cases cover Happy Path, Non-Happy Path and Scalability
[ ] Testing was performed with RPM

Impact Analysis
Checklist for Author/Reviewer/GateKeeper

[ ] Interface change (if any) are documented
[ ] Side effects on other features (deployment/upgrade)
[ ] Dependencies on other component(s)

Review Checklist
Checklist for Author

[x] JIRA number/GitHub Issue added to PR
[x] PR is self reviewed
[x] Jira and state/status is updated and JIRA is updated with PR link
[x] Check if the description is clear and explained

Documentation
Checklist for Author

[ ] Changes done to WIKI / Confluence page / Quick Start Guide

Due to issue in CLA sign, I have created new PR.
https://github.com/Seagate/cortx-rgw-integration/pull/51


MDS 4.0+ Fails in .NET 4.8 when using AG (FailoverPartner)

Describe the bug
A clear and concise description of what the bug is.
If you are seeing an exception, include the full exceptions details (message and stack trace).
Exception message:
Stack trace:
System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary. at System.ThrowHelper.ThrowKeyNotFoundException()
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at Microsoft.Data.SqlClient.SqlConnectionPoolGroupProviderInfo.CreateFailoverPermission(SqlConnectionString userConnectionOptions, String actualFailoverPartner)
in D:\a\_work\1\s\src\Microsoft.Data.SqlClient\src\Microsoft\Data\SqlClient\SqlConnectionPoolGroupProviderInfo.cs:line 107
at Microsoft.Data.SqlClient.SqlConnectionPoolGroupProviderInfo.FailoverCheck(Boolean actualUseFailoverPartner, SqlConnectionString userConnectionOptions, String actualFailoverPartner)
in D:\a\_work\1\s\src\Microsoft.Data.SqlClient\src\Microsoft\Data\SqlClient\SqlConnectionPoolGroupProviderInfo.cs:line 66
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)
....

It looks like we are checking for existence of the FailoverPartner keyword by trying to access a dictionary and seeing if it returns null here?
To reproduce
var csb = new Microsoft.Data.SqlClient.SqlConnectionStringBuilder()
{
	DataSource = db.ServerName,
	InitialCatalog = db.DBname,
	TrustServerCertificate = true,
	IntegratedSecurity = true,
};

using (var c = new Microsoft.Data.SqlClient.SqlConnection(csb.ToString()))
{
	c.Execute("SELECT TOP 1 1 FROM dbo.Entity");
}

Expected behavior
Should be able to connect when database is in an AG.
Further technical details
Microsoft.Data.SqlClient version: 4.0
.NET target: net48
SQL Server version: SQL Server 2019
Operating system: Windows 2019

Hi @cmeyertons, I'll take a look since I did merge to this file into the share src codebase.

@cmeyertons where did you provide the failover partner?

We are not providing the failover partner in our connection string, just the Data Source and Initial Catalog.
Our data source is the the direct server name, not AG listener, due to some quirks in how our setup is used.  Our DBA has said this should be fine, and it is fine in our code base using System.Data.SqlClient -- this behavior only crops up in MDS.

@lcheunglci  @JRahnama well I discovered a very hack-y workaround...
var csb = new Microsoft.Data.SqlClient.SqlConnectionStringBuilder()
{
	DataSource = db.ServerName, // the raw server name, not AG listener
	InitialCatalog = db.DBname,
	TrustServerCertificate = true,
	IntegratedSecurity = true,
        FailoverPartner = string.Empty, // <- HACK: workaround
};

@cmeyertons, glad to hear you have a workaround.

Also experiencing this on AWS RDS instances in multi-az mode - not sure how they have it set up but it seems to be the same issue.

Closing this as the PR #1614 was merged. Feel free to comment or open a new issue if you are still experiencing the problem.


How to sign

How do I get a signature from the private key and the transaction information? thank you

Probably this piece of code will help you
`
var assembleTX = try your_transaction.assemble()
try Web3Signer.EIP155Signer.sign(transaction: &assembleTX, privateKey: your_private_key)
`

I need you to provide an ethereum signature method through custom message and private key, hope to give an example, but your reply function does not exist, thank you!
------------------ 原始邮件 ------------------
发件人:                                                                                "skywinder/web3swift"                                                                                @.>;
发送时间: 2022年6月10日(星期五) 凌晨0:44
@.>;
<EMAIL_ADDRESS>主题: Re: [skywinder/web3swift] How to sign (Issue #580)
Probably this piece of code will help you
`
var assembleTX = try your_transaction.assemble()
try Web3Signer.EIP155Signer.sign(transaction: &assembleTX, privateKey: your_private_key)
`
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

you need to provide more information. What have you tried (post code). Are you trying to get the signature of a transaction that is already on chain, or a transaction that you are newly creating?
The method posted earlier is based on an older version for the library, for signing a transaction.

@dengbangquan
There is Web3Signer with two static functions that should fit. The first one is for signing transactions and the second if for signing Data objects:
Web3Signer.signTX(transaction: &transaction, ...)
Web3Signer.signPersonalMessage(someDataObject, ...)

Related to this question, how can I get the Signature Hash of a signed message?
`        let mnemonic = "..."
let keystore = try! BIP32Keystore(mnemonics: mnemonic)
let keystoreManager = KeystoreManager([keystore!])
    let mainnet = Web3.InfuraMainnetWeb3()
    mainnet.addKeystoreManager(keystoreManager)

    let address = keystoreManager.addresses![0]

    let msgStr = "This is my first Ethereum App";
    let data = msgStr.data(using: .utf8)

    do {
        let signMsg = try mainnet.wallet.signPersonalMessage(data!, account: address);
        print("???")
    } catch (let error) {
        print(error)
    }`

Related to this question, how can I get the Signature Hash of a signed message?
    let mnemonic = "..."
    let keystore = try! BIP32Keystore(mnemonics: mnemonic)
    let keystoreManager = KeystoreManager([keystore!])

    let mainnet = Web3.InfuraMainnetWeb3()
    mainnet.addKeystoreManager(keystoreManager)

    let address = keystoreManager.addresses![0]

    let msgStr = "This is my first Ethereum App";
    let data = msgStr.data(using: .utf8)

    do {
        let signMsg = try mainnet.wallet.signPersonalMessage(data!, account: address);
        print("???")
    } catch (let error) {
        print(error)
    }

@dengbangquan有Web3Signer，有两个静态功能，应该适合。第一个用于签署交易，第二个用于签署Data对象：
Web3Signer.signTX(transaction: &transaction, ...)
Web3Signer.signPersonalMessage(someDataObject, ...)

In this signPersonalMessage method, I need to build attachedKeystoreManager, but I only have one address. There is no way to build attachedKeystoreManager. Is there any other way to sign?


Windows iree-compiler crash with fp32 stable diffusion

What happened?
Compiling the attached .mlir will crash the compiler with errors like
xyz.str:3206:12: error: failed to run translation of source executable to target executable for backend #hal.executable.target<"llvm-cpu", "embedded-elf-x86_64", {cpu_features = "-avx512pf,-tsxldtrk,+cx16,+sahf,-tbm,-avx512ifma,+sha,+crc32,-fma4,-vpclmulqdq,+prfchw,+bmi2,-cldemote,+fsgsbase,-avx512bf16,-amx-tile,-raoint,-uintr,-gfni,+popcnt,-ptwrite,+aes,-avx512bitalg,-movdiri,-widekl,+xsaves,-avx512er,-avxvnni,-avx512fp16,-avx512vnni,-amx-bf16,-avxvnniint8,-avx512vpopcntdq,-pconfig,+clwb,-cmpccxadd,-avx512f,+xsavec,+clzero,-pku,-amx-fp16,+mmx,-lwp,+rdpid,-xop,+rdseed,-waitpkg,-prefetchi,-kl,-movdir64b,+sse4a,-avx512bw,-avxneconvert,+clflushopt,+xsave,-avx512vbmi2,+64bit,-avx512vl,-serialize,-hreset,-invpcid,-avx512cd,+avx,-vaes,-amx-int8,+cx8,+fma,-rtm,+bmi,-enqcmd,+rdrnd,+mwaitx,+sse4.1,+sse4.2,+avx2,+fxsr,+wbnoinvd,+sse,+lzcnt,+pclmul,+rdpru,-avxifma,+f16c,+ssse3,-sgx,-prefetchwt1,+cmov,-avx512vbmi,-shstk,+movbe,-avx512vp2intersect,+xsaveopt,-avx512dq,+sse2,+adx,+sse3", data_layout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128", native_vector_size = 32 : index, target_triple = "x86_64-unknown-unknown-eabi-elf"}>
xyz.str:25:3: note: called from
xyz.str:3206:12: note: see current operation:
"hal.executable.variant"() ({
  "hal.executable.export"() ({
  ^bb0(%arg0: !hal.device, %arg1: index, %arg2: index, %arg3: index):
    %0 = "arith.constant"() {value = 32 : index} : () -> index
    %1 = "arith.constant"() {value = 1 : index} : () -> index
    "hal.return"(%0, %1, %1) : (index, index, index) -> ()
  }) {layout = #hal.pipeline.layout<push_constants = 2, sets = [<0, bindings = [<0, storage_buffer, ReadOnly>, <1, storage_buffer>]>]>, ordinal = 0 : index, sym_name = "forward_dispatch_182_generic_16x1024x1024", translation_info = #iree_codegen.translation_info<CPUDoubleTilingExpert>} : () -> ()
  "builtin.module"() ({
    "func.func"() ({
      %0 = "arith.constant"() {value = dense<0.693147182> : vector<1x8x8xf32>} : () -> vector<1x8x8xf32>
      %1 = "arith.constant"() {value = dense<1.44269502> : vector<1x8x8xf3
      ```

### Steps to reproduce your issue

[save_temps_fp32.zip](https://github.com/iree-org/iree/files/10171714/save_temps_fp32.zip)

Run with ```
(shark.venv) PS C:\g\SHARK\save_temps> C:\g\shark\shark.venv\lib\site-packages\iree\compiler\tools\..\_mlir_libs\iree-compile.exe --iree-input-type=none --iree-vm-bytecode-module-output-format=flatbuffer-binary --iree-hal-target-backends=llvm-cpu --iree-llvm-embedded-linker-path=C:\g\shark\shark.venv\lib\site-packages\iree\compiler\tools\..\_mlir_libs\iree-lld.exe --iree-llvm-target-cpu-features=host -iree-llvm-target-triple=x86_64-pc-windows-msvc --iree-stream-resource-index-bits=64 --iree-vm-target-index-bits=64 --iree-util-zero-fill-elided-attrs --iree-flow-enable-padding-linalg-ops --iree-flow-linalg-ops-padding-size=16 .\core-reproducer.mlir -o core.vmfb

What component(s) does this issue relate to?
No response
Version information
No response
Additional context
No response

@hanhanW Do you know if this is still active?

@powderluv Bumping this one up - still active?


Using bert for Document Classification

How can i use BERT to fine tune for document classifications? Has anyone implemented it? Any example or lead would be really helpful. I want to use it for document which are way bigger than current max length(512 tokens).

Have you considered splitting the document into chunks of 512 tokens and then using the most common classification as the final classification?

Have you considered splitting the document into chunks of 512 tokens and then using the most common classification as the final classification?

Do you have a reference paper or blog link explains why? thx

@EncoreOliver This is a common workaround in text classification. Besides chunking the document people have also suggested summarizing the document into a length that the model can process. I can't think of a paper/blog off the top of my head right now.

@EncoreOliver https://arxiv.org/abs/1905.05583 The authors recommend using head and tail tokens of the document. I think that how to chunk documents depends on datasets.

@hsm207 @stevewyl Thank you guys for your insights on this topic! I put together a tutorial and a Colab notebook on applying BERT to document classification here: https://youtu.be/_eSGWNqKeeY, and credit this thread for some of the ideas (at around [13:20]) :)

can I get the colab notebook link


Convert docs to Markdown format

Changes:
docs(idea.md) style improvements

Fix typographic errors.
Improve readability.

docs(README.md) convert HTML to Markdown

Convert HTML syntax to Markdown syntax.
Improve readability of the Markdown text file.

docs(AddConfigurations.md) update structure

Convert HTML syntax to Markdown syntax.
Improve readability of the Markdown text file.

docs(BatchTask.md) update structure

Convert HTML syntax to Markdown syntax.
Improve readability of the Markdown text file.

docs(Changelog.md) update structure

Convert HTML syntax to Markdown syntax.
Improve readability of the Markdown text file.
Fix small writing error.

docs(CopySingleFiles.md) update structure

Convert HTML syntax to Markdown syntax.
Improve readability of the Markdown text file.

docs(DIYNAS.md) update structure

Convert HTML syntax to Markdown syntax.
Improve readability of the Markdown text file.

docs(HowtoUseRsyncOSX.md) update structure

Convert HTML syntax to Markdown syntax.
Improve readability of the Markdown text file.

docs(Changelog.md) update to match latest changes

Update to match latest changes of Git commit 112a10d

docs(README.md) fix changes

Manually merge changes made in commit 112a10d

@rsyncOSX I did some documentation file cleanup by converting these to Markdown syntax.
You had made some changes (3 commits) at the same time, so several of the files have merge conflicts. More details from GitHub’s own docs about merging of pull requests might help in this situation.


[tool_crash] FileSystemException: Cannot create link, OS Error: Acceso denegado., errno = 5

Command
flutter pub get

Steps to Reproduce

...
...
...

Logs
FileSystemException: Cannot create link, OS Error: Acceso denegado.
, errno = 5
#0      _Link.throwIfError (dart:io/link.dart:263:7)
#1      _Link.createSync (dart:io/link.dart:184:5)
#2      ForwardingLink.createSync (package:file/src/forwarding/forwarding_link.dart:20:16)
#3      ForwardingLink.createSync (package:file/src/forwarding/forwarding_link.dart:20:16)
#4      _createPlatformPluginSymlinks (package:flutter_tools/src/flutter_plugins.dart:1061:12)
#5      createPluginSymlinks (package:flutter_tools/src/flutter_plugins.dart:996:5)
#6      refreshPluginsList (package:flutter_tools/src/flutter_plugins.dart:1087:5)
<asynchronous suspension>
#7      FlutterProject.ensureReadyForPlatformSpecificTooling (package:flutter_tools/src/project.dart:359:5)
<asynchronous suspension>
#8      PackagesGetCommand.runCommand (package:flutter_tools/src/commands/packages.dart:177:5)
<asynchronous suspension>
#9      FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1183:27)
<asynchronous suspension>
#10     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#11     CommandRunner.runCommand (package:args/command_runner.dart:209:13)
<asynchronous suspension>
#12     FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:281:9)
<asynchronous suspension>
#13     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#14     FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:229:5)
<asynchronous suspension>
#15     run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:62:9)

[✓] Flutter (Channel stable, 3.0.1, on Microsoft Windows [Versi¢n 10.0.22000.675], locale es-ES)
    • Flutter version 3.0.1 at C:\portables\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision fb57da5f94 (2 weeks ago), 2022-05-19 15:50:29 -0700
    • Engine revision caaafc5604
    • Dart version 2.17.1
    • DevTools version 2.12.2

[!] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at C:\Users\Enrique de la Calle\AppData\Local\Android\sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/windows#android-setup for more details.

[✓] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[✓] Visual Studio - develop for Windows (Visual Studio Community 2019 16.11.3)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.11.31702.278
    • Windows 10 SDK version 10.0.19041.0

[✓] Android Studio (version 2020.3)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[✓] VS Code
    • VS Code at C:\Users\Enrique de la Calle\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.42.0

[✓] VS Code, 64-bit edition (version 1.67.2)
    • VS Code at C:\Program Files\Microsoft VS Code
    • Flutter extension version 3.42.0

[✓] Connected device (3 available)
    • Windows (desktop) • windows • windows-x64    • Microsoft Windows [Versi¢n 10.0.22000.675]
    • Chrome (web)      • chrome  • web-javascript • Google Chrome 102.0.5005.63
    • Edge (web)        • edge    • web-javascript • Microsoft Edge 102.0.1245.30

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

Flutter Application Metadata
Type: app
Version: 1.0.0+1
Material: true
Android X: false
Module: false
Plugin: false
Android package: null
iOS bundle identifier: null
Creation channel: stable
Creation framework version: fb57da5f945d02ef4f98dfd9409a72b7cce74268
Plugins
path_provider-2.0.10
path_provider_android-2.0.14
path_provider_ios-2.0.9
path_provider_linux-2.1.7
path_provider_macos-2.0.6
path_provider_windows-2.1.0

Closing this as duplicate of https://github.com/flutter/flutter/issues/96568


Dynamic sets

This PR closes #355 and #347 - but before it will be merged, it will be tested in a live application.

Well, the database usage works well - inserting 20,000 rows in a single transaction is SO fast compared to doing each insert individually!

I need to introduce a new dynamic set type: "or" sets.
Example:
select
    id,
    artistId,
    title,
    type
from
    Product
where
(
    (title = 'Product A' and artistId = 'Artist 1') or
    (title = 'Product B' and artistId = 'Artist 1') or
    (title = 'Product C' and artistId = 'Artist 2')
)

This example is similar to where ... in but allows for multi-field matching.

Potential bug. I don't know for sure whether it's the responsibility of the dynamic sets to fix, but here's what I've found.
In my application, I'm using array_diff on a list of data and existing data from the database, so I can get an array full of only the new records that need inserting.
Because array_diff preserves keys, the data I end up passing to the __dynamicValueSet has keys that aren't sequential, which seems to mess things up. I can fix it by passing array_values($myData), but should the dynamic value setting normalise this on my behalf?

Something else to chew over...
insert into Product (
  id,
  artistId,
  title
)
values ( :__dynamicValueSet )

The contents of dynamicValueSet is dynamic (well, durr). In my application, I'm building it up within a loop, and the ID is the last thing I set, meaning that the columns are no longer in the same order as how they're written in the SQL.
The SQL's column order needs to match the order of the indices in the associative arrays, otherwise bad things will happen.
I'm not sure how to solve this. Is there another insert syntax to use where each field can be set individually, or is it an OK problem to have>


Fetch current user during ActiveAdmin.register

I would like to show/hide actions for a resource according to the users privileges. Therefore I integrated my own authorization library into ActiveAdmin.
Now I would like to do something like that during the registration:
ActiveAdmin.register Privilege do
actions *([:index, :show, :new].reject{|action| !current_user.can?("admin/privileges", action.to_s) })
end
How can I access the current user inside the context of ActiveAdmin::DSL or inside main context? Is there another way to show/hide resource actions in relation to my users privileges?
Regards
Jan

I solved this problem in a few different ways. First, I created a active admin view helper to generate the links for the actions column on the table listing that were limited to the user permissions. Next, I used the cancan ability class to define the user permissions.
Finally, a call to controller.authorize_resource locked user out of actions that they weren't allowed to access.
There's another issue here on active admin that showed how to hack the action items module of resource controller to optionally display the action items based on user permission roles.

Unfortunately it's impossible for the DSL to give you access to the current user without it parsing the entire Active Admin configuration on each request. For this reason, most of the DSL methods accept a :if option. You can pass a proc to these options which will be called at runtime (with access to current user, etc).
Instead of calling the #actions method, consider creating before filters that raise exceptions when the current user is not authorized.
There is very good documentation on the wiki for integrating with CanCan, which should be of help to you.


Skip gpx files in incremental original file export mode

The original unzipped file might also be a GPX file. Therefore also skip the download of the original zip if either the corresponding FIT or GPX file exists.
This is especially useful if you use gcexport to update an existing local backup of all your originals.

@reto PR merged. Thanks for contributing. Sorry about the extremely long delay...

Hi @fjgal Thanks.


sap.m.IconTabBar.setTabDensityMode has incorrect type for parameter

OpenUI5 version: 1.106.0
Browser/version (+device/version): Chrome (latest)
Steps to reproduce the problem:

The docs and the typescript interface require incorrect type for the 'mode' parameter in the IconTabBar.setTabDensityMode method.

What is the expected result?
The required parameter type should be IconTabDensityMode as it is used in here.
What happens instead?
It uses sap.m.IconTabHeaderMode instead.

Hello @kemenesjanos,
Thank you for sharing this finding. I've created an internal incident<PHONE_NUMBER>. The status of the issue will be updated here in GitHub.
Regards,
Diana

Hello @kemenesjanos,
The issue is fixed with 5a25c4e7424f2a609533da7f96628eea6497762b
The fix will be available with SAPUI5 1.109 version.
Regards,
Konstantin


Add Class Image

Related Issue
Allow creating a class image
Close :  #135
Describe the changes you've made
As I mentioned in issue #135, we should add a class image, So I implement this feature by adding multer npm in the backend, you can check the full video below
Checklist:

[ ✔] My code follows the style guidelines of this project.
[✔ ] I have performed a self-review of my own code.
[ ✔] I have commented on my code, particularly in hard-to-understand areas.
[ ✔] I have made corresponding changes to the documentation.
[ ✔] My changes generate no new warnings.

Screenshots
https://user-images.githubusercontent.com/53515714/113410201-762be180-93d0-11eb-83d4-2e009471757f.mp4

Please add a default image for background, if someone doesn't want to add images and don't make it necessary.

@aavishkarmishra Oh yes, I forgot this point, ok no problem, Now check it please
https://user-images.githubusercontent.com/53515714/113413689-db83d080-93d8-11eb-8baa-33867d907e53.mp4

@Vidisha-kayal  Hey can you please add GSSOC level and label in this and #69 PR

@Vidisha-kayal Hey can you please add GSSOC level and label #69 PR

@nishant23122000 #69 is closed, so adding labels will not make any effect.

@aavishkarmishra  sorry , my mistake . I just show this PR and it's older version of PR #72


Coordinate with Dataverse upstream to incorporate optimizations

The Dataverse main software uses this library to create an OAI-PMH server. To achieve maximum performance, they kind of forked some of the classes this library provides, copying most of the code.
As this is now a fork under the control of GDCC, it would make sense to change the library classes to provide useful extension points that Dataverse codebase could use then to avoid maintaining forked classes.
@landreev WDYT?
The respective Dataverse classes can be found here:

edu.harvard.iq.dataverse.harvest.server.xoai

Related:

https://github.com/IQSS/dataverse/issues/8372


Inkscape optimization via scour more efficient than standalone scour?

I compared the optimization results of Inkscape with the the output of
scour -i sprechakt_plain.svg -o sprechakt_optimized_scour.svg --enable-viewboxing --enable-id-stripping --enable-comment-stripping --shorten-ids --indent=none  --remove-descriptive-elements --no-line-breaks

and somehow the scour output is always a little larger than Inkscape's (although I selected options identical to the parameters listed above.
These are the files:

Why are the files different, and how can I configure scour to produce the exact same output as Inkscape (which uses scour anyway)?

Sorry! I needed to add the --create-groups parameter – now the output is identical.


[HOLD for payment 2023-10-30] [500] Workspace - Settings -  Can't select currency item text

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: <IP_ADDRESS>
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @Krishna2323
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1697421082877669
Action Performed:

Open settings > workspaces > select workspace > settings > default currency
Try to highlight text of any currency item

Expected Result:
ext should be selectable.
Actual Result:
Text is not selectable.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos

Android: Native

https://github.com/Expensify/App/assets/93399543/bd879051-4ee4-44cf-92c4-9b819440d8bc

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

Windows: Chrome

https://github.com/Expensify/App/assets/93399543/8d224bf6-4f30-4345-b58b-70a417346db6
https://github.com/Expensify/App/assets/93399543/08115047-3792-401a-a86c-6a65bbe70ac6

MacOS: Desktop

https://github.com/Expensify/App/assets/93399543/26dcaeb7-a7d1-411c-ac56-523db4346a05

View all open jobs on GitHub
Issue OwnerCurrent Issue Owner: @Christinadobrzyn

Proposal
Please re-state the problem that we are trying to solve in this issue.
Can't select currency item text
What is the root cause of that problem?
The root cause of this issue is that the workspace currency setting select page use SelectionList
https://github.com/Expensify/App/blob/ebf8306fe2829f6292bf5245f96829b1e31483a5/src/pages/workspace/WorkspaceSettingsCurrencyPage.js#L90-L99
and the BaseListItem has select-none style
https://github.com/Expensify/App/blob/ebf8306fe2829f6292bf5245f96829b1e31483a5/src/components/SelectionList/BaseListItem.js#L51
So, we can select currency item text from workspace setting currency page.
But from the Select a currency page of request money flow, we're using OptionsSelector to display the currency list
https://github.com/Expensify/App/blob/ebf8306fe2829f6292bf5245f96829b1e31483a5/src/pages/iou/IOUCurrencySelection.js#L142-L154
which is allowed to select text.
What changes do you think we should make in order to solve the problem?
As the SelectionList is newer, I think we can let Select a currency page use SelectionList as well. Then we can have consistent behaviour.
If we want the use to select text, then we can consider removing select-none style from BaseListItem.
What alternative solutions did you explore? (Optional)
N/A

@Christinadobrzyn I found that selection lists use the new implementation of SelectionList, like Workspace currency setting, Pronouns, Timezone, Year and Country, are all not allowed to select text of items. So the IOU currency selection list can also be same with them right?
cc @pecanoro @narefyev91

Hi @eh2077 I'm not quite sure the answer to that. I think that is true but @pecanoro and @narefyev91 would probably know more about the coding.

I can reproduce this but I don't know why we would want to be able to highlight a currency (ex. drag the cursor over a currency to highlight it). The currency is designed to be clicked/tapped - not highlighted.
I don't think this is a bug or something to fix.
@narefyev91 and @pecanoro what are your thoughts?

Although with manual requests, you can highlight the currency so this is a little inconsistent between the two methods to Request money (as mentioned in the chat with @Krishna2323  https://expensify.slack.com/archives/C049HHMV9SM/p1697567221998929?thread_ts=1697421082.877669&cid=C049HHMV9SM)

@Christinadobrzyn Thanks for your input!
Yes, It's clear we only need to fix the inconsistency of manual requests. C+ team @narefyev91 suggested same in their comment https://github.com/Expensify/App/issues/29685#issuecomment-1765801466

@Christinadobrzyn yup the main issue here that we have inconsistence between currency selection in workspace setting and request money flows. We need to following the same way for both. In case that new implementation used SelectionList - my suggestion only to update it for currency selection, and do not touch any other places

@narefyev91 I agree with you! Assigning @eh2077 so we can keep both consistent

Payouts due:
Issue Reporter: $50 @Krishna2323 (in Upwork)
Contributor: $500 + $250 @eh2077
Contributor+: $500 + $250 @narefyev91
Eligible for 50% #urgency bonus? Y - looks like the PR was created on Oct 18th and merged on Oct 19
Upwork job is here.
@eh2077 can you create a regression test?

nudge @eh2077 for a regression test when possible! Thanks!

Regression test

Click FAB and choose Request money
Select Manual tab
Click to change the default currency
Verify that Select a currency page is open and the default currency is highlighted and has a green check mark on the right side
On web or desktop, try to drag and select currency item text
Verify that the text is not selectable
Search a currency USD and select it
Verify the page is redirected to the Request money page with currency changed to USD
Enter amount and split money with someone
Verify that it's able to split the amount money in selected currency with someone

@Christinadobrzyn Sorry for late. Added regression test 👆

Awesome! Thanks @eh2077! Created the regression test and paid out based on this payment structure. Closing!


support API "GET /event_types/{uuid}"

As of August 2020, following api hasn't been released.
After it be release, this library plan to support it.
ENDPOINT

GET /event_types/{uuid}

I released v0.4.1.


Integration best practice

What is the best place to invoke e.g. blankshield($('a[target=_blank]')); ?
To have all html parsed / DOM created by then I guess it should be done in document.onready.
But I wonder if its possible a user would click the unsafe link before its fixed (before the whole document is loaded and js run)?
Also at the moment the library doesn't support custom "target" attributes, e.g. target="MyNewWindow" - it'll open a new tab that will be vulnerable to the attack - I'm going to work on a patch for this issue.

I've just been using document ready. On a page with 115 requests and 7MB of assets, DOMContentLoaded fired at 1.32s. However, the user may not be looking at anything for part of that 1.32s. So in the head of the page, I added: var startTime = Date.now(), and in the footer,
$(document).ready(function() {
  console.log('ready', Date.now() - startTime);
});
document.addEventListener('DOMContentLoaded', function() {
  console.log('DOMContentLoaded', Date.now() - startTime);
});

// ready 747
// DOMContentLoaded 771

So under a second. I think it's pretty safe to say that a user won't be able to recognize and consciously take action on a link as it's being rendered in that sort of time. For DOMContentLoaded to take a while, you'd have to be rendering MB of just HTML elements.
I might add a function that uses DOMNodeInserted and MutationObserver to handle anchors that are later added to the DOM. But unfortunately, neither would handle anchors that exist in the original markup.