Added EMPD2 samples contributed by Kolaczek

Hi! I'm your friendly automated EMPD-admin bot!
Thank you very much for your data submission! I will now run some tests on your data.
In the meantime: please review your test data on the EMPD viewer using
this link: https://EMPD2.github.io/?repo=EMPD2/EMPD-data&branch=Kolaczek&meta=Kolaczek.tsv
This bot is helping you merging your data. I am checking for common issues and I can fix your climate, country or elevation data.
Type @EMPD-admin help in a new comment in this PR for usage information.
Make sure, that all the commands for the @EMPD-admin are on one single line and nothing else. Only
@EMPD-admin help
will work, not something like
<other words> @EMPD-admin help
or
@EMPD-admin help <other words>
Note that you can also run these tests on your local machine with
pytest.

Hi! I'm your friendly automated EMPD-admin bot!
This is just to inform you that I tested your data submission in your PR (Kolaczek.tsv) and found it in an excellent condition!

@EMPD-admin finish --commit

Hi! I'm your friendly automated EMPD-admin bot!
I processed your command and hope that I can help you!

finish --commit

Finished the PR!
You may want to have a final look into the viewer (https://EMPD2.github.io/?repo=EMPD2/EMPD-data&branch=Kolaczek) and then merge it.


Using unsafe routes with Windows or macOS exit node

I read and tested this blog post but I'm in a bit of a different situation. I want my exit nodes to be Windows and macOS machines, rather than a Linux machine. The blog post required some special configuration of iptables on the Linux exit node. Is that same configuration required when using a Windows and macOS exit nodes, or is this not possible with nebula?

Nebula's unsafe routes feature can get the packets from clients to the unsafe_routes Nebula node. However, it depends on the kernel to perform NAT and get the packets to their final destination - which also ensures return traffic gets back to the unsafe_routes node so that they can be forwarded along to the originating Nebula peer.
Windows has a feature called Internet Connection Sharing that might do what you need, but I'm not sure, and haven't tested it.
I couldn't find anything on MacOS.
The feature was designed with Linux in mind.
If you share more about your goals, we might be able to find another way to resolve your issue. Why do you want to use Windows and MacOS clients as unsafe_routes nodes?

@brad-defined thanks for the response. Our goal is to use Nebula to temporarily expose a route to specific internal endpoints periodically throughout the day so that we can remotely collect data from them using a wide variety of networking protocols (SNMP, HTTP, etc). Our agents are cross-platform and most of them are running on Windows devices. In order for Nebula to help us accomplish this goal, it would need to work on Windows.
Even though there is no current support for this today, are there plans to support it in the future? I know that Tailscale has managed to make this work transparently to the user and the client source code is open-source...

@brad-defined thanks for the response. Our goal is to use Nebula to temporarily expose a route to specific internal non-nebula devices periodically throughout the day so that we can remotely collect data from them using a wide variety of networking protocols (SNMP, HTTP, etc). Our agents are cross-platform and most of them are running on Windows devices. In order for Nebula to help us accomplish this goal, it would need to work on Windows.
Even though there is no current support for this today, are there plans to support it in the future? I know that Tailscale has managed to make this work transparently to the user and the client source code is open-source...

The way that Tailscale delivers this is by importing a userland implementation of the TCP/IP stack. They use this to terminate the TCP connection at the Tailscale node, and then establish a local TCP connection from the Tailscale-installed node to the ultimate destination IP:port, and proxying the connection along.
A port forwarder that terminates local connections, establishes upstream connections to the desired IP:port, and proxies the two is basically what Tailscale's built.
One way you can try this today by setting up an SSH server on the Windows host that has network access to the systems you want to collect data from, and configure that SSH server to listen on the Nebula IP of that windows host, and configure it to permit port forwarding. (This would require an SSH client to set up the port forwarding session.)
Other solutions would be to run netcat, or socat, or build a port forwarding proxy into your agent.
Nebula has a built-in SSH server, but it does not support port forwarding today (it's mainly used for status and debug info.) If it did support port forwarding, would that satisfy your use case?

Is there a reason that Nebula does not do something similar? I.e. is that userland implementation is too slow for Nebula's primary goals, or has it just not been implemented yet?

Nebula has a built-in SSH server, but it does not support port forwarding today (it's mainly used for status and debug info.) If it did support port forwarding, would that satisfy your use case?

Possibly? I haven't done much port-forwarding over SSH so I can't say for sure. If it allows our remote data collectors to access a specific non-nebula IP address via a nebula node, over a specific port, then yes (see diagram). Building a forwarding proxy into our agent would be no problem at all. A few questions if we chose to go that route:

How would the remote collectors indicate that they want to connect to a non-nebula IP address over a specific port, just use unsafe_routes like normal?
How would the nebula peer know to pass those packets to our own forwarding proxy (which would be running on a specific port on the nebula peer)?
How would our forwarding proxy extract the destination IP address and port?

@clarkmcc I happened across this issue and the timing is interesting. I started messing around with this the other day for other reasons and I have a basic integration of the Go userspace network stack with Nebula. It supports TCP proxying right now. I have not implemented UDP yet (so SNMP wouldn't work), though I want to look at adding that next and I don't expect it to be difficult.
My changes are here:
https://github.com/davidflowerday/nebula/tree/userspace-network
I've only put in a few hours on it so far but I have tested it on Windows, Mac, and Linux and it works fine on all of them. You can access any TCP IP:port combo through the host running it as long as the host is configured properly with unsafe_routes. Currently you just set tun.netstack to your config.yaml to enable it:
tun:
  disabled: false
  netstack: true

@davidflowerday this is very interesting, thank you for your work on this and for sharing! I just glanced over the gvisor library and I see that it does provide some utilities for implementing UDP forwarding. I'll do some testing and possibly dive into taking a stab at the UDP forwarding implementation later this week.

Is there a reason that Nebula does not do something similar? I.e. is that userland implementation is too slow for Nebula's primary goals, or has it just not been implemented yet?

@clarkmcc  I think pulling the TCP/IP stack into the Nebula process would be sub-optimal performance-wise, but would also enable some capabilities like the one you're asking for. It would also, for example, also enable Nebula to be embedded directly into programs and route traffic without creating a TUN device on the host. So, it's possible / likely that this will happen, but just hasn't happened yet.
In your use case, IIUC, you'll have a couple of problems to solve. Your agents are going to be installed on machines in Client networks. It's possible that some of the Client's networks will overlap in their IP ranges. You'll not only need a way for the terminating Nebula node to pass and receive traffic on the LAN - you'll also need a way for your local Nebula node to determine which of your Client's terminating Nebula nodes to use.
Meaning, it's possible that you'll have two clients, each of whom are using a 10.10/16 LAN network, and you have an agent in each network. Your local Nebula Remote Controller node will have to decide which terminal Nebula node to route traffic to <IP_ADDRESS> to - client A's network, or client B's network.
With the SSH port forwarding solution, you would set up a local listening port that would map to some arbitrary Client IP and port. Then, any connections made to the local port would be tunneled through to the Client IP and port. The local port would be set up as something like:
ssh -L <IP_ADDRESS>:<local port>:<target client IP>:<target client port> <nebula IP of client Nebula host>
The tunnel command includes which destination IP you want to hit (<IP_ADDRESS>, say), which port, and which Nebula node to route the traffic through. Using SSH would be limited to TCP forwarded connections only, I believe.
With the Unsafe Routes solution, you'll need a way to configure the local Nebula node to know which terminal Nebula node to use when the local Nebula node receives traffic for <IP_ADDRESS>. (This would likely look like some reloadable config options, with the caveat that the entire local Nebula would only be connected to one Client network at a time.)

@clarkmcc I added some basic UDP support as well. I'm curious to know if that implementation would work now for the use case you have in mind.

@davidflowerday @brad-defined thank you for bringing up those issues (and for the UDP support), to summarize:

Collectors need to be able to collect to specific devices within an agent's network even when multiple isolated agent networks have overlapping ranges.
Nebula currently creates a TUN device but I probably don't want to touch network interfaces on the agent side. This brings up the point that ideally, connections would only be initiated from connector to agent but not the other way around.

I've been running through POCs and I'm wondering if Nebula is the tool for the job, perhaps either of you can speak to that. Here's what I have in mind. Setting aside security concerns, can we just give the collector a network interface that behaves as if it's a network interface running in the customer network? What this translates to is the collector having a userspace network stack bound to a TUN device, where the network stack passes all packets over a pre-existing connection which reaches the agent's userspace network stack where we implement packet forwarders (i.e. no TUN device on the agent side).

Based on my testing so far, I could use something like libp2p to act as the "transport" which gives a lot of the same NAT traversal features that Nebula provides. The only reason I suggest libp2p is if Nebula does not appear to fit this use case.

@clarkmcc Without something like @davidflowerday 's changes, Nebula must create a TUN device to transport traffic
If you're in control of the Collector, you could solve your agent connectivity by having the Agent initiate the connection to the Collector, and then using that connection to transfer whatever traffic you require. WebSockets, long poll, or SSH would all work great. Having the Agent initiate a connection to a public Collector IP/port would solve all NAT and basic Firewall issues.
Once the connection is established, it can sit idle (with keepalives or similar to maintain NAT/Firewall state) until the Collector decides it wants to talk to the agent. The Collector can use the Agent-established TCP connection to send requests, which the Agent can then act on.

@brad-defined sadly, we're not guaranteed to be in control of the collector and it's also not guaranteed to be publically accessible, hence why I've been looking into the Nebula/libp2p options. Thanks for your feedback on this so far, it's been extremely helpful.

ig https://github.com/slackhq/nebula/issues/559 is related in that they're both discussing user-space TCP stacks?

Collectors need to be able to collect to specific devices within an agent's network even when multiple isolated agent networks have overlapping ranges.

That's a bit of a problem but I think there's at least 2 ways to solve it.

Set up each agent device with Nebula with the appropriate subnet specified in each agent's certificate. Then on the remote collector side, you could modify Nebula's configuration and restart Nebula, so that the remote collector would only have a single unsafe_route at a time to whichever agent you are trying to contact. I'm pretty sure the overlapping ranges on each agent wouldn't be an issue as long as the Nebula node (remote collector) that you're initiating the connection from only knows about 1 of them at a time.
It wouldn't be too hard to take what I've done for TCP/UDP userspace proxying and add some IP address rewriting on top of it such that from the Nebula network's perspective, each device would be on a unique subnet. In the Nebula code running on the agent, when it examines the remote IP address, it would modify the IP to be the proper target before initiating the connection. This would basically be a little mini-NAT. I'm already doing this somewhat when the target IP is the Nebula address, rewriting it to be localhost. You can see that here, which is about where you'd want to make such a change:
https://github.com/davidflowerday/nebula/blob/935072f133cbe56601090ad85c35fcf668892c53/overlay/netstack_device.go#L302

I think Nebula with my changes could do what you want, though I do have to question if you already have an agent that you control running on the agent devices, why not just add the functionality to do the collection in the agent and then have the agent send that back to your remote collector over HTTPS or something like that? Nebula may be overkill for your goals unless there's some other reason to have Nebula there (e.g. SSH/Remote Desktop access/etc.).

Nebula currently creates a TUN device but I probably don't want to touch network interfaces on the agent side. This brings up the point that ideally, connections would only be initiated from connector to agent but not the other way around.

Just to be clear - the changes that I made in the branch I posted allow you to use Nebula without a TUN device at all. Instead of using a TUN device, the Nebula network connections are terminated into the gVisor userspace TCP/IP stack and then from there regular TCP/UDP connections are established to the target. My real goal here is to be able to run Nebula inside Docker without the need for any special permissions (which means no TUN devices). My next step is to add SOCKS5 proxy support so that applications running inside a container can establish outbound connections to the rest of the Nebula network via the SOCKS proxy. I have that all written now but I need to track down one small bug before it's ready.
I haven't used Tailscale myself but my understanding is that this is the same way that Tailscale works for TUN-less operation, both as far as establishing outbound connections and the SOCKS proxy.


use emoji-function

While I was browsing the code, I noticed that you manually maintain a list of country flag emojis. This is not necessary, they can be derived from the ISO 3166-1 alpha-2 code. See https://en.wikipedia.org/wiki/Regional_indicator_symbol

I like it, thanks! Could you extend the country test case to also include the flag?

Hm? I didn't change the interface, so the tests should work as before.

Yes. The current tests are not checking for the emoji: https://github.com/vanniktech/multiplatform-locale/blob/03561cd84e1ff4a8a630a2b143ee7ef26b731e26/multiplatform-locale/src/commonTest/kotlin/com/vanniktech/locale/CountryTest.kt#L61

Hm, I see, but then, basically, one still would have to maintain the flag emojis manually, but instead in the test code. So in the end, nothing is won. I'll rather close this PR in this case.

Yeah but it would be auto generated and isn't too bad. Don't worry I can get to it also.


[testing] Remove dependency on filesystem for test_config unit tests

to: @airbnb/streamalert-maintainers
size: small
resolves #395
Background
CLIConfig uses the filesystem to find the filenames of cluster configuration files. In the config unit tests, these filesystem operations need to be mocked out because cluster configuration files are user configured. Currently __builtin__.open() is mocked out, but we also need to mock out os.path.listdir(). Instead of mocking individual functions, pyfakefs is used to create a mock filesystem.
Changes

Added new python dependency pyfakefs to create a mock filesystem
Update tests in test_config.py to use the mocked filesystem instead of mock_open

Testing

Remove prod.json cluster config and added abc.json cluster config
Step through CLIConfig.load() to make sure os.listdir() returns prod.json
Run nosetests -v tests/unit
Run ./tests/scripts/pylint.sh

Coverage remained the same at 95.075% when pulling 9c76e950933ad4cbf67fd303cccca0a121d9b7f3 on stoggi:stoggi-test-config-fakefs into 591468c349e732c4e962f46c06e4ade17ceec6ba on airbnb:master.

Coverage remained the same at 95.075% when pulling 8838c00950b92a4e4e7a1acade55f39ad1eb8dd3 on stoggi:stoggi-test-config-fakefs into 9acd5f47dd9365d6d102f46b6a369519d63af079 on airbnb:master.

Thanks for the cleanup! One tiny nit and then it'll be good to merge :)

Coverage remained the same at 95.075% when pulling b18eb87680a38a3b4988f684cb6f108cdc2ce8f9 on stoggi:stoggi-test-config-fakefs into 6114f665ae83d02d505c47cd19de12752cab3cfb on airbnb:master.


Fix running test suite under Docker Machine

This PR makes the test suite run correctly under Docker Machine, including under OS X.
Fixes include:

Avoiding the use of xargs -r (OS X fix)
Testing container connectivity from sibling containers instead of directly
Reset IFS to the default due to https://github.com/sstephenson/bats/issues/89

Looks like my appropriate/curl image locally differed from the one on Docker Hub... Lemme fix that.

I pushed a fix for appropriate/curl here: https://github.com/appropriate/docker-curl/commit/af2cc9f44f7a62666fa667d0c5664e26664fa5d7
I also pushed a new commit to this branch to remove a commented out line that should trigger a (hopefully successful) test run.

Now there seems to be some sort of error with Circle CI...
 Error deleting container: Error response from daemon: Cannot destroy container a8bc77c4639069aebb72210eeadc18b6b50fa24c3f70670547cfc3031a92c38b: Driver btrfs failed to remove root filesystem a8bc77c4639069aebb72210eeadc18b6b50fa24c3f70670547cfc3031a92c38b: Failed to destroy btrfs snapshot: operation not permitted

BTW, reading the diff for this PR is much easier if whitespace is ignored since I changed the line endings on two of the files from DOS to Unix: https://github.com/jwilder/nginx-proxy/pull/257/files?w=1

I'd like to merge this but, I have not been able to get circleci to run the tests succesfully yet.  Seems to be complaining about btrfs snapshots which might be related to this: https://github.com/jwilder/nginx-proxy/pull/257/files#diff-29944324a3cbf9f4bd0162dfe3975d88R18.

I don't think that the failure is related to docker pull docker. I'm pretty sure it's coming from the docker_clean helper here: https://github.com/appropriate/nginx-proxy/blob/osx-fix-test-suite/test/lib/docker_helpers.bash#L7
I'm not sure why it suddenly started being reported as a problem, however.

Alright @jwilder
Got it green with a couple of changes! :money_with_wings: :green_heart:

Nice @md5! Thanks for this PR!

Sure thing. It might be nice to still have the --rm when not running on CircleCI. I may have another PR for that at some point, since I don't like how many containers this leaves around when I test it locally.
I've also got another PR that I'm going to submit in the next few minutes that cuts the test time in half. Hopefully CircleCI doesn't choke on it...

I left the test containers hanging around on purpose to allow manual tests to investigate a failing test case.
My workflow is:

start the containers for the failing test case
do the curl queries to reproduce the issue and find a fix

we could provide a shell script to remove all containers named 'bats-*' to clean up after bats if no failing case were found

I was talking about the sibling containers used to run the Curl commands
now.

my mistake, then what about cleaning them in the bats teardown function?
For instance, they could be started with the docker label bats-type=teardown and the teardown function would be:
function teardown {
    CIDS=( $(docker ps -q --filter "label=bats-type=teardown") )
    if [ ${#CIDS[@]} -gt 0 ]; then
            docker rm -vf ${CIDS[@]} >&2
    fi
}

Perhaps. It's not really clear to me whether that would work under
CircleCI. It seems odd to me that “docker rm -vf" would work but not
“docker run --rm".
I was thinking of making the omission of “--rm" conditional on an
environment variable that's only set in circle.yml.


feat(drag&drop): drag and drop

This PR is to enable image drag and drop feature,

the image will be uploaded to /assets directory
image will automatically insert into the post

Closes #8

Merged, great work!


[Feat] Allow dynamic DataSource type to AddColumn

hi , it is a nice project .
in the runtime AddColumn() mode,  we must access the ITable< DataSource > instance, but in many context, we even dit not have the 'DataSource' type definition in code.  it is just an ITable,

@code {

private ITable table;

...

protected override void OnAfterRender(bool firstRender)
    {
        base.OnAfterRender(firstRender);
        if (firstRender)
        {
            var col = new Column()
            {
                Title = "some desc",
                //Field = f => f.Summary,
                FieldName = "Summary",           //pls allow string field selector rather than lambda field here :)
                Sortable = true,
                Filterable = true,
                Width = "10%"
            };

            this.table.AddColumn(col);            //for now, ITable dit not have AddColumn api :(

        }

}

I know this is an older question but perhaps it can still help someone.
I also ran into this. But i used a simple ExpressionTree to build this dynamically too.
I made a little method that you can try using.
private Expression<Func<T, object>> GetFieldExpression<T>(string fieldName)
{
    var parameter = Expression.Parameter(typeof(T), "t");
    var propertyExpression = Expression.PropertyOrField(parameter, fieldName);
    return Expression.Lambda<Func<T, object>>(Expression.Convert(propertyExpression, typeof(object)), parameter);
}

In your example you can use it as follows:
var col = new Column<ColumnType>()
{
    Title ="some desc",
    Field = GetFieldExpression<ColumnType>("Summary")
};
this.table.AddColumn(col);

This works pretty well for me to load columns dynamically using reflection. Just make sure to change ColumnType to the type you are displaying in your table (ie your TableItem).


Allow backend to skip if errors. Support for a file path as a token.

Added functionality which will prevent the backend from erroring out if either Vault is unreachable or if the RubyGem dependencies aren't installed
Also added functionality so that a file path can be provided as a token - if the file exists, it will take the top line of that file as the token and use that to connect. Useful for token obfuscation on ephemeral hosts which are deployed with temporary access tokens to Vault, for example.

Hi @wrightj5! I've merged #16 which allows reading token from a file on disk, so we don't need that code, but the logic to skip the backend if the gems aren't installed is a good idea I think. Can you seperate that out into another PR for me?  :smile


Merge Context Manager + Work Files (Fix #412)

This is a work in progress implementation for issue #412 that describes to merge the Context Manager and Work Files tool into one single tool that handles both, as usually the actions performed with these happen in relation to each other.
What's changed?

The refactoring code is now written into the work files tool. I haven't removed the Context Manager yet (WIP)
I've also started to refactor the code for the work files app.py according to the contribution guidelines, however I think this will introduce quite some discussion and pointers on how to improve - as such I'm opening this PR In an early stage to ensure as much as possible this can be discussed and I can improve it as needed.
The current implementation works roughly similar to what was described here.

Note: This is a work-in-progress and I'm opening a PR to kickstart a discussion on code style, but I'm fully aware this is not as clean as it can be. However, even at this stage, code style ideas or notes according to the contribution guidelines are very welcome.
Note 2: I've added some todos that are more notes to self as I'm developing and changing some minor things purely for testing/development sake. Feel free to be captain obvious and point out if you have questions - just wanted to highlight that many are reminders to myself.

Just realised that I commented on the issue rather than here. So for completeness here's clarification on my early comments https://github.com/getavalon/core/issues/412#issuecomment-530479801

Have recently been using the Context Switcher more, and I immediately go to the Workfiles app afterwards, so merging the two would make it very streamlined.

Tested it again, and now the tools open fine on Houdini without errors and they are nicely parented. The Avalon stylesheet still doesn't apply perfectly, it still seems to inherit quite a bit from the Houdini styling.
However it seems the styling of Houdini itself might change in the near future that might automatically resolve it. Maybe it's best to leave it as is since it isn't a big issue and then maybe with an upcoming Houdini version less of the Houdini styling pushes through in the Avalon tools.
Nice work @davidlatwe ! 🚀

I've been recently fiddling with the Work Files API to allow the tool to list the files even without requiring to switch the context prior to it. Admittedly it's a bit hacky, especially with the root dir in Maya being dependent of the workspace.mel.
Anyway, in the end it seemed to be fast and work quite well. I think from there it's a matter of cleaning up and simplifying again.
I hope to share some bits of it in about two or three days as my schedule this week is quite busy. Just wanted to let you know I am back on this PR.

@BigRoy That's awesome! This would also help in adopting 'Allzparkalon'. And like @tokejepsen is saying: most of the time when you switch context you open the Workfiles app anyways.

Here's a preview of the latest functionality:

Download zipped .mp4: avalon_workfiles_merged_simple.zip
The little log window at the bottom is Maya's script editor and is not part of the Work Files UI.
To-do:

Prior to save/open the Session should first be set to ensure it's switched prior to open/save automatically.
Set Context functionality: Double clicking the Task sets the context manually (forcing it) - I want to make that clear by having a "Breadcrumb" of sorts at the top of the UI showing the selected Asset/Task and color coding that e.g. orange when it is not the currently active Session context, if that's the case... behind that I'll add two buttons, one for "Setting the context" (like double clicking the task does now) and one for "Go to current context" that will browse back to your current active api.Session context. The buttons will be disabled when they are redundant (e.g. go to current context will be disabled when you're already there) and "set context" is disabled when you're already in the context. Then browsing should make this self-explanatory and the tooltips of the buttons can aid in that as well.
Initialize workdir: In the case when an Host has never been initialized to a specific asset+task the Workdir does not exist. What should happen then is to perform the App initialization to make sure it's setup like the studio config intended. When browsing to a non-initialized Work Directory I want the most right-widget to be swapped with a message saying "No work directory exist" with a "Initialize Work directory" button underneath. Currently performing this "App Initialize" is non-trivial yet a hacky method for it can be found here. What would be the best way to improve this?

Any first thoughts are more than welcome.

Looks good!
Initialize Work directory might sound a bit too techy/cody. Maybe Create work area

I like the ideas in #489, will they get implemented in this PR, or in another one ?
How about adding a "New" button, so there should be pretty much what artists will do when they open up context manager. Then I think we won't need double clicking to set context and the Breadcrumb.

How about adding a "New" button, so there should be pretty much what artists will do when they open up context manager. Then I think we won't need double clicking to set context and the Breadcrumb.

@davidlatwe Sorry I'm not sure I get what you mean.
Whenever you launch the host it should have already initialized the current application. Or at least that is what Avalon currently does as it launches the host. So for the current context you would never have to "create work area" since it already exists. It would by default just show an empty folder on the right hand side. Where I'd want to add a single non-selectable greyed out entry in the list stating: "No files. Use Save As to save your first file."
The tricky thing is that as soon as you click on another Asset or Task you're operating in an area that is not related to you active Context/Session. Previously it came up that there could be reasons to force yourself into the workspace context, even without Saving/Opening a file there which @mkolar mentioned before. It would basically be, setting that context and doing some work prior to saving there if I understood correctly.  Exactly that is what the "buttons" would allow, to if you want set the context. However, in most scenarios I assume an Artist doesn't care and will just open the scene (which will always first ensure the context is set appropriately).

Ah, I meant "New" for set context + open new scene.
I assume that one would open up the tool only for following cases :

Changing work space to start something new.
Changing work space to open up old scene.
Save the file, but change work space if other asset/task selected on save.

So if there's a "New" button for opening new scene, I think there wouldn't need double clicking to set context and the Breadcrumb.
And in the end, I think artist will no need to be aware of the concept of "context" when using this tool as long as there's only "New", "Open", "Save" these there buttons being presented.

With https://github.com/getavalon/core/pull/442/commits/7ad657a075856b38dd95c21fcfbcbe8fa0060a4c I've pushed the new prototype functionality. It's not flawless yet but it works to some extent.
I've tried to comment on most of the important/big changes or odd choices above. Hope that helps reviewing what's there so far.
There's a lot to potentially check:

#465 this crash might have been reintroduced with https://github.com/getavalon/core/pull/442/commits/cef28803bbd604507470f34539f54e84d6aaa4fd since I "patched" it different in the conflict merge, I set the parents explicitly as opposed to storing the variables. This should result in the same behavior in the end. So if someone is able to check potential Blender crashes that would be helpful. @jasperges
Does all the functionality work across different hosts, e.g. Maya, Nuke, Fusion, Houdini, etc.
Please check and comment on other odd code changes.
The failed Travis check is due to #492 -> PR #494

@BigRoy This new version almost works fine in Blender. If you open a workfile the Qt interface is locked and the only way to get it back or close it to open it again from within Blender.
There used to be an explicit self.close(). Did you remove that on purpose? Is the UI supposed to stay open or should it close, like it used to?

Regarding the issue @jasperges mentioned for Blender. I checked it briefly with him. This isn't actually a problem with this particular interface but with the way the Qt interfaces are running there in general. Whenever any Qt interface is open with the Modal Operator and you open a file then it'll hang the Qt application. As such, on scene open with this Work Files tool it shows that issue.

Did you remove that on purpose? Is the UI supposed to stay open or should it close, like it used to?

In this case I did leave it open on purpose and maybe wanted to add a toggle "autoclose" or some setting. But I'm happy to also force either of the one. It's just much easier to debug without the auto-closing, and additionally some Artists mumbled it a couple of times whether it could be worth looking into just having it open as "explorer" when they need to check some scene from different assets, so I removed the automatic closing on purpose for now.

I've been working on tweaking the interface for the Work Files some more:

This is based on some of @mkolar 's comments on #489 and in particular implementing point 3-5. Specifically having date modified there too, allow sorting by it and having a filename filter at the top.
I've also moved the Save As button to be next to the other ones, this allows for more screen space for listing files and makes the button sizes more consistent. @jasperges do you think this looks better?
I'll need to do some more changes to the code to have the file open/save functionality working again as I just quickly threw together the UI changes but haven't yet accounted for the change to Model+View as opposed to the previous QListWidget for the files. I think I can push it Monday.

I just pushed https://github.com/getavalon/core/pull/442/commits/d7130433f2afd2c7f32643d35f25267cf5e551ed which implements the UI as shown in this comment.

It also has Duplicate in a right mouse context menu now.
It also selects the work file with highest date modified, like with the original work files tool.

Artists on our end also requested to add a Filesize column. Any remarks regarding that? Might it get too detailed or cluttered? Any reasons not to add it?
Like before the open TODOs:

Set Context functionality: TODO
Initialize workdir: TODO

@BigRoy Yes, I think this is better. Every button the same size and a bit more room for the files list.
About adding a filesize column: I don't mind it that much, although it will get a bit cluttered. Maybe you can add the functionality so users can choose which columns to show (right mouse menu), just like in a regular file browser. Then everybody can choose for themselves what to show.

About adding a filesize column: I don't mind it that much, although it will get a bit cluttered. Maybe you can add the functionality so users can choose which columns to show (right mouse menu), just like in a regular file browser. Then everybody can choose for themselves what to show.

That could definitely be nice, but it does add some more complexity which I currently think we don't need yet. Because it'd mean we'd need to store the preferences of the user too (otherwise they always need to set their settings again). Easiest would be then to store per machine, but then we might want to have it per user - where there is no User-specific yet in Avalon. I think for now it's best if we can manage to have these tools as simple as possible as it'll be easier to maintain.

That could definitely be nice, but it does add some more complexity which I currently think we don't need yet. Because it'd mean we'd need to store the preferences of the user too (otherwise they always need to set their settings again). Easiest would be then to store per machine, but then we might want to have it per user - where there is no User-specific yet in Avalon. I think for now it's best if we can manage to have these tools as simple as possible as it'll be easier to maintain.

I completely agree with keeping it simple.
The problem for me with more columns is, that it is already about preference. Not everybody has the same preference. So how do you determine what is a good baseline for everybody and when it's really a user preference?
Apart from adding the functionality to view extra columns, can't you easily store the preferences in QSettings?
Again: for me it's fine to add the filesize column, but I would definitely draw the line there. If an artist wants to see more, they can always use the 'Browse' button and get more info there...

can't you easily store the preferences in QSettings?

Yes, you can. They would however be per machine and not per 'avalon user' (which is not a thing yet I suppose).

The problem for me with more columns is, that it is already about preference. Not everybody has the same preference. So how do you determine what is a good baseline for everybody and when it's really a user preference?

Agreed. I'll leave it as is without the filesize column. If it keeps coming up with the team then we could also look into our options for a future issue.

@BigRoy This new version almost works fine in Blender. If you open a workfile the Qt interface is locked and the only way to get it back or close it to open it again from within Blender.
There used to be an explicit self.close(). Did you remove that on purpose? Is the UI supposed to stay open or should it close, like it used to?
To keep it working, I quickly added module.window.close().

To come back to this: I changed the way Blender calls QApplication.processEvents(). This is not an issue anymore.

Initialize Work Area
Here's the Initialize Work Area in action. Does this look like the way it should be? It doesn't feel as smooth as I'd like but I can't think of anything that's more clear than this to the end user.
@jasperges @mkolar @davidlatwe any pointers?

This resolves the Initialize workdir: TODO
Remove Set Context functionality
I have removed the "double click on task" functionality to explicitly set the current api.Session to another context than the current work file. Instead, just saving or opening in another context will first switch the context.
That way accidentally switching contexts is near impossible, and also avoids the complexity for the end user. @mkolar would that still allow all your use cases to happen?
This current avoids the need for Set Context functionality: TODO

This is now in a usable state.

Please remark any open comments on code style.
If you have time, please test it. Try to break it too please.
Confirm the App work directory initialization works on your end.

Any reason why work area creation cant happen on Save As?

Any reason why work area creation cant happen on Save As?

@tokejepsen Good question. Technically it could. Then before work area initialization it would just show no files (and maybe the model would just have a non-selectable placeholder "Work Area does not exist" and open and browse get disabled. It might be somewhat "confusing" though if you find out that the folder doesn't exist at all yet.
Would you want that to happen instead? What do others think? It's not a bad idea.
Aside of clarity the only reason I'm thinking of is that I'm not too sure how well App Work Directory initialization will work for others who launch with a different means than the Launcher. Specifically because it requires the "Application" name to be available and allow to trigger a .toml file for it.

It might be somewhat "confusing" though if you find out that the folder doesn't exist at all yet.

Have noticed that artists hardly navigate the folder structure anymore when Workfiles is working.

Would you want that to happen instead?

Personally I would like to see necessary actions like work area creation hidden from artists, and simplify/speed up their workflow > Less button presses.

Personally I would like to see necessary actions like work area creation hidden from artists, and simplify/speed up their workflow > Less button presses.

Agreed. Here's the updated version.

Will have a good look and test later today !

So I have tested, and great work !
But, I would prefer having an extra button for initializing work area as the one in this comment. :relaxed:
Because if I am starting from an empty scene, it's a bit weird to save anything for getting a workspace.
Or, how about just hide that "work area does not exists..." item, and auto create work area when Save As is pressed. If the Browse button is pressed, pop a warning dialog.
Is there any reason that one need to explicitly initialize work area ?
Oh and, would be better if the task view can display "Task" in header instead of "name". :)

@davidlatwe @tokejepsen what if we just swap the bottom buttons Open, Browse, Save with one big button "Create Work Directory" at the bottom. That way the change to UI is not as odd as in my first draft?
@mkolar what's your thought on this subject?

I would avoid any unnecessary mouse clicks from the user. The main point of making the Workfiles app, was to make it faster to load/save your workfiles so the user's preference wasn't the DCCs native load/save workflow.

Because if I am starting from an empty scene, it's a bit weird to save anything for getting a workspace.

I'm not sure I follow this @davidlatwe ?
If you have an empty scene and need to create a workspace, then wouldnt you have to save your file?

If you have an empty scene and need to create a workspace, then wouldnt you have to save your file?

Yeah, that's true. I think I kept mixing the usage of context manager app when I imaging the workflow of workfile app...

The main point of making the Workfiles app, was to make it faster to load/save your workfiles so the user's preference wasn't the DCCs native load/save workflow.

I think you are right. Yeah, I could go with "Save As" to init work area if not exists. :)

I actually think that it's very clear in this implementation. If the folder doesn't exists, save as will create it alongside the actual file. quick and clean. No extra clicks.

Yeah, I could go with "Save As" to init work area if not exists. :)

@davidlatwe does that mean the current implementation is good to go?

@mkolar @tokejepsen did you try the code on your end too? Or was it just a comment based on the GIF?

If everyone approves I'd still like to do the following before merging

Delete avalon.tools.contextmanager
Remove context manager from all host implementations.
Rename column header name to Tasks in the second list seen here

did you try the code on your end too? Or was it just a comment based on the GIF?

Based on the GIF.

If everyone approves I'd still like to do the following before merging

Sounds like a good clean. Nice!

One little thing, the "Work area does not exists.." menu item is able to show context ment on right click, which shouldn't be. If clicked the menu's "Duplicate" action, error raised.

"Work area does not exists.." menu item is able to show context ment on right click, which shouldn't be. If clicked the menu's "Duplicate" action, error raised.

Well caught. Will need to patch that up, thanks!
I also noticed that, because it's a regular Item, whenever you would have a name filter set top right that you could be filtering this placeholder item too. Not a major issue but something to watch out for. Probably will need to see if we can easily disable the filter then.

Context label has been removed in commit be07276, was there a replacement ? 0.0

@davidlatwe True that's my fault. But where to put context label when Edit Context.. action was removed?

Can we change them into normal menu item instead ? Or keep the context manager for now, until artists get familiar with new workfile app ?

I'm not user so my word is not much valuable at this point, but I would rather add empty action with context label and without callback instead of keep context manager, because me as user wouldn't understand why he is there if I can do it through workfiles.

I'm also not a user, but I think it's a good thing it has been removed, because I don't think it's ever needed to switch the context without loading or saving a work file. (Although we probably have the need to publish several character rigs from one base rig. So inside the file we have to run the Creator for different assets... but that is an exception, not the rule.)
I think it's a good idea to show the current context somewhere, but inside a menu feels like the wrong place for that. Not sure where it should go, though...

Just tested this out myself and it works nicely!
One question did pop up. If there arent any task assigned to the asset, you wont be able to save a workfile, so you would have to open the project manager to create one.
Maybe its taking too much responsibility on, but should the user be able to create tasks on assets through this UI?

Personally I don't think so. I think most of the time this is not something an artist should do (a production manager/coordinator should do this or automated from Shotgun/FTrack/Kitsu) and definitely not in the workfiles app.
Just my two cents.

So, does this mean it's ready to merge? 🔥🚀

So, does this mean it's ready to merge?

Except for the hound violations?

Except for the hound violations?

Was just editing my comment to state that, yes. I can't seem to find the open Hound violations...

Ahh, fair enough. Ready to merge!

But what about the context label ? :O
I can agree that context label is not good to be in side the menu, but it's a place that work for most of the DCC hosts for now to indicate current context, right ?

How about changing the window title?

How about changing the window title?

That did cross my mind but should be in a separate issue/PR to discuss.
Anyway, if t's just me, then maybe it's okay to be merged. Either add back context label as a  normal menu item, or merge it as is, both work for me now. ☺
Still, IMHO, deprecating context manager with new workfile App is one thing, removing context indicator altogether is another, so I think this kind of UI change should be brought up earlier. 💡

I've tried to add not enabled action item to menu PR: https://github.com/BigRoy/core/pull/5

Still, IMHO, deprecating context manager with new workfile App is one thing, removing context indicator altogether is another, so I think this kind of UI change should be brought up earlier.

Totally agree that these shouldn't be taken lightly. I just envisioned that to be part of it and assumed the discussion was already debating that too.
Anyway, shall we include https://github.com/BigRoy/core/pull/5 and then open an additional issue to track any better ways to approach that @davidlatwe ?

Anyway, shall we include https://github.com/BigRoy/core/pull/5 and then open an additional issue to track any better ways to approach that @davidlatwe ?

That would be great :D
Just pulled and tested that branch, and only require one tiny fix and good to go for me !

Just pulled and tested that branch, and only require one tiny fix and good to go for me !

Just pulled fix. Can you check please?

Perfect !! 🚀

Ready for take-off!

I've set AVALON_SCENEDIR to 'scenes' for Maya. But using this new workfiles app, I get an error when initializing the work directory (that doesn't exist yet). I've add default_dirs = ["scenes"] to the maya.toml, but still no luck. Am I missing something?

If you are using Avalon Launcher, maybe you need to head back to first page or restart the launcher so the environment vars get update from those .toml change.

Just realized small detail... Will that context label change on context change?

Yeah, it did change in my test, and still does.

@jasperges  is everything working there now ?

@davidlatwe On Monday I can check again. I don't have access to Maya at home. Will let you know.

@davidlatwe All is good! I still have some small issues, but I'm pretty sure that's because of the Avalon configuration. So, go, go, go! :rocket:

Merging this !


Omit command name in parameters when only one command is loaded

If only one command is loaded, allow the user to omit the command name in parameters. If multiple commands are loaded (or none at all), the existing behavior is unchanged.
For example, given an application named 'app' with a single command named 'example' that has an optional parameter 'foo', the user should be able to call it like this:
app
app --foo=bar
app example
app example --foo=bar

Thanks, it looks good to me.  I have pulled your code and the updates are on nuget as of ManyConsole <IP_ADDRESS>.


Automated Package Upgrade

:tada: This PR is included in version 25.3.74 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


102 ignore distribution files

Leverages new feature for sirepo.lib (https://github.com/radiasoft/sirepo/issues/3896) to allow files which are expected to be created during the rsopt run to be ignored during parse.
Automatically does this for anything supplied to setup.input_distribution.
A user set field setup.ignored_files is made available for users to supply other file names.
A multi code example that runs a simulation in opal then elegant along with a demonstration of some advanced preprocess manipulation is also added

Code looks good and I met with @cchall to discuss any questions. Unfortunately, I have been unable to perform the recently updated tests (specifically test_setups.py). The reason appears to be a difference in Sirepo capabilities on Jupyter and @cchall's install (see: https://github.com/radiasoft/sirepo/issues/4360).
To run the tests, I will need to install a new Sirepo instance. @cchall says there is no rush on accepting the PR.


Update Summary docs to improve clarity

@Vivihung asked some great questions about how the summary works. Questions are listed below. The change attempts to answer those questions more clearly.

What is the data flow between the client and server?
How is the summary generated? Does it require multiple ops?
How can a Fluid host leverage the summarizer?
a. Does the host have any control over the summarizer?
b. Why does the host care about the summarizer?
Should the container developer care about the summary?
Where is the summary uploaded to?
Will the summary tree be generated every time? Is that too often?
Where is the summary generated? The docs indicate both the client & the service as origins.

This looks great! I think the new sections/paragraphs on why and what do I need to know are vital.


update wgpu

Any idea what's going on with the rustfmt job? I'm surprised that dependencies are failing there. O_o

It's a wgpu issue, a commit is pushed to upstream but not released yet

recommited the thing to trigger CI, lol


feat: first pass on creating extension config

This PR is still a WIP.
Actionable items:

[x] configure webpack to transpile jsx to html and js
[x] add tailwaind for styling
[x] configure react dom

Looks good

@omoniyi24 @DaScOrPiO This PR is ready for review


[BUG] Ion Beam window was set to electron beam when milling.mill_stages ran

when using _validate_mill_ui in autolamella for auto milling, caused the ion beam window to switch to electron, not sure if threading issue.
need to try to recreate

it was a threading issue :(


Support for deployment based on the planet alignments (AstroBuild)

I would like to use DICSS with correct planet alignment.
The project which provides such feature is here:
https://github.com/lhartikk/AstroBuild

That sounds like an implementation detail, and therefore is beyond the scope of this project.


PowerShell 4.2.0 Displaying Wrong Error - Set-CsPhoneNumberAssignment

We upgraded to PowerShell 4.2.0 in the hopes that the Set-CsPhoneNumberAssignment errors were fixed, but they are not. Users who are missing the Phone System license are displaying the following:
FAILED | BadRequest The user 'd2960676-bad7-4d7b-96e5-a5ec9066b82b' already has a telephone number set in the on-premises Active Directory and synchronized to the cloud. To manage this user's telephone number in the cloud, first clear the phone number in the on-premises directory and synchronize to Active Directory.
When using PowerShell 3.1.1 the correct error is displayed:
FAILED | BadRequest User 'd2960676-bad7-4d7b-96e5-a5ec9066b82b' lacks appropriate licenses to assign a 'DirectRouting' number.
Since Microsoft is forcing companies to move to 4.x.x we need the errors displayed to be correct and valuable. We are not willing to upgrade to the 4.x.x PowerShell if the errors are not correct.
User account for testing was created today and have never been on-prem or had a number assigned to it.

Hi @rmichalskinw,
Can you please try this with 4.2.0 again once, there were few improvements done around this.
Thanks,

@pbafna03 - this has been confirmed:


fix(curriculum): Updated Cafe Menu Step 58 font casing

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Closes #53911

Hi @busolatom. Congrats on your first pull request (PR)! 🎉
Thank you for your contribution to the page! 👍
We are happy to accept these changes and look forward to future contributions. 📝


jsx description

I wanted an explanation of the jsx function, so I added it.

Thanks for your contribution. Just a few comments.


The timescale container is always restarting

I was trying to use the tool with the timescale pipeline. However, the timescale container is always restarting:
$ docker-compose -f ./timescale-docker-pipeline-reference-unenc.yml up -d
Starting activemq  ... done
Starting timescale          ... done
Starting grafana            ... done
Starting mysqldb            ... done
Starting timescale-ingester ... done
Starting telemetry-receiver ... done
$ docker-compose -f ./timescale-docker-pipeline-reference-unenc.yml ps
       Name                     Command                 State             Ports
--------------------------------------------------------------------------------------
activemq             /bin/bash -c bin/activemq  ...   Up           61616/tcp, 8161/tcp
grafana              /run.sh                          Up           3000/tcp
mysqldb              docker-entrypoint.sh mysqld      Up           3306/tcp, 33060/tcp
telemetry-receiver   /bin/sh -c cmd/idrac-telem ...   Up
timescale            /bin/sh -c cmd/initialize_ ...   Restarting
timescale-ingester   go run cmd/timescalepump/t ...   Up

Its docker logs are like this:
$ docker logs --tail 50 --follow --timestamps timescale
2021-12-07T01:27:26.172757282Z 	Is the server running locally and accepting
2021-12-07T01:27:26.172759726Z 	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
2021-12-07T01:27:27.092481235Z psql: error: could not connect to server: No such file or directory

Hello Artlands, This issue is fixed with the latest change. Could you verify with the latest code and mark this issue fixed? Do you see the timescale container restarting on a VM or independent physical machine?

Closing the issue based on the fix provided. Please contact us/re-open this issue if the problem is still observed.


FSX service controller

New ACK Service Controller
Support for FSX would be really great, especially when deploying Pods with NFS PVC's
List of API resources
List the API resources in order of importance to you:

FSX

/lifecycle frozen

/remove-lifecycle stale


Find an alternative to Vercel hobby plan used for PR/main doc publishing

Expected Behavior
Multiple detekt maintainers can manage doc publishing.

Current Behavior
Any changes to Vercel configuration need to be approved by Nico.

Context
detekt has multiple maintainers but only Nico can change Vercel configuration due to use of the hobby plan.
See #7550 for more.

Agree that this is not ideal:

Any changes to Vercel configuration need to be approved by Nico.

To be fair. The only settings we need to change on Vercel's dashboard is the Node version. We change that less than once a year.
All the other changes applied via the vercel.json file can be deployed by any maintainer (provided that it's on an detekt/detekt fork).
Anyway, the solution here could be to create an account on the @detekt.dev email and move the plan over there.

All the other changes applied via the vercel.json file can be deployed by any maintainer (provided that it's on an detekt/detekt fork).

So it didn't build in my PR because it wasn't a branch in detekt/detekt?
I think Vercel forces node updates periodically anyway so that's probably not actually a task that needs to be done in the config.
This isn't a high priority for anyone and like I said I think when GitHub eventually releases its own solution we move to that, I just wanted to open an issue so we don't forget.

So it didn't build in my PR because it wasn't a branch in detekt/detekt?

Yeah correct. I had to approve it because it was coming from your personal fork.

I think Vercel forces node updates periodically anyway so that's probably not actually a task that needs to be done in the config.
This isn't a high priority for anyone and like I said I think when GitHub eventually releases its own solution we move to that, I just wanted to open an issue so we don't forget.

Agree 👍

This issue is stale because it has been open 90 days with no activity. Please comment or this will be closed in 7 days.


Add Pprof Profiling to Existing Benchmarks

Resolves #272.
By the way running the benchmarks in the background and will update #228 once if this is approved.

Codecov Report
Base: 38.75% // Head: 38.68% // Decreases project coverage by -0.06% :warning:

Coverage data is based on head (3da996b) compared to base (c301cee).
Patch has no changes to coverable lines.

:exclamation: Current head 3da996b differs from pull request most recent head 2b3fccc. Consider uploading reports for the commit 2b3fccc to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #273      +/-   ##
==========================================
- Coverage   38.75%   38.68%   -0.07%
==========================================
  Files          93       93
  Lines        6087     6087
==========================================
- Hits         2359     2355       -4
- Misses       3728     3732       +4

Impacted Files
Coverage Δ

...rithms/linfa-trees/src/decision_trees/algorithm.rs
37.94% <0.00%> (-1.79%)
:arrow_down:

algorithms/linfa-nn/tests/nn.rs
75.60% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

The windows failures are because pprof won't compile correctly there. I had to use wsl to compile. Is there something like bench_dependencies for cargo so we can resolve this? Tried googling but didn't find anything

Actually just found docs

Do the benchmarks also output a text report?

actually, making the change still yielded compile time errors. So, I didn't push it but this is what is more or less was
[package]
name = "linfa-clustering"
version = "0.6.0"
edition = "2018"
authors = [
    "Luca Palmieri<EMAIL_ADDRESS>    "xd009642<EMAIL_ADDRESS>    "Rémi Lafage<EMAIL_ADDRESS>]
description = "A collection of clustering algorithms"
license = "MIT/Apache-2.0"

repository = "https://github.com/rust-ml/linfa/"
readme = "README.md"

keywords = ["clustering", "machine-learning", "linfa", "k-means", "unsupervised"]
categories = ["algorithms", "mathematics", "science"]

[features]
default = []
serde = ["serde_crate", "ndarray/serde", "linfa-nn/serde"]

[dependencies.serde_crate]
package = "serde"
optional = true
version = "1.0"
default-features = false
features = ["std", "derive"]

[dependencies]
ndarray = { version = "0.15", features = ["rayon", "approx"]}
linfa-linalg = { version = "0.1", default-features = false }
ndarray-linalg = { version = "0.15", optional = true }
ndarray-rand = "0.14"
ndarray-stats = "0.5"
num-traits = "0.2"
rand_xoshiro = "0.6"
space = "0.12"
thiserror = "1.0"
partitions = "0.2.4"
linfa = { version = "0.6.0", path = "../.." }
linfa-nn = { version = "0.6.0", path = "../linfa-nn" }
noisy_float = "0.2.0"

[dev-dependencies]
ndarray-npy = { version = "0.8", default-features = false }
linfa-datasets = { version = "0.6.0", path = "../../datasets", features = ["generate"] }
criterion = "0.4.0"
serde_json = "1"
approx = "0.4"
lax = "0.15.0"

[target.'cfg(not(windows))'.dev-dependencies]
pprof = { version = "0.11.0", features = ["flamegraph", "criterion"] }

[target.'cfg(not(windows))'.bench]
name = "k_means"
harness = false

[target.'cfg(not(windows))'.bench]
name = "dbscan"
harness = false

[target.'cfg(not(windows))'.bench]
name = "appx_dbscan"
harness = false

[target.'cfg(not(windows))'.bench]
name = "gaussian_mixture"
harness = false

Adding the corresponding cfg to the actual bench files didn't resolve the issue either.

The zip file only contains the flamegraphs. I guess the profiler doesn't output any extra text right? Also, Criterion still outputs graphs just like before right?
As for the format, I still prefer flamegraphs since that's what I'm used to, but ideally I'd like to have both flamegraphs and the proto format.

As part of this PR, can you also update the Linfa benchmark standards? We should do it once the profiling code is finalized.

The zip file only contains the flamegraphs. I guess the profiler doesn't output any extra text right? Also, Criterion still outputs graphs just like before right?
As for the format, I still prefer flamegraphs since that's what I'm used to, but ideally I'd like to have both flamegraphs and the proto format.

It is one or the other you either output flamegraphs or use criterion as the measurement method. See [here](https://bheisler.github.io/criterion.rs/book/user_guide/profiling.html#--profile-time) for documentation evidence. As for the format I believe it is either flamegraph format or proto format. I don't think you can do both without running the benchmark twice. Not a problem to add just not sure if the value add is worth it.

As part of this PR, can you also update the Linfa benchmark standards? We should do it once the profiling code is finalized.

yep once we agree on this PR being ready for merge I plan to.


Crash when decompiling a Unity APK

sorry if this might not be enough info to work with, it was running for ~15 minutes, gathered about 4gb of ram before it died, the game i was decompiling is "DATA WING" and i'm on windows 7 if that helps

Arm64 binaries just really don't like custom attributes at the moment, use the latest nightly from the new-analysis branch, and run cpp2il with the --simple-attribute-restoration argument, then you should get more usable output.


No selections found. If you are reading from the global accessors, try using the first argument instead.

I have some trouble with v3 $refetch...
I get the error
No selections found. If you are reading from the global accessors, try using the first argument instead.

But i didn´t find what it means or how can i solve it.
How ever if i call $refetch(true) this message is shown and nothing happens.
Thank you for your help.
Version: 3.0.0-alpha-7b21563c.0
React: true

It usually means you are using the V2 way import { query } from "gqty";
In V3 these should be used instead:

const query = useQuery(); in React, and
resove(({ query }) => { ... }); in core.

So the documentation. is wrong ?
https://gqty.dev/guides/react/read
i changed it like this but nothing changed...
at a second example he miss the array entity by refresh.
Inizial it execute. But after call refresh(true), it execute and an entity (from type array) is not longer a part of the query.

Do you mean the sub-selections under an array is missing when you call $refresh(true)?

@fasibio I'd like to reproduce the issue, please share more info when available!

Please reopen if this issue still exists.


Base: Review docs to prepare for v2.0.0

Docs are ready, just need to finish upgrading the production uis before releasing


Is it possible to fire an event with current position

Hi Nicholas,
Thank you for your previous assistance!
For bookmark purposes such as to record what was the last position an end-user watch stream till.
Is it possible to raise an even every X ms with current position of the stream?

Hello Nocholas,
Did you have a change to look into it?


ol.style.Circle high DPI support

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

@fredj Looking at the examples, it seems to me that this is fixed. Can you please check?

It's not fixed: https://github.com/openlayers/ol3/blob/master/src/ol/style/circlestyle.js#L150-L152
The pixelRatio value is ignored

Does this mean is not going to fix it? Circles look blurry on retina displays where everything else is sharp. This makes Circle totally unusable.


gitea fails to fetch woodpecker config from a directory

Component
server
Describe the bug
Steps to reproduce:

Create a file foobar/woodpecker.yml in a Forgejo/Gitea repository
Change the pipeline path to foobar/ in the Woodpecker next (as of today) interface
Push the branch containing the change

What happens

The interface shows a message that the config cannot be found
The Woodpecker admin logs show user defined config 'foobar/' not found

What is expected

The pipeline at foobar/woodpecker.yml runs

Additional information

When the pipeline path is set to foobar/woodpecker.yml, it runs as expected
The bug is likely in the gitea Dir function

Workaround
None.
System Info
woodpecker next

Additional context
No response
Validations

[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A join our Discord Chat Server or the Matrix room.

I try to fix this for Forgejo once/if https://github.com/woodpecker-ci/woodpecker/pull/1535 is merged.

I think this is still relevant, is it okay to reopen?

It is an unlabeled wontfix.

It is an unlabeled wontfix.

The question was more if he was okay with me reopening it.
This does need fixing, I am just asking if he is okay with me using his issue, or if I should open a new one.

For the time being I'm going to reopen this one, feel free to close if you'd rather I open my own for Gitea.


watch-admin not working in dockware/contribute image

If I try to run
make watch-admin
I get
BaseUrl for proxy is set to "undefined"

 ⚠  You need to add the "APP_URL" as an environment variable for compiling the code. ⚠

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>dev: `mode=development webpack-dev-server "http://<IP_ADDRESS>"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /var/www/.npm/_logs/2020-08-11T07_30_19_654Z-debug.log
makefile:19: recipe for target 'watch-admin' failed
make: *** [watch-admin] Error 1

Container runs with docker-compose:
sw:
        image: dockware/contribute
        container_name: sw
        ports:
            - "80:80"
            - "22:22"
            # Admin Watcher Port
            - "8888:8888"
            # Storefront Watcher Port
            - "9999:9999"
        environment:
            - PHP_VERSION=7.4
            - XDEBUG_ENABLED=1
            - SW_CURRENCY=EUR
            - PHP_IDE_CONFIG=serverName=PHPStorm

Same thing works great with dockware/dev:6.1.5

hi
thank you for sharing your issue
i did recently change the contribute version to use the psh watcher.
because for contribution the psh is still required
i just saw it hasn't been pushed yet
but can you try to use the psh watcher?
thanks

Hey,
thank you for your support. The psh watcher works fine.

Hi
awesome ,
thanks for your feedback
i've just pushed the new contribute image, that wraps the psh watchers in our watcher :)


JsonApiSerializer doesnt conform the specs for fetching relationship urls

i think this needs a whole resource type tbh but im sure they may be other ways too.
take this example:
/api/users/1?include=user_meta
This works great, and the data is formatted as expected.
however as per the spec each included/related resource has its own links which MUST be satisfyable as per the spec at:
/api/users/1/user_meta
/api/users/1/relationships/user_meta
The serializer does an amazing job of generating those links and requires nothing on our end to complete (which made me drool the first time i used it).
no lets satisfy either of those urls:
/api/users/1/user_meta
in my code, i have something like this:
//$meta is a collection of models (laravel)
$meta = functiontofetchusermetabyid($id);

return Fractal\Resource\Collection($meta, UserMetaTransformer::class, $this->type);

this will return something like this:
{
    "data":[
        {
             "type":"user_meta",
             "id":"1",
             "attributes":[..the atts...],
             "links":{
                  "self":"\/api\/user_meta\/1"
             }
         }
    ]
}

however this isnt what the spec says, it should look like this:
{
  "links": {
    "self": "/users/1/relationships/user_meta",
    "related": "/users/1/user_meta"
  },
  "data": [
    { "type": "user_meta", "id": "1" },
    { "type": "user_meta", "id": "2" }
  ]
}

There are a few issues here:

it should have a top level links object
resource objects shouldnt have any attributes
resource objects shouldnt have a links object

Ive reread the docs a few times, and its not right:
A "resource identifier object" MUST contain type and id members.
A "resource identifier object" MAY also include a meta member, whose value is a meta object that contains non-standard meta-information.
there are other places in the docs that it also mentions nothing about including resources for these requests at all, so basically the response should only contain:

a data object with nested resource objects
a links object containing the current link and related link.

Im currently having to prepare a custom response and not use fractal for this, but it would be great if it was possible.
Todo so  i propose the addition of a new resource type, currently there are:

null resource
item resource
collection resource

The new resource type should be a relationship resource
I think this would be need to implemented as a resource class, in the scope class, and then within the serializers as well, but looking at it, it doesnt look overly hard to implement.
Then we could all use:
$resource = new Fractal\Resource\Relationship($books, new BookTransformer, 'books');

which would format the data as expected.
Im happy to have a go at making a pull request for this if you agree this is a needed addition.

just a few more notes:
the transformer used could be any existing or new transformer, but in reality all it needs to do is return either an item resource with an id, or a collection resource with an id.
Any other data returned would just be ignored.
optimally a transformer which only proccessed the item by returning its id would be favourable, but re-using an existing transformer for the item would (shouldnt) break anything as all we need is the id key returned in the array.
We could enforce this via interfaces, but even though its favourable to have a different transformer for optimisation this should be left down the end user.

I agree completely with this, I'm adding it to my todo list right now

@willishq Oh, yes. There are critical troubles with relationships :(

I simply can't include ONLY relationships links into object. For example, I have countries and cities, I don't want to include thousands of cities in country payload, but I should include a top-level relationships field  with links to nested city resources (e.g. /countries/1/cities). Now there is no possibility for that
Even when base url for serializer is provided and relations are included, relation does not contain links:

data: [
{
  type: "countries",
  id: "19",
  attributes: {
  name: "Австралия",
    currency_code: "AUD",
    currency: "Австралийский доллар"
},
  links: {
    self: "http://desk.nalivator.app/countries/19"
  },
  relationships: {
    cities: {
      data: [
        {
          type: "cities",
          id: "3979034"
        }
      ]
    }
  }
}

],
included: [

This makes almost impossible to work with relationships with spec-fullfilling clients (like ember.js for example)

Is there any update on this? I am attempting to implement an API following the JSON-API spec and have hit this road block with Fractal.

Same here, would be very interested in knowing how this is getting along. @willishq?

Anything news on this?

I'm also interested about any news on this?

At this point I believe it won't happen, sadly.

porco dio

Any updates?


refactor(card): Separate css vars

fixes https://github.com/patternfly/patternfly-next/issues/1152

Deploy preview for pf-next ready!
Built with commit 3dd8defa4cfad79216811164025c58b48e545a2f
https://deploy-preview-1153--pf-next.netlify.com


noexec mountpoints

Stemcells mount /tmp and /var/tmp with noexec, which causes some heartburn for tools like bosh-init, which unpack things to /tmp
We should remount these mountpoints in watcher, as such:
sudo mount -o remount,exec /tmp
sudo mount -o remount,exec /var/tmp

Testing has found that the main problem with bosh-init may have actually been the length of the $HOME path...

v4.2.1 has a new /u/$name approach.  Will test in the lab to ensure that this works with bosh-init, and if so, we can close this issue.

Haven't had this problem since the /u/$name fix


feat: consistent flatModuleFile naming for bundles

I'm submitting a...
[ ] Bug Fix
[x] Feature
[ ] Other (Refactoring, Added tests, Documentation, ...)

Checklist

[x] Commit Messages follow the Conventional Commits pattern

A feature commit message is prefixed "feat:"
A bugfix commit message is prefixed "fix:"

[x] Tests for the changes have been added

Description
Albeit this is not 100% conformant to the file naming suggested in the Angular Package Format spec, it makes the flattened JavaScript bundles more identifiable as they include the all information bits of the module ID.
Does this PR introduce a breaking change?
[ ] Yes
[x] No

Big thanks again for getting this in so fast!!


Qt Quick Controls 2.0 chapter [WIP]

This is work in progress - do not merge!
I've outlined where to add Qt Quick Controls 2.0 references and a general outline for the main chapter on Qt Quick Controls 2.0 and am looking for feedback.

Good feedback - thanks!
I'm more after high level feedback on the flow. Does the high level story hold (answering all the bullet questions in this order) and have I covered all the key topics?

Good feedback - thanks!
I'm more after high level feedback on the flow. Does the high level story hold (answering all the bullet questions in this order) and have I covered all the key topics?

I think it looks fine.
Some other things that may be worth mentioning when you get to the content stage:

Palette support, although not supported for each style yet, might be worth mentioning as the best solution for tweaking colours without overriding whole delegates and hence losing the style.
The imagine style would be worth mentioning as a designer-friendly solution for creating a style without having to code.

@mitchcurtis - thank you for all your review comments. I believe that I've addressed all of them (some with comments). The only discussion that I'm not sure if we can close is the one about using QtQuick.Dialogs before Qt.labs.platform. The text explains why, but it does introduce a Qt Quick Controls 1 dependency, to avoid a QtWidgets dependency.
My proposal is that we merge this as a version 1.0 and improve it later on.
I propose is that we insert this as a new chapter 7, i.e. after Model-View-Delegate but before Canvas Element. The chapter does use models and views, so the order makes sense. It also uses dynamic element instantiations (chapter 13) but I think that we have to treat that as an advanced topic.
Do you agree?

Sounds good. I'm not 100% sure how to view the diff with my comments inline so I haven't reviewed the latest changes thoroughly, but we (The Qt Company) can always send pull requests afterwards if something needs fixing.


Implemented a single columned list for the kubernetes metrics reference page

This PR closes #42342 and #38438
As #38438 demands a generator code change, I've changed the generator codes for the metrics.md
(No manual changes have been made to the metrics.md rather than the ones generated by the generator itself!)
I'll be linking the PR from k8s/k8s below once I finish making it
(Edit) The pull request for generator code change: https://github.com/kubernetes/kubernetes/pull/120350
(Edit) https://github.com/kubernetes/kubernetes/pull/120350 has been merged successfully
Appropriate styling changes have been made as per the generated html, As requested in #42342

/hold
pending merge of https://github.com/kubernetes/kubernetes/pull/120350

@sftim I'm seeing some merge conflicts on metrics.md, Should we be changing anything in other PRs as well?

The new format looks awesome, thank you @Gauravpadam! :smiley:

The new format looks awesome, thank you @Gauravpadam! 😃

Thanks @dgrisonnet, Glad you liked it! 😃

@sftim I'm seeing some merge conflicts on metrics.md, Should we be changing anything in other PRs as well?

See github.com/kubernetes/website/pull/42879

/label tide/merge-method-squash
/lgtm

@tengqm @sftim We're good to merge this PR, Both https://github.com/kubernetes/kubernetes/pull/120350 and https://github.com/kubernetes/kubernetes/pull/120369 have been merged successfully into k8s/k8s

/unassign @tengqm

/assign @sftim

“unassigned" ... ;)

Thanks for the review @tengqm!

/hold cancel
Good to go now.
/lgtm

Thanks for the reviews @sftim @tengqm!

Preview: https://deploy-preview-42823--kubernetes-io-main-staging.netlify.app/docs/reference/instrumentation/metrics/
/approve

This was reverted in #44281 and would be reinstated by https://github.com/kubernetes/website/pull/44286


Universal back and forward doesn't work in ForkLift

Universal back and forward option doesn't work in ForkLift. Can you fix this or is it better to contact the ForkLift developer about this?

Thank you for reporting this issue!


action user.formatters_reformat_selection(user.formatters) deletes a character if nothing is highlighted

The command is referenced here: https://github.com/knausj85/knausj_talon/blob/0414eddd348a0314101978c4ed7eaf3cbfc00373/misc/formatters.talon#L8

it also seems to delete the preceding character when text is highlighted.


Error: No compatible version found<EMAIL_ADDRESS>

Installing fails due to an incompatible typescript version.
 Error: No compatible version found<EMAIL_ADDRESS>npm ERR! Valid install targets:
npm ERR! ["0.8.0"]
npm ERR!     at installTargetsError (/usr/share/npm/lib/cache.js:488:10)
npm ERR!     at /usr/share/npm/lib/cache.js:375:15
npm ERR!     at saved (/usr/share/npm/lib/utils/npm-registry-client/get.js:147:7)
npm ERR!     at Object.oncomplete (/usr/lib/nodejs/graceful-fs.js:230:7)
npm ERR! You may report this log at:
npm ERR!     <http://bugs.debian.org/npm>
npm ERR! or use
npm ERR!     reportbug --attach ./npm-debug.log npm
npm ERR!
npm ERR! System Linux 3.2.0-57-virtual
npm ERR! command "node" "/usr/bin/npm" "install"
npm ERR! cwd ./tern_for_vim
npm ERR! node -v v0.6.12
npm ERR! npm -v 1.1.4
npm ERR! message No compatible version found<EMAIL_ADDRESS>npm ERR! message Valid install targets:
npm ERR! message ["0.8.0"]
npm http 200 http://registry.npmjs.org/acorn/-/acorn-0.11.0.tgz
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     ./npm-debug.log
npm not ok

It probably isn't relevant, but this was after setting a non-ssl registry using npm config set registry http://registry.npmjs.org/.

Also, I realize that this might be an issue with tern itself, not the vim plugin, but that's where I saw this issue. I can move this issue to the tern repository if desired.

For me, running npm install in the tern_for_vim directory works just fine. Maybe try clearing your npm cache (or upgrading if your npm is old -- I had weirdness like this when using an old npm once).

Oh I bet it's an old version of npm. I was trying to install this on an older Ubuntu 12.04 machine.
❯ npm --version
1.1.4

Locally, I have version 2.1.17. I'll try to get a newer version of npm and retry. Thanks for looking into this.

Installing on an Ubuntu 14.04 machine where the npm package version is 1.3.10 works just fine. Thanks for taking a look and sorry for the false alarm. Hopefully this helps anyone else who runs into this issue with an older version of npm.


Unable to validate CPHD data because parse_xml_from_string returns xml_ns=None

Have been unable to read some recently received CPHD files. I believe they are version 1.1.0. Admittedly I don't understand enough about XML namespaces. However, in base.py there is an attempt to Iterparse the XML extracted from the CPHD. There seems to be no start-ns events generated. Does this imply malformed CPHD? Seems like there is code in parse_xml_from_string to address this case by checking the length, but if xml_ns = None then I think CPHDType::from_xml_string errors out looking for 'default' in None.

It is difficult to say without seeing the file, but it does sound like the XML could be missing a namespace declaration. The version-specific CPHD namespaces are defined in the accompanying schema files (e.g. nsgreg link for CPHD 1.1.0 schema):
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
   elementFormDefault="qualified"
   attributeFormDefault="unqualified"
   targetNamespace="http://api.nsgreg.nga.mil/schema/cphd/1.1.0"        <----- this
   xmlns="http://api.nsgreg.nga.mil/schema/cphd/1.1.0">
...

There are a few ways to declare namespaces in XML 1.0. Perhaps the simplest is to specify the default namespace in the root element, as can be seen in this syntax-only test XML file
<CPHD xmlns="http://api.nsgreg.nga.mil/schema/cphd/1.1.0">
...

As noted in NGA.STND.0068-1_V1.1.0 _CPHD Section 2.1:

The two-byte section terminator “\f\n” (0x0C, 0x0A) marks the end of the File Header. The section terminator enables a reader application to identify the end of the File Header block and enables viewing of the file header using standard functions on some systems (e.g., the “more” command on a UNIX system).

This property makes it possible to inspect the version of the CPHD file as well as to navigate to the portion of the file that contains the XML block using readily available tooling, e.g.
$ more /data/bistatic_spotlight_example.cphd | head -n 3
CPHD/1.1.0
XML_BLOCK_SIZE := 24846
XML_BLOCK_BYTE_OFFSET := 1024

$ tail -c +1024 /data/bistatic_spotlight_example.cphd | head -c 100
<CPHD xmlns="http://api.nsgreg.nga.mil/schema/cphd/1.1.0"><CollectionID><CollectorName>SyntheticCol

Hopefully this gets you closer to resolving the issue.

Thanks for the quick response. Using the head/tail commands provided, clear (now) that the namespace schema was not specified in the XML. The version seems to be 1.1.0, so... I guess I can try to hack the schema in and see if that gets further.
FWIW I altered CPHD.py to test xml_ns before looking for 'default'.
@classmethod
    def from_xml_string(cls, xml_string):
        ...
        root_node, xml_ns = parse_xml_from_string(xml_string)
        if xml_ns == None:
            ns_key = None
        else:
            ns_key = 'default' if 'default' in xml_ns else None
        return cls.from_node(root_node, xml_ns=xml_ns, ns_key=ns_key)

Makes it a further, but then dies trying to parse the CODTimePoly and DwellTimePoly. I only mention this because it does seem like parse_xml_from_string can legitimately return a None and the error check here might be necessary.
I'm now afraid that there are issues beyond the missing namespace schema, but as I mentioned will try to force the schema for now and see if that helps.

Yup. Still dies trying to convert CODTImePoly and DwellTimePoly even if I force the namespace to match the 1.1.0 standard. I'll see if I can work that a bit.
From my perspective can close the ticket unless you want to consider the error handling adjustment above.

Glad to hear you're a bit closer and thanks for the note. We agree that there are some odd edge cases with respect to the XML parsing/handling as you've pointed out. We'll likely leave this as is and only fix critical bugs (reading valid files) as we're working on the next major release of SarPy that has a much lighter-weight metadata model.
If you encounter any further issues, let us know.


SB-942: Upgrade to the official release of org.apache.maven:maven-indexer:6.0

Upgraded.

(It may take a few more hours before the artifact becomes available on Maven Central).

@cs-devops-bot rebuild

@cs-devops-bot rebuild


Do not include buildTransitve/buildMultiTargeting by default

Target bugfix branch

@forki should actually fix 5.X.X this time

thanks.


[SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

What changes were proposed in this pull request?
Inline type hints for python/pyspark/profiler.py
Why are the changes needed?
We can take advantage of static type checking within the functions by inlining the type hints.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Existing tests

Test build #145693 has started for PR 34731 at commit b947cc4.

Test build #145693 has finished for PR 34731 at commit b947cc4.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Let's get https://github.com/apache/spark/pull/34685 done first.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50163/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145693/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50163/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50163/

Test build #145743 has started for PR 34731 at commit 48e355e.

Test build #145745 has started for PR 34731 at commit 27207aa.

Test build #145743 has finished for PR 34731 at commit 48e355e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #145745 has finished for PR 34731 at commit 27207aa.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145745/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145743/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50213/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50216/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50213/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50213/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50216/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50216/

@dchvn Maybe could you rebase onto or merge master branch and see what happens?
@zero323 Now that #34363 has been merged. Could you give it another pass?

@dchvn Maybe could you rebase onto or merge master branch and see what happens? @zero323 Now that #34363 has been merged. Could you give it another pass?

Sure thing @ueshin. I'll review it after it is synced with master and  https://github.com/apache/spark/pull/34731#pullrequestreview-819670002 is addressed.

@dchvn Maybe could you rebase onto or merge master branch and see what happens? @zero323 Now that #34363 has been merged. Could you give it another pass?

Sure thing @ueshin. I'll review it after it is synced with master and #34731 (review) is addressed.

Updated, thanks!

Looks pretty good to me except https://github.com/apache/spark/pull/34731#pullrequestreview-819670002

Merged into master.
Thanks everyone!

Thanks all! ❤


Broken Background Image Link

The URL used for the background image by the maintainer is broken
https://www.survivingwithandroid.com/wp-content/uploads/2014/11/android_weather_app.jpg
That's why the website is looking like this
Need to use some other image instead of it

@1010varun
Please assign me this issue under IWOC and check the PR made.


Sign the OSX app

Currently the app isn't signed, so gets flagged under the unknown developer security thing.

+1
Tried making Comma Chameleon my default tool to open CSV files (1) for a demo (2) because Excel mangles data, but...

Saw this tool as highly recommended, but can't even open it.

@drchriscole you can right click then option-click on open to enable the app, but yes, this is something we need to fix properly.

Looks like https://www.npmjs.com/package/electron-osx-sign might be the thing to use

@davetaz am I right in thinking we already have an apple developer license for the ODI because of the mobile apps?

Correct, you should be an admin on it I think. Check your apple id at developer.apple.com. You should then be able to provision an app profile and somehow get the certificates needed for solving this.

@davetaz brilliant, thanks.

Just clocked this. Just one issue to flag, we might need to investigate how the releases are built as I think you need to build on OSX if you want to sign an app for OSX. I'm pretty sure Travis has some OSX images though

Yep, need to build (or at least sign) on OSX. Hadn't considered doing travis build on OSX, that's an interesting possibility; I was thinking it would have to be a manual process (which obviously would suck). I'll take a look.

electron-packager (which we use already) uses electron-osx-sign so there's no need to change that. Just need to be on the right OS and have the certificate.

Ace 👍

@Floppy thanks. The right-click allowed me to run it. Nice trick! Thanks.

So I cannot duplicate the unidentified developer warning message - can someone who had this problem previously confirm if their Security & Privacy settings were set like so?

To re-create:

download app
move to application directory
go to finder, right click on .csv file
choose open with Comma Chameleon

Thanks @Stephen-Gates, for whatever reason I can't replicate it exactly but I do have something that will let me check if my signing remedies it

I've used RB App Checker Lite to verify if the app is being signed correctly. For comparison here is the current package as generated by the gulp script, then that same package signed with electron-osx-sign

The above signature differences are replicated in a more manual signing flow where electron-packager and electron-sign are utilised
While the correct signatures have been applied I am seeing no difference in behaviour in terms of being the default opening app. I have updated releases - only MacOS Comma Chameleon-darwin-x64 has been signed - from this related branch if anyone here would care to test if any behaviour has changed. For now the same build flow has been utilised to update the releases page by a minor increment. https://github.com/theodi/comma-chameleon/releases/tag/0.5.2
Given I expect that the signing won't resolve the problems indicated thus far I think the issues in this ticket need separating out, one for app showing up as a Signed Identified App, and the other to relate to setting electron app as default for opening CSV files. There is also a question of how the current utilisation of csvlint should work - as a binary it should also be signed.
It may be necessary to change the build process wholesale by switching to electron-builder. It seems a lot of people defer to this after encountering many of the same issues I encountered trying to get this set up. It also has support for yarn, and may play well with dependabot (cc @floppy)

@quadrophobiac
Downloaded App

Used Finder to open a CSV with Comma Chameleon

Didn't like it

Adjusted settings and still couldn't open csv file from finder but could open using File > Open in Comma Chameleon

Yeah that's the same issue I encountered. I've researched this and I don't see that there's anything else to be done in terms of signing.

So is the solution to submit to the App Store?

In short I don't know at present - my gut says no, because mac app store and non-app store distribution are both supposed to be supported. Trouble is most guides assume that xcode is being used to sign apps

@Stephen-Gates can you try these steps

download the Comma Chameleon link I emailed you (it's a way of mocking the quarantine steps here)
Set your gatekeeper to Allow apps downloaded from: App Store
You should see this alert

Change the Gatekeeper settings to Allow apps downloaded from: App Store and identified developers
You should get

and thereafter the app should open fine.

None of this resolves the 'set as default CSV opening app' but I think it confirms that the app is displaying the requisite behaviour for a developer signed app.
Given the restrictions that Sierra brought to Gatekeeper working towards a Mac App store distribution is definitely the best solution to this problem overall. It's also possible that sandboxing might be needed, and doing that is part of the Mac App Store distro process. So if you can confirm that you see the above Stephen I'll move to close this issue

Yep. I get the some as above @quadrophobiac

In response to the initial ticket

Currently the app isn't signed, so gets flagged under the unknown developer security thing.
I think this is a separate issue to the open CSV by default. It's unclear from electron documentation how to accomplish this without submitting to the App Store. Creating a sandboxed app with the correct manifest details may resolve it. I noticed a closed issue - https://github.com/theodi/comma-chameleon/issues/75-  on the same request, may warrant reopening it in light of this lack of functionality.
In the meantime including the 'right click workaround' within the readme seems the most expedient solution to the problem


Document N+1 mitigation techniques in the book

Is your feature request related to a problem? Please describe.
Now, resolver signature looks like:
fn field(&self, /*args*/).
This can easily lead to N + 1 DB query problem. (Or even N**K if query has K levels of depth). To avoid this problem, special libraries should be used, that somehow accumulate queries and merge them into one.
Describe the solution you'd like
(Idea itself is very simple. It is possible that is was already discussed and rejected, but I didn't find such an issue). I propose to allow batched resolvers. Signature now looks smth like:
fn field(self: &[Self], ctx:&Ctx, args: ...)
Now, from N**K requests we go to K (each level will be processed with one resolver execution and one db request).
Describe alternatives you've considered
Alternative: do nothing.
However, I believe batched resolvers  provide good enough performance with high flexibility.

@MikailBag for N + 1 query problem the usual solution is either dataloader or lookahead.
As for proposed solution I'm unsure how well it will fit. Usually, resolvers may contain allowed-resoling-depth checks, authorization-rights check, authentication checks, etc, which all are query-scoped. Resolving multiple queries at the same time, but now with the same context, seems to be less flexible.

If you're interested in the lookahead approach you could consider juniper-eager-loading which handles all the boilerplate. We're using it in production at @tonsser and its been working well so far. Full disclosure: I'm the maintainer of the library.

As mentioned by others, the possible approaches here are

datalaoder
using lookahead
uniper-eager-loading

We should really add to the book, since it's a common stumbling block.

Thanks for the responses!
I will study your proposals (so far juniper-eager-loading looks the most promising).
Regarding book, I would like to see quite complex example, e.g.:

Response data are aggregated from DB and in-memory source (e.g. calculated).
Application doesn't directly use diesel, it uses some trait, MyAppDbProvider.
And without too much glue code :)

@MikailBag I maintain a complete example app. You can find it here https://github.com/davidpdrsn/graphql-app-example. It does use Diesel directly however.


fix: do not set mode name into variables

see that https://github.com/kiveai/kive/pull/3141#issuecomment-2392997630

Do you wanna commit the dist folder?

I was thinking about the fact if we commit it, we could just download the repo and import a plugin w/out running scripts. The folder will be "ready to import into figma"
What do you think?

We already have them committed in the original repo and I think it is nice that you can just download a zip folder with repo and import a plugin into Figma w/out running any scripts, what do you think?

We already have them committed in the original repo and I think it is nice that you can just download a zip folder with repo and import a plugin into Figma w/out running any scripts, what do you think?

ok 👍