Build to .NetStandard (#90)

This PR is from a new branch to one mentioned in #90 - as previously mentioned branch had loads of trial and error noise in the commit history.
Currently targets netStandard1.3, tests target net461; running correctly in visual studio and travis via command line.
netStandard1.3 output can be consumed by a framework 4.6+ application. Referencing the source code directly would require adding a nuget package reference to System.Net.Http v4.3.3
my #90 comments go on a bit... so feel free to just ask any questions?

@jonnii mind helping me look at this?

FWIW other than the build stuff this looks pretty good to me, I think the majority of your "problems" are from running the CI on travis... it's great, but I've had a much easier time in .net land using appveyor. Here's an example of a script which builds and versions packages using netcore/standard and gitversion.
I'm not 100% sure if it's even possible to get a net461 and netstandard (with multiple target frameworks) to work linux, I think you have to do that on windows. It may be easier to just target netstandard2.0 and be done with it... that may be the easiest quickest way to resolve all these issues.

e.g. if you did this https://docs.travis-ci.com/user/languages/csharp/#.NET-Core you could create a matrix build for netcore2.0 and netcore2.1...

Updated; now building net461 & netstandard2.0 + running tests for each.

Travis Build
I've looked at the matrix concept. The matrix builds an array of parameters, with one build per unique combination of parameters, with builds running in parallel.
This doesn't help us too much, because the root of the issue is that we need to build and/or test the net461 DLL in a different way to the netcore DLLs. (different as in not just parameter changes executed the same way by the same script)
The dotnet command is part of the CLI toolset, and is separate from msbuild.
This vstest issue tracks what needs to be "fixed" in the CLI and/or msbuild tools, to address the root of the problem we're faced with.
I think where I've ended up is a reasonable solution.
We use msbuild to restore / generate the output.
We use mono to run the net461 tests using the command line runner
We use dotnet test (crucially with --no-build option) to run the netcoreapp2.0 tests
Visual Studio Build
Looking at the logs.... Test Explorer seems to only run the tests against the net461 dll. On my machine it fairly often fails with a non fatal Out of Memory error, when trying to discover tests after a build... but manually clicking "Run All" in Test Explorer always works.
It would be straightforward to run the netcoreapp2.0 tests via VS2017 command prompt with something similar to this; depending on what you need - with or without the --no-build flag
dotnet test GDAXSharp.Specs/GDAXSharp.Specs.csproj --no-build --configuration Debug--framework netcoreapp2.0

Further, if you're flipping between branches - and (say) /master is the new format csproj and /feature/xyz is the old format /obj --> you'll have to delete the /obj folders each time you swap over.

Use git clean -fxd it is your friend in this situation.
What's the story with packaging and deploying a new version? Is that automated too? With the new tooling you can use dotnet pack and dotnet nuget push... the pack also takes care of the build for you too and puts everything in the right framework dependent subfolder.

Use git clean -fxd it is your friend in this situation.

cool... historically I've always used svn so these tips are great.
Re packaging... no idea... one for @dougdellolio. I've assumed its a manual process to date.
dotnet pack would need to be run with the --no-build flag, otherwise it will fail on Travis due to the linux / net461 framework issue. With --no-build it should be able to pickup the content from the msbuild step.
So... yeah... automating that part should be possible...
Question - how does that packaging / push to nuget work when people fork? For example, I've been running this build out of my fork, on travis... how does it stop those builds from getting pushed to NuGet ?

Although maybe packaging etc. should be a separate PR, which considers moving to Appveyor?

So what I do on my own projects is this in appveyor:
deploy:
  provider: NuGet
  skip_symbols: true
  api_key:
    secure: Z1QcGBawSRfoLPWCtkhcZEMD4f+QLbw2o01T/BEuHL9hERKTSrVOutzAp8QCyDIf
  on:
    appveyor_repo_tag: true

The condition is to push to nuget when there's a new tag. Tags are used to control versioning, so you can create a release in github and it will automatically tag and push to github. This is controlled using gitversion. You can do something similar with travis i'm sure.

That makes sense.... but is that exposing the key which means anyone could mess with the nuget package and publish rouge releases?

In appveyor that's an encrypted secret.

all nuget releases I have done have been manual. I’d rather not release a package for every single merge for this project as I don’t think it’s necessary. Had busy day today so didn’t get a chance to look into appveyor and building there but will hopefully get to it tomorrow.

@dougdellolio you can set it up so only the tagged or labelled releases get published, so you can pick and choose when along with seeing the diff between versions.

@chrisw000 i have added a build to appveyor #130 . is there anything you want me to try or need me to do?

I believe it should "just work", merge it into /master if you're done and I'll pull into my branch

Travis failure due to it being unable to download a mono package. I don't think this has anything to do directly with this project - potentially a travis issue?
AppVeyor built and passed incredibly quickly.... that build output all looks good for .netStandard2.0 and net461 + 2 sets of tests passing.
Should I just remove the Travis file or do you want to keep both for now?

i tested this with an existing app and everything seems fine... i tried running tests through visual studio and get OutOfMemoryException. Resharper runs tests fine. any idea? otherwise I am ready to merge this

Yes, I mentioned this on the other thread. In Visual Studio, I got OutOfMemoryExceptions when it tries to auto discover the tests after a build.
If you click "Run all Tests" manually it will discover, run and pass all the tests OK.

@chrisw000 thanks for all of your work here

actually... just noticed... the .sln file needs a minor tweak - remove the link to the travis file - and put in the appveyor file.... just to tidy it up.


Face Detection using Haar Cascade

@akshitagupta15june Added the model for Face Detection using Haar Cascade.
Section- Face Detection (Issue-89)
Please check it.

Can you add a proper readme and the mention issue number which you have solved?

Yes, please check it.

your pull request will be reviewed by mentor @mayank8200

your pull request will be reviewed by mentor @mayank8200

Okay, thanks!

please add a readme file @srimontidutta

please add a readme file @srimontidutta

On Haar Cascade?
Also, in a separate PR containing only the readme or the code files as well?


Add RPC metrics emitter as Observer/SpanObserver

Coverage decreased (-1.7%) to 80.337% when pulling f24565db8dca00906839eaef7658b23c0166c5bf on metrics-observer into d25ce142fa357c3d8f725ef21574f8aa161df117 on master.

Coverage decreased (-1.7%) to 80.337% when pulling f24565db8dca00906839eaef7658b23c0166c5bf on metrics-observer into d25ce142fa357c3d8f725ef21574f8aa161df117 on master.

Coverage decreased (-1.6%) to 80.412% when pulling a9f909361935d8eef92df46bc75b382aa745fc14 on metrics-observer into d25ce142fa357c3d8f725ef21574f8aa161df117 on master.

Coverage decreased (-1.6%) to 80.412% when pulling a9f909361935d8eef92df46bc75b382aa745fc14 on metrics-observer into d25ce142fa357c3d8f725ef21574f8aa161df117 on master.

Coverage decreased (-1.7%) to 80.271% when pulling 14b55d9dfef3e7a59172670926ec7c66b826afdb on metrics-observer into d25ce142fa357c3d8f725ef21574f8aa161df117 on master.

Coverage decreased (-1.7%) to 80.271% when pulling 14b55d9dfef3e7a59172670926ec7c66b826afdb on metrics-observer into d25ce142fa357c3d8f725ef21574f8aa161df117 on master.

Coverage decreased (-1.5%) to 80.492% when pulling 024fac394e6021461daeb7cf65f42664b5331595 on metrics-observer into d25ce142fa357c3d8f725ef21574f8aa161df117 on master.

Coverage decreased (-0.9%) to 81.227% when pulling f1f36e86a80d0cf88a3fbab559d057d68144d5df on metrics-observer into 0505b4d1de2bb75570485613940873c9f7c36135 on master.

Coverage increased (+1.1%) to 83.229% when pulling 2ab6f8edebfefd1684304383fe82329759a9f11a on metrics-observer into 1f64068568678437210fc4fe84f2faa3fdb813a0 on master.

Coverage increased (+1.1%) to 83.229% when pulling 2ab6f8edebfefd1684304383fe82329759a9f11a on metrics-observer into 1f64068568678437210fc4fe84f2faa3fdb813a0 on master.

Coverage increased (+1.1%) to 83.229% when pulling 09d8e2b4df1a0fef4241dfe83c719931c5949a6d on metrics-observer into 1f64068568678437210fc4fe84f2faa3fdb813a0 on master.

Coverage increased (+1.1%) to 83.229% when pulling 82004631cab15f697f1736b009e6b1e8babf8d0d on metrics-observer into 1f64068568678437210fc4fe84f2faa3fdb813a0 on master.

Coverage increased (+1.1%) to 83.229% when pulling e1df9f229563551a959ea81f9ba5c3c23094a899 on metrics-observer into 1f64068568678437210fc4fe84f2faa3fdb813a0 on master.

Coverage increased (+1.2%) to 83.272% when pulling adc742b4779669ade2277e347fcf5128363935f5 on metrics-observer into 1f64068568678437210fc4fe84f2faa3fdb813a0 on master.

Coverage increased (+1.2%) to 83.272% when pulling c5ad810aeb0af7ec450e0e6626fbd8bc6ca4233e on metrics-observer into 1f64068568678437210fc4fe84f2faa3fdb813a0 on master.


[Epic]: Improve Quality Assurance

This Issue gives a view over all issues that should improve quality assurance.
It also includes issues from other repos.
The issues are ordered by priority.

[ ] https://github.com/zitadel/website/issues/771
[ ] https://github.com/zitadel/website/issues/772
[ ] https://github.com/zitadel/typescript/issues/7
[ ] https://github.com/zitadel/typescript/issues/8
[ ] https://github.com/zitadel/typescript/issues/9
[ ] https://github.com/zitadel/zitadel/issues/5831
[ ] https://github.com/zitadel/zitadel/issues/5840
[ ] https://github.com/zitadel/zitadel/issues/5841
[ ] https://github.com/zitadel/zitadel/issues/5854
[ ] https://github.com/zitadel/schema/issues/3
[ ] https://github.com/zitadel/zitadel/issues/5858
[ ] https://github.com/zitadel/website/issues/770
[ ] https://github.com/zitadel/website/issues/773
[ ] https://github.com/zitadel/zitadel/issues/5833
[ ] https://github.com/zitadel/zitadel/issues/5834
[ ] https://github.com/zitadel/zitadel/issues/5842
[ ] https://github.com/zitadel/zitadel/issues/5843
[ ] https://github.com/zitadel/zitadel/issues/5846
[ ] https://github.com/zitadel/zitadel/issues/5847
[ ] https://github.com/zitadel/zitadel/issues/5855
[ ] https://github.com/zitadel/zitadel/issues/5857
[ ] https://github.com/zitadel/website/issues/774
[ ] https://github.com/zitadel/zitadel/issues/5853
[ ] https://github.com/zitadel/typescript/issues/10
[ ] https://github.com/zitadel/zitadel/issues/5848
[ ] https://github.com/zitadel/actions/issues/17
[ ] https://github.com/zitadel/schema/issues/4
[ ] https://github.com/zitadel/terraform-provider-zitadel/issues/93
[ ] https://github.com/zitadel/zitadel/issues/5835
[ ] https://github.com/zitadel/zitadel/issues/5836
[ ] https://github.com/zitadel/zitadel/issues/5837
[ ] https://github.com/zitadel/zitadel/issues/5838
[ ] https://github.com/zitadel/zitadel/issues/5839
[x] https://github.com/zitadel/zitadel/issues/5859
[x] https://github.com/zitadel/zitadel-charts/issues/85
[x] https://github.com/zitadel/terraform-provider-zitadel/issues/92
[x] https://github.com/zitadel/typescript/issues/6
[x] https://github.com/zitadel/zitadel/issues/5832
[x] https://github.com/zitadel/zitadel/issues/5844
[x] https://github.com/zitadel/terraform-provider-zitadel/issues/90
[x] https://github.com/zitadel/terraform-provider-zitadel/issues/91
[x] https://github.com/zitadel/zitadel/issues/5845
[ ] https://github.com/zitadel/zitadel/issues/5849
[x] https://github.com/zitadel/zitadel/issues/5850
[x] https://github.com/zitadel/zitadel/issues/5851
[x] https://github.com/zitadel/zitadel/issues/5852

@eliobischof Which of these issues should we have a look next?
Are all still relevant or can we close some?
I would assume that we only have a look at the new apis for now, and leave the old ones out, at the momenet.

@hifabienne the ZITADEL API prefixed issues in this list don't differentiate between new and old API. Would you rather create separate issues? As they are not really different in their nature, I'd keep them together for consistency.

I increased the priority of the portal tests, as the portal becomes more and more important.
I decreased the priority for testing the old login UI
I Increased the priority of completing any unit/integration tests over any acceptance tests, as this enables us to increase the coverage faster

@hifabienne I've created another Epic for the customer portal. Do you want to remove the related sub-issues from this Epic?

@hifabienne I've created another Epic for the customer portal. Do you want to remove the related sub-issues from this Epic?

yes


Chemical injection chamber not accepting hydrogen chloride

couldn't find anything on the web about this nor any other threads here but trying to get injection chamber to accept the input but its not happening. any suggestions?

Your first screenshot shows the item tab, not the gas tab. Please check that the Chemical injection chamber is set to input gas from the top.

okay will do when i get back to my system in a few days, currently not available

Gas tab

By the look of your pipes there's already a gas in it, hydrogen chloride is light blue (as seen in the C.Infuser). Please break all the pipes & try again.

Hello, i had the same problem, injection chamber did not accepted hydrogen chloride from any side, but only from pipes, when i put a gas tank next to it, and set it to output on the side facing the injection chanber, it started working... maybe some problems with the pipes ? hope this helps


add website to autor list

Please add my website https://dreamofjapan.de to the list of authors.
thanks in advance

done; see 72d653c


cds terminates unexpectly

Hi,
We had a weird issue where cds would spawn ( normal cds run, watch, .. ) and it would successfully launch showing all standard log messages and then the process would terminate immediately ...
This was happening with a new onboarded dev so we initially assumed a node or cds installation issue. We spend many hours trying to pin point the issue but could not find a resolution.
Then later on it happened to me as well ...
And then I realized the issue ....
The local docker postgresql instance was running fine but it had not yet a created DB.  Then the CDS process just terminates without giving any kind of (error) message.
I am not sure this is controlled by cds-pg but it would be nice to show some kind of message stating that 'Connection successful but no db 'x' was found ...'.
That would have saved us many hours and a headache ...
Thanks,
Steven

Would be great if you can create a PR with a test that shows this issue.

Would be great if you can create a PR with a test that shows this issue.

Can you elaborate a bit on what you would expect as 'test' ?
The test is quite simple :

just start your local docker instance
remove the existing DB that is configured for the project
execute cds run

Now interesting enough I just tried this on your beershop example and there the cds does not quit.  And it simply serves the api and then gives a return error within the api
<error xmlns="http://docs.oasis-open.org/odata/ns/metadata"> 3D000 <message>database "beershop" does not exist</message> </error>
So not sure why it is behaving differently on mine .. hmmmm.


ui: seed verification page issues

The following were observed on the seed verification page while testing on a 5th generation iPad simulator:

Clicking an empty space (outside any of the word buttons) causes issues with the auto scroll

Confirm button doesn't look disabled
(android)

(ios)

On android, selecting one or more wrong words on the seed verification page shows an error just above the button. On ios, I have to click the button on ios to get similar error
(android)

(ios, after clicking the button)

fixed in master


All docker images deployment on Azure

Hi @EdwinVW
Do you have any plan to demonstrate - "Deployment and running all docker images on Azure portal".
Currently it is limited up to any local machine only.
Regards,
Kashif

Hi @findkashif,
Because everything runs in docker containers, you can run Pitstop just as well in Azure. I have tested this once using an AKS (Azure Kubernetes Service) cluster. If you have k8s installed, you can switch between k8s clusters using the kubectl config use-context command. If you switch to your AKS cluster and start the application (using the start-all script), everything is spun up on the AKS cluster in the cloud. See the k8s documentation for more info.


MAKE ICON LIGHTWEIGHT

Hey guys, I don't like the new logo, even a blue color doesn't fix it. On Mac it looks like a old and heavy photoshop icon. The Visual Studio code is a simple and lightweight solution, so please guys, make the accent on that.

VSCode Version: the latest
OS Version: macOS High Sierra

Steps to Reproduce:

Look at the Dock

Reproduces without extensions: Yes

I don' know what you mean with latest version, so maybe you are using 1.17 with the orange icon. For that I refer to the blog post The Icon Journey.

@maxibanki as pointed out please refer to the blog post

as pointed out by @MishUshakov, blue color doesn't exactly solve the issue. The original icon was perfect. without the white outline around the icon, it is very difficult to see the logo against many colors on the desktop (in windows)

.


Implement MQTT export function

Implement a new export function for mqtt brokers.
It requires paho-mqtt python module. pip3 install paho-mqtt
To activate set the export_format key in your config to MQTT,
configure the keys export_mqtt_server, export_mqtt_port,
export_mqtt_user,export_mqtt_password and export_mqtt_basetopic
to reflect your environment.
Stefan, i am not sure this matches your style- and quality standards but i tried to stay as close to your style as i could.
I tested it in my environment and had no problems running it against a mosquitto mqtt server.
Please have a look and let me know if you can merge this.
Cheers, Tom

This is great, thanks!
One thing I would like you to test. I release this script as debian package, so it would be nice to use the debian dependency to the paho mqtt package, so nobody needs to care about installing these dependencies using pip3.
The debian package is called "python3-paho-mqtt", the version in buster is 1.4.0-1.
In the pypi repo there is a newer version of paho-mqtt --> 1.6.1 (https://pypi.org/project/paho-mqtt/).
To test whether your code works with 1.4.0, please do the following:
pip3 uninstall paho-mqtt
sudo pip3 uninstall paho-mqtt  # if you installed it as root
sudo apt update
sudo apt install python3-paho-mqtt

Afterwards please check whether it is installed or simply start your script.
Thanks!

Sure, i can do this:
$# lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

$# dpkg -l python3-paho-mqtt
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name              Version      Architecture Description
+++-=================-============-============-=================================
ii  python3-paho-mqtt 1.4.0-1      all          MQTT client class (Python 3)
root@fhem:~#

Works like a charm with 1.4.0 as well.
Tom

thank your for merging it. i just found a little problem that occurs when run in SOLARVIEW export with a dummy value for the mqtt broker. i will fork your repo again tomorrow, fix it real quick and file a PR.

Managed to fix it right now. See https://github.com/tirolerstefan/kaifa/pull/13


Disable silent stemming

Up until this point, pyrouge has applied Porter stemming (-m) silently, unchangeably, and by default. There is no way to turn this off, even if the user provides an explicit set of ROUGE parameters they want to use. It's also unclear given the existing code structure that the -m parameter is being added, and it seems most users don't realize they are using stemming.
Applying stemming like this is a deviation from the default ROUGE parameters, and tends to fairly significantly inflate ROUGE scores across the board. For this reason, using stemming really needs to be an explicit choice the user makes, rather than being done by default.
My only change here is only deleting the -m when it's added separately in __add_config_option. This makes the set of default parameters clearer, forcing the choice to use stemming to be explicit. This may not be the right solution if some people use and cite pyrouge knowing fully that it stems by default, but my impression is that this is not the case.

return options + ['-m'] + [self._config_file]
That's of course horrible code and I don't know what 8-years-ago me was thinking. I also agree that moving the -m flag into the list of default options is the proper way to fix it. However, I'm weary of changing anything, since it's possible that there are users who rely on the current behaviour.


Potentially wrong gradient matrix

This is a duplicate of https://github.com/PatrickZH/DeepCore/issues/13.
In our AbstractMatrixDownsamplingStrategy implementation, we use the same matrix as defined by DeepCore. As discussed in this issue, the definition of the matrix looks weird, and I am also confused that we don't even need the actual gradient (with torch.no_grad()). We should check out whether this actually makes sense.

on it

After the potential fix PR, https://github.com/eth-easl/modyn/pull/540, I realize there could be a reason for their way of calculating the gradients.
Essentially, we should use the full gradients to select points. But since full gradients are expensive to compute, researchers propose to use the last layer gradients to approximate full gradients.
Since it is possible to use last layer gradients to approximate, it is also possible to use gradients of second-to-last layer, which is the gradients of loss regarding to the embedding. Since the mapping from embedding to output is linear, then the gradients of second-to-last layer is essentially last layer gradients scaled by the mapping layer, which is what they did.

I guess it's not fully clear to me yet:

From my understanding indeed, the papers proposing the initial techniques use the last layer gradients.

I guess it's not fully clear why what they do is equivalent to scaling the last layer gradient. For the bias, they take the last layer gradients, and concatenate that with the bias gradient * embedding? I don't immediately see how tat is equivalent to taking d last layer / d penultimate layer.

What is your proposal here? Should we keep the DeepCore implementation? Should we add a flag that allows to switch between DeepCore and your PR? I am not sure I fully understand the issue. My gut feeling is that we should follow the literature, but I wouldn't have a problem with adding a flag to switch between both implementations.

I guess it's not fully clear to me yet:

From my understanding indeed, the papers proposing the initial techniques use the last layer gradients.
I guess it's not fully clear why what they do is equivalent to scaling the last layer gradient. For the bias, they take the last layer gradients, and concatenate that with the bias gradient * embedding? I don't immediately see how tat is equivalent to taking d last layer / d penultimate layer.

What is your proposal here? Should we keep the DeepCore implementation? Should we add a flag that allows to switch between DeepCore and your PR? I am not sure I fully understand the issue. My gut feeling is that we should follow the literature, but I wouldn't have a problem with adding a flag to switch between both implementations.

You are right we should anyway follow the original literature. I have explicitly read GradMatch paper and they use the last layer gradient. I will check the other policies as well. Also in Craig they did something (similarly wrong): https://github.com/eth-easl/modyn/blob/bf96bfa546c829a4844f25e0143497deb9539e04/modyn/trainer_server/internal/trainer/remote_downsamplers/remote_craig_downsampling.py#L110
Will check the paper and make changes if inconsistent with the paper!

I guess it's not fully clear to me yet:

From my understanding indeed, the papers proposing the initial techniques use the last layer gradients.
I guess it's not fully clear why what they do is equivalent to scaling the last layer gradient. For the bias, they take the last layer gradients, and concatenate that with the bias gradient * embedding? I don't immediately see how tat is equivalent to taking d last layer / d penultimate layer.

What is your proposal here? Should we keep the DeepCore implementation? Should we add a flag that allows to switch between DeepCore and your PR? I am not sure I fully understand the issue. My gut feeling is that we should follow the literature, but I wouldn't have a problem with adding a flag to switch between both implementations.

I introduced a flag to switch between two forms. So bias gradient * embedding is the penultimate layer gradient. It is concatenated with the last layer gradient to somehow "enrich the information". I didn't find the claim in original literature on this form, but in @francescodeaglio 's thesis, there is a specific section justifying it

. So I regard it as reasonable.


WIP: nixos-rebuild test suite

I’d like to change nixos-rebuild to python at one point, because the complexity of the shell script has gone out of bounds, and there is no good way of refactoring bash scripts in a rigorous way.
The first step towards that is building a strong test suite which captures many (if not most) of the current branching.
The strategy here is to instrument nixos-rebuild to print json-strings, which can be read by the test suite and matched on.

Did you see nixos/tests/switch-test.nix?
cc @dasJ

Did you see nixos/tests/switch-test.nix?

No, but we can add it to the vm tests in nixos-rebuild

It specifically doesn’t test nixos-rebuild, but the system activation, which is something the nixos-rebuild test suite shouldn’t test.

As in these tests should stub out the actual activation, only go through evaluation if necessary.

I agree with @Profpatsch. nixos-rebuild should be tested by itself and whether the call to switch-to-configuration works as expected. The contents of switch-to-configuration can then be tested with the existing switchTest.


Export WaitForFilter interface

https://github.com/EventEmitter2/EventEmitter2/issues/289

@RangerMauve could you review this.

Out in 6.4.7


When running python answer_bot.py it returns 'codec can't decode byte 0x9d in position 94' among other things

Here's the screenshot:

Thank You

I am having the same error "UnicodeDecodeError: ........"

me as well

At https://github.com/sushant10/HQ_Bot/blob/master/answer_bot.py#L64,
Replace
sample_questions = json.loads(open("Data/questions.json").read())
with
sample_questions = json.loads(open("Data/questions.json", encoding="utf8").read())

Worked!! Thank You @shaunidiot


Ability to override go cmd

Hey there folks, first of all well done! gigo is exactly the tool I was looking to add to my toolchain. pip's simplicity is just priceless.
Have you guys considered adding support to overriding the go executable? For appengine projects that would be extremely useful since most of the time one needs to run goapp instead of just go so appengine's SDK related stuff are properly handled.
I got a simple solution going on a fork, though that might not be the best solution to the problem. Thoughts?
https://github.com/drborges/gigo/commit/9957495bb91f6515ed6bff188646a86d1ca5a1dc
Cheers.

Awesome idea. We don't really deal with appengine today, but I can see the
value of supporting go overrides.
Let me noodle on this. I love the idea.

On August 17, 2015 11:03:40 AM Diego Borges<EMAIL_ADDRESS>wrote:
Hey there folks, first of all well done! gigo is exactly the tool I was
looking to add to my toolchain. pip's simplicity is just priceless.
Have you guys considered adding support to overriding the go executable?
For appengine projects that would be extremely useful since most of the
time one needs to run goapp instead of just go so appengine's SDK
related stuff are properly handled.
I got a simple solution going on a fork, though that might not be the best
solution to the problem. Thoughts?
https://github.com/drborges/gigo/commit/9957495bb91f6515ed6bff188646a86d1ca5a1dc
Cheers.

Reply to this email directly or view it on GitHub:
https://github.com/LyricalSecurity/gigo/issues/8


build error with target=high_perf

I got error messages as below when running simulation with target=high_perf.
default target has no problem but high_perf target has the problem.
Please help me.
[config & simualtion]
configs/swerv.config -target=high_perf
make -f tools/Makefile verilator target=high_perf
[simulation log]
BUILD_PATH=snapshots/high_perf /home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/configs/swerv.config -target=high_perf
SweRV configuration for target=high_perf
swerv: no_secondary_alu               = 0
swerv: load_to_use_bus_plus1          = 1
swerv: lsu_stbuf_depth                = 10
swerv: dma_buf_depth                  = 5
swerv: verilator                      =
swerv: load_to_use_plus1              = 0
swerv: opensource                     = 0
swerv: lsu_num_nbload                 = 8
swerv: fast_interrupt_redirect        = 1
swerv: fpga_optimize                  = 0
swerv: timer_legal_en                 = 1
swerv: num_threads                    = 2
swerv: dccm_num_banks                 = 8
swerv: dccm_enable                    = 1
swerv: dccm_region                    = 0xf
swerv: dccm_offset                    = 0x40000
swerv: dccm_size                      = 128
swerv: bht_addr_lo                    = 3
swerv: bht_size                       = 4096
swerv: inst_access_addr5              = 0x00000000
swerv: data_access_mask6              = 0xffffffff
swerv: data_access_enable7            = 0x0
swerv: data_access_addr1              = 0x00000000
swerv: inst_access_enable4            = 0x0
swerv: data_access_mask3              = 0xffffffff
swerv: inst_access_mask3              = 0xffffffff
swerv: data_access_mask4              = 0xffffffff
swerv: data_access_mask0              = 0xffffffff
swerv: data_access_enable3            = 0x0
swerv: inst_access_enable3            = 0x0
swerv: inst_access_mask7              = 0xffffffff
swerv: inst_access_addr4              = 0x00000000
swerv: inst_access_addr1              = 0x00000000
swerv: inst_access_mask5              = 0xffffffff
swerv: inst_access_mask2              = 0xffffffff
swerv: data_access_enable2            = 0x0
swerv: data_access_addr7              = 0x00000000
swerv: data_access_addr3              = 0x00000000
swerv: data_access_enable0            = 0x0
swerv: inst_access_enable1            = 0x0
swerv: data_access_enable4            = 0x0
swerv: inst_access_enable6            = 0x0
swerv: data_access_enable1            = 0x0
swerv: inst_access_mask1              = 0xffffffff
swerv: inst_access_addr6              = 0x00000000
swerv: data_access_mask2              = 0xffffffff
swerv: data_access_addr6              = 0x00000000
swerv: inst_access_enable7            = 0x0
swerv: inst_access_addr7              = 0x00000000
swerv: data_access_addr2              = 0x00000000
swerv: data_access_addr0              = 0x00000000
swerv: inst_access_enable0            = 0x0
swerv: data_access_addr5              = 0x00000000
swerv: data_access_mask7              = 0xffffffff
swerv: inst_access_addr3              = 0x00000000
swerv: data_access_mask5              = 0xffffffff
swerv: inst_access_enable5            = 0x0
swerv: data_access_addr4              = 0x00000000
swerv: inst_access_mask0              = 0xffffffff
swerv: data_access_enable6            = 0x0
swerv: data_access_mask1              = 0xffffffff
swerv: inst_access_addr0              = 0x00000000
swerv: inst_access_addr2              = 0x00000000
swerv: data_access_enable5            = 0x0
swerv: inst_access_mask6              = 0xffffffff
swerv: inst_access_mask4              = 0xffffffff
swerv: inst_access_enable2            = 0x0
swerv: ifu_bus_id                     = 1
swerv: dma_bus_prty                   = 2
swerv: sb_bus_tag                     = 1
swerv: dma_bus_id                     = 1
swerv: lsu_bus_tag                    = 4
swerv: bus_prty_default               = 3
swerv: ifu_bus_tag                    = 4
swerv: ifu_bus_prty                   = 2
swerv: lsu_bus_prty                   = 2
swerv: sb_bus_prty                    = 2
swerv: sb_bus_id                      = 1
swerv: lsu_bus_id                     = 1
swerv: dma_bus_tag                    = 1
swerv: btb_index1_lo                  = 3
swerv: btb_addr_lo                    = 3
swerv: btb_size                       = 512
swerv: pic_region                     = 0xf
swerv: pic_meigwctrl_offset           = 0x4000
swerv: pic_meip_count                 = 4
swerv: pic_mpiccfg_offset             = 0x3000
swerv: pic_meipt_count                = 127
swerv: pic_meip_mask                  = 0x0
swerv: pic_meipl_offset               = 0x0000
swerv: pic_meitp_offset               = 0x1800
swerv: pic_meigwclr_offset            = 0x5000
swerv: pic_meidels_count              = 127
swerv: pic_size                       = 32
swerv: pic_meie_count                 = 127
swerv: pic_meidels_offset             = 0x6000
swerv: pic_meipl_count                = 127
swerv: pic_total_int                  = 127
swerv: pic_meitp_mask                 = 0x0
swerv: pic_offset                     = 0xc0000
swerv: pic_mpiccfg_mask               = 0x1
swerv: pic_meitp_count                = 127
swerv: pic_meigwctrl_mask             = 0x3
swerv: pic_2cycle                     = 1
swerv: pic_meipt_offset               = 0x3004
swerv: pic_meipt_mask                 = 0x0
swerv: pic_meidels_mask               = 0x1
swerv: pic_meie_offset                = 0x2000
swerv: pic_meip_offset                = 0x1000
swerv: pic_meigwctrl_count            = 127
swerv: pic_meigwclr_count             = 127
swerv: pic_meigwclr_mask              = 0x0
swerv: pic_meipl_mask                 = 0xf
swerv: pic_mpiccfg_count              = 1
swerv: pic_meie_mask                  = 0x1
swerv: ret_stack_size                 = 4
swerv: icache_banks_way               = 2
swerv: icache_size                    = 32
swerv: icache_bank_width              = 8
swerv: icache_2banks                  = 1
swerv: icache_ln_sz                   = 64
swerv: icache_ecc                     = 1
swerv: icache_waypack                 = 0
swerv: icache_enable                  = 1
swerv: icache_num_ways                = 4
swerv: iccm_enable                    = 1
swerv: iccm_region                    = 0xe
swerv: iccm_offset                    = 0xe000000
swerv: iccm_size                      = 64
swerv: iccm_num_banks                 = 4
swerv: Writing snapshots/high_perf/eh2_pdef.vh
swerv: Writing snapshots/high_perf/eh2_param.vh
swerv: Writing snapshots/high_perf/common_defines.vh
swerv: Writing snapshots/high_perf/defines.h
swerv: Writing snapshots/high_perf/pd_defines.vh
swerv: Writing snapshots/high_perf/whisper.json
swerv: Writing snapshots/high_perf/perl_configs.pl
riscv64-unknown-elf-cpp -Isnapshots/high_perf  /home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/testbench/asm/hello_world.s  > hello_world.cpp.s
riscv64-unknown-elf-as -march=rv32gc hello_world.cpp.s -o hello_world.o
Building hello_world
riscv64-unknown-elf-ld -m elf32lriscv --discard-none -T/home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/testbench/link.ld -o hello_world.exe hello_world.o
riscv64-unknown-elf-objcopy -O verilog --only-section ".data*" --change-section-lma .data*-0x10000 hello_world.exe data.hex
riscv64-unknown-elf-objcopy -O verilog --only-section ".text*"  hello_world.exe program.hex
riscv64-unknown-elf-objdump -S  hello_world.exe > hello_world.dis
riscv64-unknown-elf-nm -f posix -C hello_world.exe > hello_world.tbl
Completed building hello_world
echo '`undef ASSERT_ON' >> snapshots/high_perf/common_defines.vh
verilator '-UASSERT_ON' --cc -CFLAGS "-std=c++11" snapshots/high_perf/common_defines.vh /home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/design/include/eh2_def.sv snapshots/high_perf/eh2_pdef.vh
-Isnapshots/high_perf -I/home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/testbench -f /home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/testbench/flist
-Wno-WIDTH -Wno-UNOPTFLAT /home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/testbench/tb_top.sv /home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/testbench/ahb_sif.sv --top-module tb_top
-exe test_tb_top.cpp --autoflush
cp /home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/testbench/test_tb_top.cpp obj_dir/
make -C obj_dir/ -f Vtb_top.mk OPT_FAST="-O2"
make[1]: directory '/home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/obj_dir' enter
g++  -I.  -MMD -I/usr/local/share/verilator/include -I/usr/local/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=0 -faligned-new -Wno-bool-operation -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -std=c++11  -O2 -c -o test_tb_top.o test_tb_top.cpp
/usr/bin/perl /usr/local/share/verilator/bin/verilator_includer -DVL_INCLUDE_OPT=include Vtb_top.cpp Vtb_top___024unit.cpp Vtb_top_eh2_ifu_aln_ctl__pi15.cpp Vtb_top_eh2_dec_gpr_ctl__pi18.cpp Vtb_top_eh2_ifu_mem_ctl_thr__pi38.cpp Vtb_top_eh2_dec_cam__pi39.cpp Vtb_top_eh2_dec_tlu_ctl__pi40.cpp Vtb_top_eh2_lsu_bus_buffer__pi42.cpp > Vtb_top__ALLfast.cpp
g++  -I.  -MMD -I/usr/local/share/verilator/include -I/usr/local/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=0 -faligned-new -Wno-bool-operation -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -std=c++11  -O2 -c -o Vtb_top__ALLfast.o Vtb_top__ALLfast.cpp
g++: internal compiler error: killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
/usr/local/share/verilator/include/verilated.mk:206: recipe for target 'Vtb_top__ALLfast.o' failed
make[1]: *** [Vtb_top__ALLfast.o] Error 4
make[1]: directory '/home/kdlee/work/riscv/chipsalliance/swerv_eh2/Cores-SweRV-EH2/obj_dir' exit
tools/Makefile:86: recipe for target 'verilator-build' failed
make: *** [verilator-build] Error 2

try to run with Cadence or VCS ...

It's perhaps running out of memory, check your "ulimit" setting, and also watch "top".
Also it looks like you might be using an older Verilator, try a newer one as it will split up the files for GCC and might avoid this GCC problem.

try to run with Cadence or VCS ...

Thanks.
But those commercial simulators are not available now for me.

It's perhaps running out of memory, check your "ulimit" setting, and also watch "top".
Also it looks like you might be using an older Verilator, try a newer one as it will split up the files for GCC and might avoid this GCC problem.

Thanks for the suggestion.
I will try it and let you know the results, soon.

@wsnyder  - what version should we use? is there a switch to force the "split" verilator output? I see the failure with 4.030 ..

You can use "--output-split 20000", but would recommend using 4.102.

Hi @wsnyder , could you, please, check our build command with the latest verilator I'm getting:
verilator '-URV_ASSERT_ON' --cc -CFLAGS "-std=c++11" snapshots/default/common_defines.vh design/include/ehx2_def.sv snapshots/default/ehx2_pdef.vh
-Isnapshots/default -I/testbench -f testbench/flist
-Wno-WIDTH -Wno-UNOPTFLAT testbench/tb_top.sv testbench/ahb_sif.sv --top-module tb_top
-exe test_tb_top.cpp --autoflush
cp testbench/test_tb_top.cpp obj_dir/
make -C obj_dir/ -f Vtb_top.mk OPT_FAST="-O2"
make[1]: Entering directory obj_dir' Vtb_top.mk:57: ....veripool/verilator/4.102/include/verilated.mk: No such file or directory make[1]: *** No rule to make target ..veripool/verilator/4.102/include/verilated.mk'.  Stop.
make[1]: Leaving directory `obj_dir'
make: *** [snapshots/default/defines.h] Error 2

....veripool/
I suspect you have VERILATOR_ROOT misset in your environment.
Note that SweRV + verilator is tested nightly as part of https://github.com/verilator/verilator_ext_tests so is good (except for I broke it for unrelated reasons a few minutes ago ;)

are you running our makefile?

BTW, any idea why EH2 compiles slower than EH1 ?

Yes, assuming you mean tools/Makefile.
I haven't compared compile times between the projects, so I don't know, but as a guess does RV_FPGA_OPTIMIZE default the same?

there was the gap before FPGA_OPTIMIZE I guess this might be due high usage of parameter structure in 2nd gen cores vs defines in EH1. VCS/XRUN compile times almost the same for both designs ...

@wsnyder - with new verilator the compilations is terribly slow ( high performance target) even single threaded version takes more than 30 min (I killed it)
I guess you don't use module - subroutine concept:  everything is flattened...

You can use "--output-split 20000", but would recommend using 4.102.

After adding the option "--output-split 20000" to verilator, simulation works well.
My error is resolved, now.
Thank you very much.


Withdrawal and Deposits issue

A user should not be able to withdraw a negative amount and deposit a negative amount example:
Current balance: 1200
‘Withdraw -1100 makes current balance 2300’
Same with deposit.

@NicoCG this issue was fixed on the last releases. Thank you!
@all-contributors please add @NicoCG for tests


Add nu.txt

Adding Nile University of domain: nu.edu.eg

I'm sorry for the objection but this is not right and we (students) can only obtain the university email after paying tuition and completely being registered at the university and also have already registered for courses.

@MosaabGKA I am sorry, the domain nu.edu.eg is removed from our trusted domains list because the school allows to register an email address there even if you are not a student of the school. If you are the student of this school, please apply with official document (OFFICIAL DOCUMENT tab at https://www.jetbrains.com/shop/eform/students).

Mr. @philipto I'm sorry for the objection but this is not right and we (students) can only obtain the university email after paying tuition and completely being registered at the university and also have already registered for courses.
So, how can we apply for a re-looking to our case? Please.


Add a way to view API diffs between versions

Add a way to see added / removed / changed API signatures for Frameworks, .NET Standard and Windows 10 SDK versions.

⚠ Idea migrated from UserVoice
Created By: Adam Hill
Created On: 2017/12/07 03:32:06 +0000
Votes at Migration: 1
Supporters at Migration: 1

#log-suggestion

🚀 ATTENTION: Internal request logged.

Thank you for the feedback! At this time, we have no plans on implementing this functionality on docs.microsoft.com.


replace builtin json library

We currently use golang's standard json library to marshal and unmarshal entities, but also in datahub's stream parser.
It is known that other libraries perform much better and have better memory efficiency as well. Datahub should use a better library. Some candidates:
https://github.com/minio/simdjson-go
https://github.com/ohler55/ojg
https://github.com/json-iterator/go

the goccy/go-json lib did not work as expected, we will try again with another lib.


update README file

update readme file in accordance with installation instructions

@mistershashanksingh you could have done this in the PR that you created earlier why create another for such small change? Kindly do these changes in a single PR and don't omit anything from the README.MD file. You can add the requirement.txt part in the readme without omitting.

Kindly do it in the PR where you created requirements.txt file


Allow clean_unused_cache to run on purge-old-deployments.

Testing plan:
Tested it by running the scripts individually and purge-old-deployments.
GIFs or screenshots:

After reading this a bit more, the right answer is pretty clearly to just support this option in purge_old_deployments itself, since it's potentially useful there, and in any case, we've pretty clearly relying on the invariant that all these scripts support the same parameters.


Fixes rendering smudges of elements with border for iOS New Architecture

Summary:
0.73.0 introduced smudges appearing while rendering screen with elements, which use borderWidth styles as in https://github.com/facebook/react-native/issues/42604.
https://github.com/facebook/react-native/pull/38234 removed a feature flag disableTransactionCommit, which by default was false, meaning that prior to this RCTMountingManager.mm was using CATransaction for rendering (unless feature flag react_fabric:disable_transaction_commit was applied).
This change brings back CATransaction into  RCTMountingManager.mm
Changelog:
[IOS] [FIXED] - Fix rendering issues of elements with borderWidth (issue 42604)
Test Plan:
Download reproduction repository from linked issue and apply changes to the local version of react-native.
Using Reload button or switching screen should not show any more smudges when rendering elements with borderWidth

This should be resolved by https://github.com/facebook/react-native/commit/5fbdc994f945ee8d834632649e3b82451660e8fe
let me know if you are still seeing an issue.
We don't want to bring bar CATransaction because it has a performance cost associated with it.

Amazing, thank you @sammy-SC .
I can confirm it resolves the unwanted behaviour visible in the reproduction repository of the linked issue.


Move from tfsec to trivy in all other team repositories

User Story
As a Modernisation Platform Engineer
I want to migrate from tfsec to trivy
So that we continue to benefit from up-to-date secure code analysis
trivy is now being favoured over tfsec, they are still using the same base rules so this isn't urgent but we should still change at some point.
https://github.com/aquasecurity/tfsec/blob/master/docs/guides/trivy.md
Value / Purpose
The move on the modernisation platform repo was done on the following issueMove from tfsec to trivy
#3336
see the two PR's in the issue on what needs to be changed
we now need to implement that change on all other repo's that the modernisation platform owns
Useful Contacts
No response
Additional Information
No response
Proposal / Unknowns
Hypothesis If we... [do a thing] Then... [this will happ]
Proposal A proposal that is something testable, don't worry whether it works or not, it's a place for ideas.
Unknowns Potential pitfalls that could cause the story to expand beyond its original scope. Ideally this section will remain blank.
Definition of Done
Example

[ ] Add and change existing code to use trivy
[ ] All repos migrated over to trivy
[ ] Another team member has reviewed additional pr's
[ ] Tests are green

Before we do this we need to check and correct the earlier trivy implementation - ie we need to enable skipping features

https://github.com/ministryofjustice/modernisation-platform-terraform-ssm-patching/pull/352/files
One down.

Merged - https://github.com/ministryofjustice/modernisation-platform-terraform-ssm-patching/pull/352/files
Waiting -
https://github.com/ministryofjustice/modernisation-platform-terraform-ec2-instance/pull/347
https://github.com/ministryofjustice/modernisation-platform-terraform-module-template/pull/344
https://github.com/ministryofjustice/modernisation-platform-terraform-loadbalancer/pull/401/
https://github.com/ministryofjustice/modernisation-platform-terraform-member-vpc/pull/369
Created branches on all modules.


fix: ip-info test

Changes

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#631  👈
#620
#616
#615
#596
#583
#582
#581
#580
#578
#573
#572
#571
#570
#566
#565
#579
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @MasterPtato and the rest of your teammates on  Graphite


language_tabs not working - part 3

I'm rather disappointed that even after #156 and #450, this doesn't work properly.
My setup has two languages, c++ and lua. If I choose C++, only C++ code snippets show. If I choose Lua, both are shown. Looking at inspector, I've noticed that the C++ blocks don't have display: block/none; on them.
This is on master.
Edit: Doesn't work on dev either.

@tn0502 can you post a gist of your index.md?

https://gist.github.com/tn0502/f666e5578bce01aa9b4dc73c5df7930f
Thanks for taking a look.

@tn0502 ah, yea, + isn't a valid character in HTML classes, and I think that's causing the problem. If you switch all your c++ references to read cpp instead, that should fix it!


fix(utils): adjust wording

This mini PR adjusts wording in the utils templates according to https://github.com/music-encoding/music-encoding/issues/776#issuecomment-839864587

Thank you!


Fix typo

Description

Just a small typo fix.
Related Issue(s)

Checklist

[ ] I have read the contribution guidelines
[ ] I have considered the performance impact of these changes
[ ] Suitable unit/system level tests have been added and they pass
[ ] Documentation has been updated

Thanks @bonanitech !


formatting conflict with null-ls

Hi, I want to use null-ls to format the code on save. but it ask me to choose which formatter I want to use. I already disabled the  following in the lspconfig
   client.resolved_capabilities.document_formatting = false
   client.resolved_capabilities.document_range_formatting = false

I have the same setup, in order to apply dartls config with flutter-tools, you need to pass your lsp config into flutter-tools setup method instead of doing them in lspconfig:
-- lspconfig.lua
      if server.name == "dartls" then
         opts.on_attach = disable_lsp_formatting_on_attach
         require("custom.plugins.flutter-tools").setup(disable_lsp_formatting_on_attach, opts.capabilities)
      end

-- custom/plugins/flutter-tools.lua
    function setup(attach, capabilities)
        require("flutter-tools").setup {
            lsp = {
                on_attach = attach,
                capabilities = capabilities,
           },
       }
    end

Hi @br1anchen
I did what you sent and it works like a charm.
Thank you,

Hi, I want to use null-ls to format the code on save. but it ask me to choose which formatter I want to use. I already disabled the following in the lspconfig
   client.resolved_capabilities.document_formatting = false
   client.resolved_capabilities.document_range_formatting = false

Excuse me. Could you tell me how to use null-ls format flutter code?Thanks!

I have the same setup, in order to apply dartls config with flutter-tools, you need to pass your lsp config into flutter-tools setup method instead of doing them in lspconfig:
-- lspconfig.lua
      if server.name == "dartls" then
         opts.on_attach = disable_lsp_formatting_on_attach
         require("custom.plugins.flutter-tools").setup(disable_lsp_formatting_on_attach, opts.capabilities)
      end

-- custom/plugins/flutter-tools.lua
    function setup(attach, capabilities)
        require("flutter-tools").setup {
            lsp = {
                on_attach = attach,
                capabilities = capabilities,
           },
       }
    end

<EMAIL_ADDRESS>can't find the server .Could you show me the code?


should catch urllib2.URLError exceptions instead of crashing

https://bugzilla.redhat.com/917380#c6
$ rpm -q youtube-dl
youtube-dl-2013.08.02-1.fc19.noarch
$ youtube-dl http://abc
Traceback (most recent call last):
[...]
urllib2.URLError: <urlopen error [Errno -2] Name or service not known>
The Python traceback triggers automatic bug reporting tools, such as ABRT, and users will report such crashes.

Thank you for the report, and sorry for the delay. For quite a while now, youtube-dl will only print a summary of the error by default:
$ youtube-dl http://abc
[generic] abc: Requesting header
WARNING: Could not send HEAD request to http://abc: <urlopen error [Errno -2] Name or service not known>
[generic] abc: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno -2] Name or service not known> (caused by URLError(gaierror(-2, 'Name or service not known'),))

This should not trigger abrt. If you pass in -v, you'll get a complete stack trace, but that's the whole point of -v.
I do think that the current situation works well, but then again, I don't use abrt, and our users don't report abrt bug reports here. Therefore, I am closing this entry, but feel free to comment and we will reopen it if necessary.
If you get reports from -v as well, is there a way we can tell abrt that this is not an unforeseen error in youtube-dl? Would it be sufficient to post additional text after the stacktrace?

The tracebacks at http://bugz.fedoraproject.org/youtube-dl all seem to without -v.
Error handling and sanity checking of input/runtime data is a good thing.

https://bugzilla.redhat.com/show_bug.cgi?id=1045282 should be reproducible with youtube-dl http://localhost:1/. Nowaydays that shows up as
[generic] localhost:1: Requesting header
WARNING: Could not send HEAD request to http://localhost:1/: <urlopen error [Errno 111] Connection refused>
[generic] localhost:1: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno 111] Connection refused> (caused by URLError(error(111, 'Connection refused'),))

https://bugzilla.redhat.com/show_bug.cgi?id=1090230 should be reproducible by running
import http.server
class MyHandler(http.server.BaseHTTPRequestHandler):
    handle_one_request = lambda *args: 0
http.server.HTTPServer(('', 1201), MyHandler).serve_forever()

in the background and then youtube-dl http://localhost:1201/. Nowadays that shows up as
[generic] localhost:1201: Requesting header
WARNING: Could not send HEAD request to http://localhost:1201/: ''
[generic] localhost:1201: Downloading webpage
ERROR: Unable to download webpage: '' (caused by BadStatusLine("''",)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

https://bugzilla.redhat.com/show_bug.cgi?id=1098690 , https://bugzilla.redhat.com/show_bug.cgi?id=1122959 , https://bugzilla.redhat.com/show_bug.cgi?id=1176320 seem to be a condition where sys.stderr.write(b'something') fails. This seems to be worthy of a bug trace by any means; without the trace I'd be even more surprised about what is happening here.
https://bugzilla.redhat.com/show_bug.cgi?id=1115734 is not reproducable in a current version of youtube-dl; it (i.e. https://secure.brightcove.com/services/viewer/federated_f9?) produces a rather nice output:
ERROR: Cannot find playerKey= variable. Did you forget quotes in a shell invocation?

https://bugzilla.redhat.com/show_bug.cgi?id=1159600 seems to have been fixed since reporting; the download works fine for me now. Just to be certain, I've changed the offending code so that we get a more usable trace should a similar problem occur again.
https://bugzilla.redhat.com/show_bug.cgi?id=1181110 has been fixed in youtube-dl 2015.01.01. In any case, throwing an error seems fine here.
https://bugzilla.redhat.com/show_bug.cgi?id=1135594 seems to be a true network error. It have not found a way for it to occur under Python 2.7.9 though, and the IncompleteRead error is already handled quite nicely by us. In any case, I've added explicit handling in youtube-dl 2015.02.10 (dd1ab5e70c0024b94a37358d90791fe8949727b2).
https://bugzilla.redhat.com/show_bug.cgi?id=1002310 seems to have been resolved (the report author commented so) and just needs to be closed. In any case, this again seems to have been a real bug in youtube-dl for which a bug report was warranted.
https://bugzilla.redhat.com/show_bug.cgi?id=1005359 is in a very similar situation: the comments make it clear that it should have been closed, and the actual error was in a custom build of youtube-dl anyways. And again, a bug report is definitly warranted here.
https://bugzilla.redhat.com/show_bug.cgi?id=1093517 is someone who calls youtube-dl with a BOM in the URL, an unforeseen situation which was handled incorrectly beforehand. Again, I believe this warranted a stack trace. In any case, version 2<IP_ADDRESS> adds some rather nice handling of this case:
$ youtube-dl test:unicodebom
[TestURL] Test URL: http://www.youtube.com/watch?v=BaW_jenozKc
WARNING: [UnicodeBOM] Your URL starts with a Byte Order Mark (BOM). Removing the BOM and looking for "http://www.youtube.com/watch?v=BaW_jenozKc" ...

Summarizingly, I do believe that not only that most of the reports in the current list can be closed, but also that they fall into one of two categories, namely actual bugs that have been fixed in youtube-dl, or rather surprising environments, which are unquestionably worthy of investigation and a bug report. Can you name a single invocation or report that shows the problems you want to fix with this bug issue, instead of "all of them".

I disagree that we should do more to handle errors. In many cases, an extractor fetches JSON and simply goes over keys. It seems far more Pythonic to simply let the KeyError bubble up and wrap it (so it does not show up in abrt without -v).

Thanks for the many comments.

Can you name a single invocation or report that shows the problems you
want to fix with this bug issue, instead of "all of them"?

I pointed at the list of bugs users have decided to report (mostly because of ABRT intercepting the crash) to give examples of what issues they run into. The users seems to be overwhelmed with the traceback output and believe the program to be faulty in some way. Else they would not report/forward such tracebacks, but read them and draw their own conclusions. It is also difficult to decide for users whether to report to Fedora (bugs can be specific to it) or whether to report directly to youtube-dl developers. ABRT makes it too easy to report something to Fedora with a few mouse-clicks and be done. Where a package maintainer doesn't examine those incoming reports, they will be useless, because not even a valid traceback would be forwarded upstream.
Again, thanks for the comments. I'll notify all youtube-dl maintainers here at Fedora about them.

Thank you all very much for helping to triage the bug reports from Red Hat's bugzilla.

$ youtube-dl test:unicodebom
[TestURL] Test URL: http://www.youtube.com/watch?v=BaW_jenozKc
WARNING: [UnicodeBOM] Your URL starts with a Byte Order Mark (BOM). Removing the BOM and looking for "http://www.youtube.com/watch?v=BaW_jenozKc" ...

How to solve this problem with all Youtube links. which do i use?
youtube-dl 2021.12.17
Microsoft Windows 10.0.19045.2788


Switch all existing features to use checkIfFeatureEnabled

When I implemented the options system I only switched the agc feature to use checkIfFeatureEnabled rather than chrome.storage.sync.get directly. This was so tha people could see how to use it.
Since then several other features have switched over to use it.
So I can now go through all the remaining features and switch them over.

Please leave BioCheck. I have done this and would like to avoid commit conflicts


Please add a tag to current master and update METADATA

As seen at http://pkg.julialang.org/detail/SpecialMatrices.html page, Pkg.test("SpecialMatrices") fails.
Master is OK for both julia-0.4 and julia-0.5
Pkg.checkout("SpecialMatrices")
Pkg.test("SpecialMatrices") # OK

Pkg.free("SpecialMatrices")
Pkg.test("SpecialMatrices") # ERROR

Please add a tag to the current master commit,  https://github.com/JuliaLang/METADATA.jl/tree/metadata-v2/SpecialMatrices is 3 years old !

Second this. There should really be a new tag.


'plugin_name' is undefined error

When trying to create a new plugin, I get this error immediately after the cookiecutter setup:
Unable to create file 'metasyncontrib/{{ cookiecutter.__package_name }}/distribution.py'
Error message: 'plugin_name' is undefined
Context: {
    "_cookiecutter": {
        "email"<EMAIL_ADDRESS>        "plugin_name": "llm",
        "privacy": "none",
        "user_name": "Raoul Schram"
    },
    "cookiecutter": {
        "__decorate": "@metadist_testname",
        "__metadist_import": "from metasyncontrib.testname.base import metadist_testname",
        "__package_name": "testname",
        "__plugin_camel": "TestName",
        "__plugin_slug": "metasyn-testname",
        "_checkout": null,
        "_output_dir": "/Users/samuel/Desktop/Creating/Python Projects/metasyn/plugin_test/dist_plugin",
        "_repo_dir": "/Users/samuel/.cookiecutters/metasyn-distribution-template",
        "_template": "https://github.com/sodascience/metasyn-distribution-template",
        "email"<EMAIL_ADDRESS>        "plugin_name": "TestName",
        "privacy": "none",
        "user_name": "TestUserName"
    }
}

Sorry, I wasn't watching this repo. I have already fixed it while trying it out myself! It now also correctly runs the CI tests, so this shouldn't happen in the future.

Anyway, thanks for reporting!


feat(style): add inline-style example.

Deploy preview ready!
Built with commit a438777c961f84ad58aea7fe59df66a1183b5bfe
https://deploy-preview-9--react-progressive-bg-image.netlify.com


chore: use optional chaining

🔗 Linked issue
Review comments from #104
❓ Type of change

[ ] 📖 Documentation (updates to the documentation, readme, or JSdoc annotations)
[ ] 🐞 Bug fix (a non-breaking change that fixes an issue)
[X] 👌 Enhancement (improving an existing functionality like performance)
[ ] ✨ New feature (a non-breaking change that adds functionality)
[ ] 🧹 Chore (updates to the build process or auxiliary tools and libraries)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description
Refer to @pi0 in this comment. I think the optional chaining it's still relevant.
📝 Checklist

[X] I have linked an issue or discussion.
[ ] I have updated the documentation accordingly.

Thanks for splitting this part of the change. Can you please remind me why this change was required? Purely for code readability?

@pi0 Yes, exactly.

Ah, thanks for confirming. If this is only the case, I think it might be better to keep it as is.
I also really want to get rid of legacy syntaxes and prefer new ones but optional chaining can be particularly tricky to use especially in defu which is used in many many different projects. Many of them use Nuxt 2 with webpack 4 which does not understand this syntax (natively) as it uses older acorn parser even tough all recent browsers support it.


Python client: async logging needs reimplementation

The FasttrackML python client enhances attributes for Metric and Param, but those will not be picked up by the MLflow async handler, which is employed by our library for those calls.
Probably simplest to reimplement async in the FasttrackML module.

Tested logging asynchronously and everything seems to work fine, params value are passed correctly and metric context too


build: use c8 for coverage

:tada: This PR is included in version 0.2.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[Question] Is there any guidance for using Cypress Real Events inside a Docker container?

Description
I'm using CRE to more accurately create component and e2e tests that use keypress events instead of the mouse.
I'm moving over to running these tests in a Docker container and getting test failures where I use CRE methods like realPress() and realClick().
The hypothesis
I think the issue is not with CRE, but more with exposing the Chrome Devtools Protocol to Cypress in a container. I've done a fair bit of Googling, but haven't come across anyone running into this issue or good suggestions for fixes.
Dockerfile
The Dockerfile is pretty basic, just building a Node image with latest Chrome stable on Debian 12. Any suggestions or ideas are much appreciated.

I don’t have a lot of time to test but we do run all of our CI in docker containers anyway. You can check out our CI which is working in Docker perfectly.
Here are docker containers handcrafted by the cypress team that include all the necessary deps and browsers: https://github.com/cypress-io/cypress-docker-images

Thank you, I'll give these a look.

Closing for now feel free to open new issue


chore: refactor fns to utilize latest go SDK offerings

as we're trying to improve the go sdk we're running into a blocker here with sample apps. These apps are using old patterns and b/c of this, are tangled up in parts we don't want touched directly. This updates it to take advantage of what the SDK offers, removing the need for a lot of boilerplate. Additionally, it frees up the SDK to make the important but technically breaking changes.  None of the other fns hit this, as they're utilizing the SDK helpers instead of creating boilerplate to do it. After this update, we're in the free to update the SDK with this breaking change, that won't actually break anyone. The power of solid helper fns heh

@mraible , were you able to get this tested?

@jsteenb2 I have not tried yet, but I can if you'd like.


Recommended way to set width?

Without attempting to set size, the modal is kind of too large for my purposes.  (I'm using the Tingle to display alert boxes like "Please limit your selection to 3 items max" sort of thing.
It would be good if the documentation suggested the best way to set the width (and height) without breaking your css.
(Thanks for this btw, just what I needed to as not to break some of the other frameworks which implement tediously complicated mechanisms to display a simple modal with an "OK" button.)

Seconded. I've been trying to figure out how to do this for a while now without breaking other bits of the site.

I changed the width by css, seems ok to me.
.tingle-modal-box{     max-width:700px; }

Indeed, tingle provides a minimalist CSS, feel free to use the best method that fits your needs!
For example, you can add a custom class:
var modal = new tingle.modal({
    cssClass: ['tingle--modal']
});

and add a custom CSS to match .tingle--modal with the desire width.


Javascript only

This so-called 'gem' only works as advertised with Javascript.
Don't waste your time...
-- cc

Got any more details?

See:  https://github.com/jwadhams/json-logic/issues/24
-- cc

You raise a good point. Would you like to make a PR to adjust the read me to say that we currently do not support the full json logic spec at this time? I don't personally have time right now to fully implement that missing functionality, but you're right, we shouldn't be claiming we have that functionality for adding methods when we do not. I inherited this, so I was not aware we did not support more than single-argument methods, and it's a valid point.

Sure...  It's not going to be much though...  I've really moved on from this and the project I was looking to use this on is done.
I don't do much at Github so I'll have to refresh my memory about the workflow here...and you'll have to excuse me if I violate any protocols you've established about it.
-- cc

I submitted a pull-request for the doc changes you wanted on 5/4/2021.  It remains un-merged.  If you're not going to merge it, please tell me.  I'd like to get rid of my fork.
-- cc

Hey I'll check it out!
On Fri, Jun 25, 2021 at 11:43 AM Capone Cicero @.***>
wrote:

I submitted a pull-request for the doc changes you wanted on 5/4/2021. It
remains un-merged. If you're not going to merge it, please tell me. I'd
like to get rid of my fork.
-- cc
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/bhgames/json-logic-ruby/issues/23#issuecomment-868762712,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACNA233CXG62Z4Y7GXXYIDTUTE5ZANCNFSM4PZLILPA
.

--
Jordan Prince | Software Engineering
O:<PHONE_NUMBER> | C:<PHONE_NUMBER>
This e-mail is only intended for the person(s) to whom it is addressed and
may contain confidential information.  Any unauthorized review, use,
disclosure, or distribution is prohibited. If you received this e-mail in
error, please notify the sender by reply e-mail and then delete this
message and any attachments from your system. Thank you for your
cooperation.


Improve EDS communication in developer onboarding sessions

From May 2022 gathering
Look at slide that is in both SW developer onboarding and citizen developer onboarding
Slides:
What do we want to communicate here?

New task too look at this together with HCD.
Currently EDs is mentioned under Equinor IT platforms - maybe there should be a separate section for design.

From citizen developer onboarding:
https://statoilsrm.sharepoint.com/sites/CitizenDevelopmentKnowledge/SitePages/Design-and-citizen-deve.aspx
https://statoilsrm.sharepoint.com/sites/CitizenDevelopment/Shared Documents/Forms/AllItems.aspx?id=%2Fsites%2FCitizenDevelopment%2FShared Documents%2FOnboarding videos%2FEDS.mp4&viewid=67e56cef-3745-4dc8-a355-a98036155708&parent=%2Fsites%2FCitizenDevelopment%2FShared Documents%2FOnboarding videos

Will discuss with Mikael Feb 3, haven gotten in touch with Thorvald.


server test failed

TypeError: environment.teardown is not a function
  at ../../node_modules/jest-runner/build/runTest.js:440:25

did you run this in the finished version of the project? Testing doesn't work in that


AI does not support receiving input and output tokens used

Is your feature request related to a problem? Please describe.
I would like to track and limit the token usage per person. Therefor  I need to receive the used tokens per input and output
Describe the solution you'd like
I want meta data to be returned:
"usage": {
"inputTokens": number,
"outputTokens": number,
"totalTokens": number
}
in the docs: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html#API_runtime_ConverseStream_ResponseElements

It is for conversations!
I would like to set a max tokens on user level in DynomaDB and for each request if there are sufficient tokens, it not then deny the request, otherwise continue and receive the usedTokens and update the Dynamodb


Build fails because header component SCSS references a non-existent variable

Module build failed:
      margin-top: $space-l;
                 ^
      Undefined variable: "$space-l".
      in /Users/tschluchter/Developer/WholesaleDesignSystem/components/src/ws-header/ws-header.scss (line 230, column 19)
 @ ./demo/index.scss 4:14-127
 @ ./demo/demo_app.js
 @ multi (webpack)-dev-server/client?http://localhost:8080 ./demo/demo_app.js ./webpack.config.js
webpack: Failed to compile.

Didn't pull SCSS from the right repo locally. User error :)


Issue with split tall images.

Steps to reproduce

Enable split tall images
Download the chapter

Expected behavior
Images are downloaded and split
Actual behavior
Error: Couldn't Split Downloaded image
Crash logs
No response
Yōkai version
1.7.14
Android version
Android 8.1
Device
Xiaomi Mi Pad
Other details
It works without any issues on TachiyomiSY and Mihon, however, in Yokai and TachiyomiJ2K there's an error.
Is the method to split tall images different in Yokai and J2K compared to SY and Mihon?
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
[X] I have written a short but informative title.
[X] If this is an issue with an extension, or a request for an extension, I should be contacting the extensions repository's maintainer/support for help.
[X] I am reporting an issue exclusive to this fork. I have also checked that is not an issue on the main version of Mihon
[X] I have tried the troubleshooting guide.
[X] I have updated the app to version 1.7.14.
[X] I have updated all installed extensions.
[ ] I have admitted that I am a clown by having checked this box, as I have not read these acknowledgements.
[X] I have filled out all of the requested information in this form.

Seems to be false positive, it's checking if it should split the image, but for some reason Jay send notification that the image can't be split even tho in reality it's "This image is fine, I should not split this".


Keep cursor focus in editor after running script

Description of your problem
What steps will reproduce the problem?

Just write some Python codes and save them as a script
Hit F5 to run it
Observe where the focus is after successfully running the script

What is the expected output? What do you see instead?
I would like to see the cursor focus remaining in the editor, not jumping to the iPython console.
Please provide any additional information below
None
Versions and main components

Spyder Version: 3.2.3
Python Version: 2.7
Qt Version: 5.6.2
PyQt Version: 5.6.0
Operating system: Windows 10

Dependencies
Not really. This is just a generic problem.
Please go to the menu entry Help > Optional Dependencies (or
Help > Dependencies), press the button Copy to clipboard
and paste the contents below:

It's not possible to get that behavior right now, sorry.

You're welcome to send us a pull request to provide this feature.


Update settings.yml

Build failed because of:
big picture configration in settings.yml has an issue:
      there is a big_picture.main.elements entry with category: "Platform" but we do not have that category "Platform" in the landscape.yml!
big picture configration in settings.yml has an issue:
      there is a big_picture.main.elements entry with category: "Protocol" but we do not have that category "Protocol" in the landscape.yml!


Select variables to be used as labels in _Nd_sampler

Fixes #46

I noticed that some changes from #45 leaked into #47, I'll fix this evening


[SPARK-27506][SQL] Allow deserialization of Avro data using compatible schemas

What changes were proposed in this pull request?
The current implementation of from_avro and AvroDataToCatalyst doesn't allow doing schema evolution since it requires the deserialization of an Avro record with the exact same schema with which it was serialized.
The proposed change is to add a new optional field to allow passing the schema used to serialize the records. This allows using a different compatible schema for reading by passing both schemas to GenericDatumReader. If no writer's schema is provided, nothing changes from before.
How was this patch tested?
A new unit test was added.

Can one of the admins verify this patch?

cc @gengliangwang

Can one of the admins verify this patch?

Can one of the admins verify this patch?

@giamo, what use case does this PR target? Most of operations can be already done by Spark APIs.

@HyukjinKwon Imagine that every hour we run a Spark job that loads some data containing Avro records from an external source into a Spark dataframe, uses the from_avro function to deserialize the records and does some processing with them. We use a schema registry to store the schema of the records and sometimes we need to publish a new version by adding or removing fields in a backward compatible way. Consider that it's important for our business logic that a whole batch of records is deserialized using the latest available schema version.
Without doing schema evolution, the job works fine. Now imagine that at some point the business requires we evolve the schema by adding a new field, so the data loaded in the next run of the Spark job now contains some events serialized with the old version of the schema and some others serialized with the new one. Our job now breaks because it tries to read the older data with the new schema and this is not currently supported by from_avro. This is a real use case that we are facing, by the way :)
A workaround using the Spark API would require detecting the differences between the schema versions and patching the older records to make them have the same structure as newer records. This would be non-trivial to implement, less efficient and actually unnecessary since Avro already offers the functionality by passing both the reader and an optional writer schema to a GenericDatumReader (which is used by Spark underneath). This PR simply allows to do the same in the Spark interface.

@mgaido91 @HyukjinKwon changes pushed

@mgaido91 @HyukjinKwon do you think we can move this forward?

I think this change makes sense, @HyukjinKwon WDYT?

Overall LGTM.
CC @cloud-fan

Thanks! I think I need an "ok to test" from a committer to start the Jenkins build, right?

ok to test

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13273/
Test PASSed.

Test build #108171 has started for PR 24405 at commit a1bef67.

Test build #108171 has finished for PR 24405 at commit a1bef67.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108171/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13284/
Test PASSed.

Test build #108183 has started for PR 24405 at commit 5e2f37d.

Test build #108183 has finished for PR 24405 at commit 5e2f37d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108183/
Test PASSed.

@giamo, can you clarify the usecase with codes in PR description? We talk by codes.
The API looks a bit awkward because it takes two Avro schemas.

The API looks a bit awkward because it takes two Avro schemas.

Agree. It looks to me that we just need to make AvroDeserializer more tolerant, e.g. allow reading avro int as spark long type.

cc @dbtsai @gengliangwang do we have this feature in the avro data source? e.g. can we read avro files with a different but compatible schema?

@cloud-fan yes, that's what the GenericDatumReader constructor that I linked above allows you to do. There are plenty of examples online.

I mean the Spark avro data source.

cc @dbtsai @gengliangwang do we have this feature in the avro data source? e.g. can we read avro files with a different but compatible schema?

Currently we don't have such feature for Avro data source.
The file reading always splits the file into partitions and read with one unify partition. To support such feature we will have to know the writer schema of each file.

Can we address https://github.com/apache/spark/pull/24405#issuecomment-520899129? I feel like we're just adding options just as per requested without considering the API shape or clarification.

@giamo, can you clarify the usecase with codes in PR description? We talk by codes.
The API looks a bit awkward because it takes two Avro schemas.

@HyukjinKwon with "codes" you mean examples on how to use the function in the various cases?

Yes, please. What I am worried is that,

I would like to be very sure if this is something worth and regular users want.

From what I read the comments and codes here, basically it allows to take a wider schema whereas currently it's strictly restricted to the avro schema converted from Spark's, am I correct?
Can you show it with actual code examples (better if that's runnable)? I lately updated PR template (see https://github.com/apache/spark/commit/0ea8db9fd3d882140d8fa305dd69fc94db62cf8f) . It would be great if this is followed.

Can we merge those additional parameters into options: java.util.Map[String, String]? I don't think it's a good idea to keep adding parameters to the function while we have options. If Avro Datasource does not have equivalent options, we can add the options for specific for both functions.

Out of curiosity, how do we handle such case in Avro DataSource? Are we able to merge options if there are?

@HyukjinKwon I've updated the PR description, maybe the use case will be more clear now.

I can't speak for other users, but in our case we need this feature in a real project and for the moment we had to do a custom workaround in our code to make it work

It's possible to pass the writer schema inside the options map, if you guys think it would be more clear. Should I implement the change?

In the case of Avro DataSource the situation is a bit more tricky, because you can read multiple Avro files and each one was potentially written with a different schema and you would have to specify the writer schema for each one of them

It's possible to pass the writer schema inside the options map, if you guys think it would be more clear. Should I implement the change?

WDYT @mgaido91, @gengliangwang, and @cloud-fan? I think we should stop adding parameters and merge it into options.

@HyukjinKwon +1.  It is reasonable to avoid adding parameters to the API in such a case.
@giamo Could you also update the doc ./docs/sql-data-sources-avro.md

+1 for having it in the options too, thanks @HyukjinKwon !

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/15568/
Test PASSed.

Test build #110593 has started for PR 24405 at commit 3128047.

Test build #110593 has finished for PR 24405 at commit 3128047.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110593/
Test PASSed.

Can one of the admins verify this patch?

A bit confused by the example provided:
scala> val decodedDf1 = df1.select(from_avro(data = 'eventBytes, jsonFormatSchema = schema2, options = Map("writerSchema" -> schema1).asJava) as "decoded")

So them schema1 is provided as writerSchema, we use schema1 to read avro bytes back? Isn't schema1 only 2 fields? Are you meant schema2 here?

ok to test

@giamo mind updating PR? looks like we're getting closer to merge.

Test build #111933 has started for PR 24405 at commit 3128047.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16949/
Test PASSed.

Test build #111933 has finished for PR 24405 at commit 3128047.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111933/
Test PASSed.

A bit confused by the example provided:
scala> val decodedDf1 = df1.select(from_avro(data = 'eventBytes, jsonFormatSchema = schema2, options = Map("writerSchema" -> schema1).asJava) as "decoded")

So them schema1 is provided as writerSchema, we use schema1 to read avro bytes back? Isn't schema1 only 2 fields? Are you meant schema2 here?

The example is correct, jsonFormatSchema is the one used for deserialization, just as before.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17134/
Test PASSed.

Test build #112144 has started for PR 24405 at commit 54cb6f1.

Test build #112144 has finished for PR 24405 at commit 54cb6f1.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112144/
Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19634/
Test PASSed.

Test build #114811 has started for PR 24405 at commit 54cb6f1.

Test build #114811 has finished for PR 24405 at commit 54cb6f1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114811/
Test PASSed.

@giamo, sorry it took long. very close to go. Can you address the comments?

 There are only few rather minor comments left to address ...

I'm happy to cherry-pick @giamo's work and fix the last few comments

Please go ahead. Let me credit to both as co-authors.

I've opened up a follow up under https://github.com/apache/spark/pull/26780/

@gengliangwang can you close this one as well?

Close this one since #26780 is closed.
@giamo Thanks for work!


Light mode support

Dark mode is applied for layer tree and editor panel by default, I wonder if it is possible to support light mode?

@exotfboy currently the only way to change the appearance of the editor would be to adapt the CSS (there is no option to switch between dark and light mode as there is no light mode currently).
But thanks for your suggestion, I'll add the discussion label to this issue. Would be interesting to find out if there's a general demand for a light mode.

Just stopping by to ➕ this. When working on a computer that doesn't have the backlight on full the inputs are very difficult to make out. Would love an option when launching Maputnik to set it to light and have it launch with the colors inverted. White bg, slightly grey inputs, black text, ect.
If it's not too much of a lift to set that up on the tool side, happy to PR an alternative CSS for that myself.


Populating missing languages

Q
A

Is bugfix?
yes

New feature?
no

Breaks BC?
no

Tests pass?
yes

Fixed issues
#11502

MessageController now properly populates missing languages in case of extraction with "db" format and no new messages.
Previously with no new messages to extract missing languages were ignored.
Unit tests for extracting messages with "db" format added.
Test checking case #11502 added in BaseMessageControllerTest.
testMarkObsoleteMessages() had to be overridden because in case of DB messages source message is wrapped in @@ and not translation.

Additionally I've made few slight modifications in MessageController:

Excessive else has been removed in explicit cases.
Failure to create language directory throws exception now in case of "php" and "po" formats.
Db component is verified with Instance::ensure() to allow tests configuration.
Single-character variable $m renamed to keep Code Climate happier.
Console output "Nothing new...skipped" replaced with "Nothing to save" to make it compatible with other message saving methods and BaseMessageControllerTest.

DbMessageControllerTest is based on SilverFire's code for DbMessageSourceTest.

Should I modify CHANGELOG to 2.0.12 as well?

Yes, I think so. No more issues are going to me solved in 2.0.11 except the ones assigned to milestone else we won't release it ever :)

Merged https://github.com/yiisoft/yii2/commit/1749c46c5d7a14a2c45fbad5f723edbbca37db34 Thank you!


Revert "[core-http] Move challenge based auth to core-http"

Reverts Azure/azure-sdk-for-js#11226
At @schaabs's request

Thanks @jonathandturner. Too add context, we need to hold on moving this into core-http until we've done some more investigation of new authentication challenge scenarios which were recently introduced.


Mouse button events are inconsistent or don't work on 1.6 Alpha

Describe the bug
Using the 4 different methods of handling pointer events for mouse buttons results in either broken behavior, or no behavior at all.
Tested with JVM, and wasmJs/js on Chrome (Ubuntu 22.04).
The code for testing all of these is:
var t by remember { mutableStateOf("Hello, world!") }

Scaffold { contentPadding ->
  Box(
    modifier = Modifier
         .fillMaxSize()
         .padding(contentPadding),
    contentAlignment = Alignment.Center,
  ) {
    Text(text = t)
  }
}

The following modifiers were tested by adding them after the padding modifier.
onPointerEvent(
  eventType = PointerEventType.Press,
  pass = PointerEventPass.Initial,
  onEvent = { event ->
    t = event.button?.index?.toString() ?: "null"
  }
)

JVM works, but only with PointerEventType.Press (and uses different values on Linux vs Windows/Mac)
wasmJs/js prints "null" for everything
pointerInput(Unit) {
  awaitEachGesture {
    t = currentEvent.button?.index?.toString() ?: "null"
  }
}

All targets freeze when running this and need to be killed by the process manager
pointerInput(Unit) {
  awaitPointerEventScope {
    while(true) {
      val event = awaitPointerEvent()
        t = event.button?.index?.toString() ?: "null"
      }
  }
}

JVM works
wasmJs/js prints "null" for everything
onClick(matcher = PointerMatcher.mouse(PointerButton(0)), onClick = { t = "Mouse Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(1)), onClick = { t = "Mouse Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(2)), onClick = { t = "Mouse Middle" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(3)), onClick = { t = "Horizontal Scroll Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(4)), onClick = { t = "Horizontal Scroll Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(5)), onClick = { t = "Mouse Back" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(6)), onClick = { t = "Mouse Front" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(7)), onClick = { t = "Mouse Thumb" })

JVM works (although mouse back and front and thumb aren't fired unless the left, right, or middle mouse button are being held down)
wasmJs/js t never changes

Thank you for reporting this!

@eymar I saw in https://github.com/JetBrains/compose-multiplatform-core/pull/972#issuecomment-1929187494 that there is refactoring work for input events for ios and web. Will this issue be addressed for wasm or JVM as part of that, or is it something separate?

@eymar I saw in JetBrains/compose-multiplatform-core#972 (comment) that there is refactoring work for input events for ios and web. Will this issue be addressed for wasm or JVM as part of that, or is it something separate?

This issue is rather separate.
The refactoring won't bring new features/fixes itself, but it should make them easier to implement

@eygraber , apparently 1.6.10-dev1583 has it already fixed. I guess the refactoring indeed brought the fix too.

Thanks! Still a couple of things not working on 1.6.10-dev1584:
onPointerEvent(
  eventType = PointerEventType.Press,
  pass = PointerEventPass.Initial,
  onEvent = { event ->
    t = event.button?.index?.toString() ?: "null"
  }
)

On WASM in Chrome, the horizontal scroll and mouse thumb button isn't detected. Mouse forward and back are 4 and 3 respectively
On JVM mouse forward and back are 6 and 5 respectively

pointerInput(Unit) {
              awaitEachGesture {
                t = currentEvent.button?.index?.toString() ?: "null"
              }
}

On WASM in Chrome the browser tab is frozen (probably for the same reason as #2)
On JVM null is printed in an infinite loop

pointerInput(Unit) {
  awaitPointerEventScope {
    while(true) {
      val event = awaitPointerEvent()
        t = event.button?.index?.toString() ?: "null"
      }
  }
}

On WASM in Chrome mouse move, mouse left click, and vertical and horizontal scrolling is a 0
On JVM mouse forward and back are 6 and 5 respectively; mouse move and vertical scrolling is null

onClick(matcher = PointerMatcher.mouse(PointerButton(0)), onClick = { t = "Mouse Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(1)), onClick = { t = "Mouse Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(2)), onClick = { t = "Mouse Middle" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(3)), onClick = { t = "Horizontal Scroll Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(4)), onClick = { t = "Horizontal Scroll Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(5)), onClick = { t = "Mouse Back" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(6)), onClick = { t = "Mouse Front" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(7)), onClick = { t = "Mouse Thumb" })

On WASM in Chrome the horizontal scroll and mouse thumb button isn't detected, and mouse forward and back are detected as horizontal scroll
On JVM mouse forward, back and thumb are only detected if the left, middle, or right mouse button is being held down

On WASM in Chrome, the horizontal scroll and mouse thumb button isn't detected. Mouse forward and back are 4 and 3 respectively

"horizontal scroll" - do you mean holding Shift and moving the mouse wheel? Or is there a special mouse button for horizontal scroll?

pointerInput(Unit) {
   awaitEachGesture {
       t = currentEvent.button?.index?.toString() ?: "null"
    }
}

here, it freezes on jvm for me as well. the block passed into awaitEachGesture never suspends. Adding any await call makes it work correctly on jvm: awaitPointerEvent() (something else to be fixed for web though).

I'm not sure about this snippet:
.onClick(matcher = PointerMatcher.mouse(PointerButton(3)), onClick = { t = "Horizontal Scroll Right" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(4)), onClick = { t = "Horizontal Scroll Left" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(5)), onClick = { t = "Mouse Back" })
  .onClick(matcher = PointerMatcher.mouse(PointerButton(6)), onClick = { t = "Mouse Front" })

PointerButton 3 and 4 are Back and Forward respectively:
val Back = PointerButton(3)
val Forward = PointerButton(4)

But you expect back and front clicks on 5 and 6.
https://github.com/JetBrains/compose-multiplatform-core/blob/4b5650e600ea0612a899478317a5ba7fa775ad13/compose/ui/ui/src/skikoMain/kotlin/androidx/compose/ui/input/pointer/PointerButton.skiko.kt#L29
The pointer button value isn't always the same on all platforms. For example for desktop, there's such mapping:
https://github.com/JetBrains/compose-multiplatform-core/blob/085c24ab598923cdb859f5a2a997d5464420c56a/compose/ui/ui/src/desktopMain/kotlin/androidx/compose/ui/scene/ComposeSceneMediator.desktop.kt#L718

Does it clarify the PointerButton usage a bit?

"horizontal scroll" - do you mean holding Shift and moving the mouse wheel? Or is there a special mouse button for horizontal scroll?

My mouse has a dedicated scroll wheel for horizontal scrolling.

PointerButton 3 and 4 are Back and Forward respectively

Not always 😅
On Linux the codes are different, which I believe is an issue from Swing / awt.
There's an issue for that where I linked a spreadsheet with some investigations I did to see what codes are fired for which action across OS and mouse manufacturer.

I see. Thanks for sharing more details.
currentEvent.button?.index is not 1 to 1 mapping to the value from OS (awt). It's remapped for compose purposes here
Probably the internal remapping should involve the info about the host OS.
Also, now I see a problem that manually creating a PointerButton(index) for onClick purposes can be error-prone, because of the implicit remapping by Compose (desktop) under the hood.
The goal of PointerButton(index) was to make its values common for all Compose platforms.

I think in the scope of this issue, I'll fix the web part:

button will be null in all events except press and release (same as on Desktop). 0 value is indeed a bug.

A couple of questions:

Have you figured out the freeze problem?

pointerInput(Unit) {
  awaitEachGesture {
    // suspend call is missing that's why it freezes
    t = currentEvent.button?.index?.toString() ?: "null"
  }
}

I'll try to get a more advanced mouse. Is horizontal scroll button/wheel clickable like middle mouse wheel?
Does scrolling with horizontal scroll weel work in Compose for you (at least, can you see the pointer event)?
I'm wondering what mouse button index does the browser send for this button when you click it?

I haven't figured out the freezing problem, but I didn't spend too much time on it.
On my mouse the horizontal scroll wheel isn't clickable (and it hasn't been on any mouse that I've tried). I don't remember if the horizontal scroll wheel actually scrolls. I'll try it out.

In regards to:

On Linux the codes are different, which I believe is an issue from Swing / awt.

Are there any plans to address that in Compose?

Does scrolling with horizontal scroll weel work in Compose for you (at least, can you see the pointer event)?

Using 1.6.10-beta02 and Modifier.horizontalScroll(rememberScrollState()) the horizontal scroll wheel works! On JVM I have to hold down shift and scroll the horizontal scroll wheel.

@eymar checking in on https://github.com/JetBrains/compose-multiplatform/issues/4134#issuecomment-2080152772

In regards to:

On Linux the codes are different, which I believe is an issue from Swing / awt.
Are there any plans to address that in Compose?

It seems the Linux issue is still in the backlog.
We'll revise it during the next planning.

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.