Gateway to Pangeo: Modules for Tutorials

I have been thinking about how best to create a Introduction to Pangeo "course" as we have been getting lots of questions from users who are unsure even about the fundamentals of docker containers and Jupyterhub, which one would need to understand to some degree in order to get packages working and to understand what can and cannot be realistically achieved by using the Pangeo platform.
I would like to solicit feedback on how best to market Pangeo to beginners, how to introduce users to the fundamental components that make up Pangeo and to help them envision a workflow that they can port to their own work. There a many things with using Pangeo that might be lost on an entirely new user (such as installing their own conda packages). I feel that it might be best to design such a "course" in more of a niche setting - for example, if we want to design the course in the context of working with NASA datasets and AWS, the modules might look something like this:

Introduction to Basic Python Tools for Collaborative, Reproducible and Open Science
Introduction to Cloud Computing
-- Systems administration (user roles, tagging, cost management), cloud best practices
-- Data storage, getting data in and out of the cloud
Introduction to AWS Open Datasets
-- How do we utilize and work with AWS Open Datasets?
Introduction to xarray
--  Small computations using xarray on your personal computer
Introduction to Dask
-- Dask for scaling, tie in to the cloud
Introduction to the Pangeo ecosystem
-- Scalable computing on the cloud using Pangeo; brief overview of Pangeo bones (e.g. Jupyterhub basics), conda install specific packages
-- How do I install packages I need?
Example complete machine learning workflow

Would something like that be helpful, encourage more uptake and interest in Pangeo and is there a need to develop new notebooks/materials or are there existing notebooks/materials that can be recycled in this context?

Amanda, this is a great idea! A potential user basically just tweeted a plea for this: https://twitter.com/MorganEONeill/status/1225613102483304448

I need a roadmap and/or nested hierarchy of these things and/or literally just a link to the right tutorial/ppt/walk-through of all these different nifty resources...

It reminds me a bit of these great "developer roadmaps", .e.g.
https://roadmap.sh/frontend

Maybe we should try to create such a diagram for Pangeo.

We made a stab at something sort of like that for the HoloViz tools:
https://pyviz-dev.github.io/holoviz/background.html#the-holoviz-ecosystem
(src: https://github.com/holoviz/holoviz/blob/master/doc/flowcharts/holoviz.mermaid.txt)

Making such a roadmap is a great idea. It would help to identify course/modules needed to get started with Pangeo. And I'm sure a big part of these introduction modules are already existing somewhere!

I think this (ATan's) is a fantastic idea as well; and I think now that OSM is done we'll discuss this further at eScience this week; so please send more input and / or request a call-in if you are so inclined. I think Amanda's plan and the related comments here sharpen an idea that came up with Scott of having a (geographically distributed) hack day in March where Pangeo educators ("POETs") who can participate take time out to dig into their favorite Pangeo education topic.

I propose a 3-week sprint at 10 am PDT March 17, 24, 31 to follow up. Details to follow; I just use this and a slack remark and a discourse remark to 'mark the date'. We'll probably use zoom as UW recently got a global license  and it handles large groups gracefully.

Sounds great!

I wanted to take this opportunity to place here an outline of the upcoming course that leverages the Pangeo ecosystem that I’ll be debuting at UAlbany come Fall 2020.
The class, provisionally entitled "Advanced Meteorological Data Analysis and Visualization" will be open to senior-year undergraduates, as well as all graduate students, and its content will be delivered via an "online" course. In the 13 or so weeks I will have available, I currently have about 15 units (will cut back to 13 as I continue to prepare the course) that include the following:

Reproducibility / Binder / Docker / Singularity
Git
Cloud computing
Core Python Language: Python Fundamentals, Functions & Classes, Organization & Packaging (consider having this as a pre-req / review document)
Scientific Python Fundamentals: Numpy and Matplotlib
GIS / Cartopy
GiS / QGIS / Rasterio / Folium *Check out Cloud-optimized GeoTIFF (https://www.cogeo.org/) *
Databases
Pandas
Metpy / Siphon
Xarray
Xarray / Zarr (datasets: ERA5, CFSR, CMIP6)
Interactive visualizations: ipywidgets/ Bokeh / Geoviews
Interactive visualizations: ipyleaflet / Hvplot
Dask

I plan to construct the course content in a manner consistent with a Carpentries-developed course, with Jupyter notebooks presenting both the software and the documentation. The content will be inspired by and hopes to build and extend on these three resources:

Ryan Abernathey’s Research Computing class at Columbia: https://earth-env-data-science.github.io/intro
Brian Rose’s Climate physics / modeling class at UAlbany: https://brian-rose.github.io/ClimateLaboratoryBook
Damien Irving’s Data Carpentry lesson for AOS: https://carpentrieslab.github.io/python-aos-lesson/

I envision that this class would be a logical outgrowth /extension of Damien’s lessons, but for a semester-long class rather than a one-day Carpentries session. Ample time for a really deep  and wide dive!
Ideally, many of these notebooks that will be developed as part of the class could become part of Pangeo's gallery of content/notebooks. My plan is also to obtain an education grant from one of the commercial cloud providers such that much of the work will be done in the cloud.
Students would be expected to contribute to FOSS developed for this course (as well as allied with it) and will also have a final project that demonstrates reproducibility ... (following Irving, 2016: https://doi.org/10.1175/BAMS-D-15-00010.1 ).
Ultimately, this class, given its online nature, could be taken by anyone, regardless of UAlbany matriculation status, and the intent is to make its core content available to anyone via The Carpentries.
Feedback is welcome ... Github link will be forthcoming over the next couple of months.

Having given a number of tutorials to NCAR staff in the last year or two, one thing we've definitely noticed is that there are many attendees who don't even have a good footing in Python.  We've (@jukent, @andersy005, et al.) been taking it on ourselves to build a "base" (or Python newbie) layer into our tutorial material to get people up to speed on Python before they jump into even the most basic Python tools.  Our staff usually can be trusted to know "a programming language" and even be familiar with "scientific programming" in that language, but learning and building comfort with Python seems challenging for some of them.
Has any consideration been given to what the "introduction for people who don't know Python" should look like?  I'm aware of a number of different "Python for Programmers" materials (some in physical book form), but after quite a bit of experience with giving tutorials to existing, experience scientists who "just" don't know Python, I'm not sure those materials are the best.  Perhaps you all know of something better?


Regex replace with single space replaces with no space

VSCode Version: Code 1.8.1 (ee428b0eead68bf0fb99ab5fdc4439be227b6281, 2016-12-19T14:49:23.350Z)
OS Version: Windows_NT ia32 10.0.14393
Extensions:

Extension
Author
Version

cpptools
ms-vscode
0.9.2

Steps to Reproduce:

In an MD file, add this text: HRESULT OnAmbientPropertyChange(DISPID   dispid);

fire up replace dialog and insert this regex: \b\s{3}\b and in the replace box put a single literal space

expected:  "DISPID dispid" actual: "DISPIDdispid"

@mikeblome Very nice find!
FYI @sandy081 The ReplacePattern in /platform/ was very weird, it was trying to rematch on the specific matched string. i.e. in this case it would try to match again /\b\s{3}\b/ against "   " (3 spaces), which of course would fail.
IMHO the only correct way to fix this in the find widget was to record capture groups in the actual model search and then use them for group substitutions, which is what I have implemented. I did not adopt these changes in the ReplacePattern in /platform/ because I could not understand how it is used by the search service, so I "forked" it.
I am not sure if the search/replace in files suffers from the same issue or not, I leave it up to you to investigate that and adopt the new ReplacePattern I've written.

@alexandrudima You are right that replace pattern is performing match on the given text to get the replace string. But, this is not doing on a match. It depends on what value is passed to getReplaceString. To get replace string for the current match, I am passing the text starting from the current match. I agree it is bit weird but rematching was done to fix the look ahead reg ex. I am not sure how this could be the reason, since it was not matching the specific match, instead the complete text from the beginning of the match.
But I liked your approach of recording capture groups while searching. Will investigate how to adopt this in Search.


Reduce line count

This PR is formatting changes only, totally non-urgent.
Reduce line count by setting a couple of rustfmt config options to be the same as they are in rust-bitcoin. Final patch removes whitespace in trait impl blocks, and was the impetus for this PR.

I think increasing chain_width is mostly a bad change.

But 0c7a972bc3693f7cc0a6fed108ad11f2b8afdbb6 looks good otherwise.

So, assuming the comment I wrote in the rustfmt config file of rust-bitcoin is correct, we can't keep chain_width with a custom setting and not all the others if we want to use use_small_heuristics = "Max" - is the chain width annoying enough to warrant dropping this whole patch? Real question, this is your and @sanket1729's call, I'm jonny come lately to this crate.

I split each config option change into a separate patch so they can be viewed easily. I actually think the only massive win is fn_single_line and maybe struct_lit_width.

FTR docs say the values are a percentage of max_width but anything over 80 causes an error, so there is a bug somewhere.
fn_call_width cannot have a value that exceeds max_width. fn_call_width will be set to the same value as max_width

Not really a fan of the array_width change either.

No worries, I'll drop that commit - that is exactly why I kept them all separate :)

Force push removes the array_width change (was the last patch of the series).

Otherwise https://github.com/rust-bitcoin/rust-miniscript/commit/75d3fa8da07c4ae62f6194c3078f529c924f479b looks good.

This is the commit hash of the removed patch.

Rebased, no other changes.

Re-done after planning module merging. @TheBlueMatt this one should make you all warm and fuzzy inside :)

@tcharding, what order do you prefer I merge the PRs?

This one first please, it will cause loads of conflicts but that is always going to be the case whenever it merges.


Release cobigen-eclipse-v4.4.4

This issue has been automatically created. It serves as a container for all release related commits

Automatically processed.
The decisions taken by the developer and the context of the decisions throughout the script:

User confirmed that tests are running on CI and the build is not failing.
User confirmed that tests are running on CI and the build is not failing after pushing release changes to development branch.


Transition message on jira.zephyrproject.org needed

We need some message on jira.zephyrproject.org indicating it's no longer being used for tracking Zephyr issues, with a pointer referring folks to the GitHub issues page instead.

+1

done

@nashif : Nice. But note that on https://jira.zephyrproject.org/ , in the "Introduction" box (below the notice which you added) it says:

The Zephyr™ Project uses this JIRA system to collect and track requirements and reported issues.

Maybe it would make sense to do something about that sentence too, e.g. remove "and report issues", or maybe remove the sentence completely.


how can i add multiple bots within the same NestJs App

What is your question?
I have three channels within different self-bot, and I want to listen multiple bot messages in a NestApp.
How can i add multiple bots within the same NestJs App?
Thanks for your help!

@iafine Thank you for your question! Multiple bots not supported due to the technical capabilities of @nestjs
You can import all your bot modules to app module
AppModule
├── FirstBotModule (With NecordModule.forRoot)
└── SecondBotModule (With another NecordModule.forRoot)
But Necord using global module for explore all providers with commands.
For this reason, usage of multiple bots may to be hard. (Injecting client provider and other services overlaps)
Also, don't forget about ratelimits by ip, usage multiple popular bots can have restrictions
If you know solution how to support it better feel free to create a PR


ci(all): add bootstrap command for examples

Now it runs bootstrap on both the example and the plugin:
melos bootstrap
  └> /Users/runner/work/plus_plugins/plus_plugins
Running "flutter pub get" in workspace packages...
  ✓ android_intent_plus
    └> packages/android_intent_plus
  > SUCCESS
Generating IntelliJ IDE files...
  > SUCCESS
 -> 1 packages bootstrapped
melos bootstrap
  └> /Users/runner/work/plus_plugins/plus_plugins
Running "flutter pub get" in workspace packages...
  ✓ android_intent_example
    └> packages/android_intent_plus/example
  > SUCCESS

ok, the problem is just that android_intent_plus example doesn't follow the pattern android_intent_plus_example.


Change navbar links

We should change the navbar to have the following links:

About (Home)
Docs
Apply

I believe Chase has already done Community in the sparkDev branch, so we don't need to do it again. We can get rid of extra links.

I'm actually thinking we will call the Docs page Resources instead. Thoughts?


[pigeon] Regression in number handing on Android

What package does this bug report belong to?
pigeon
What target platforms are you seeing this bug on?
Android
Have you already upgraded your packages?
Yes
Dependency versions
pigeon: ^22.4.0
Steps to reproduce

Configure pigeons to generate Messages.java for Android
Add a function that takes an integer as a parameter to messages.dart
Generate pigeons
Implement the generated interface from Messages.java in native plugin's kotlin source code
Call the method from Flutter

Expected results
The native function gets executed, and the integer is passed from Flutter to Kotlin
Actual results
The native function does not get executed, and instead an exception is thrown: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long (see full log under "Logs" section)
It appears to be a regression caused by this pull request: https://github.com/flutter/packages/pull/7392, which was made in response to this issue https://github.com/flutter/flutter/issues/136111
pigeon: ^21.2.0 does not suffer from such an issue
pigeon: ^22.0.0 and all versions above do suffer from such an issue
Code sample
I have created a repository with a reproducible example: https://github.com/vanyasem/pigeon-bug
Logs
Logs
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): Failed to handle message
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at com.example.pigeon_bug.Messages$IntPigeonBugApi$-CC.lambda$setUp$0(Messages.java:106)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at com.example.pigeon_bug.Messages$IntPigeonBugApi$$ExternalSyntheticLambda0.onMessage(Unknown Source:2)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at io.flutter.plugin.common.BasicMessageChannel$IncomingMessageHandler.onMessage(BasicMessageChannel.java:261)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at io.flutter.embedding.engine.dart.DartMessenger.invokeHandler(DartMessenger.java:292)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at io.flutter.embedding.engine.dart.DartMessenger.lambda$dispatchMessageToQueue$0$io-flutter-embedding-engine-dart-DartMessenger(DartMessenger.java:319)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at io.flutter.embedding.engine.dart.DartMessenger$$ExternalSyntheticLambda0.run(Unknown Source:12)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at android.os.Handler.handleCallback(Handler.java:959)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at android.os.Handler.dispatchMessage(Handler.java:100)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at android.os.Looper.loopOnce(Looper.java:232)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at android.os.Looper.loop(Looper.java:317)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at android.app.ActivityThread.main(ActivityThread.java:8592)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at java.lang.reflect.Method.invoke(Native Method)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:580)
E/BasicMessageChannel#dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction(26398): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:878)
E/flutter (26398): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(channel-error, Unable to establish connection on channel: "dev.flutter.pigeon.pigeon_bug.IntPigeonBugApi.testIntFunction"., null, null)
E/flutter (26398): #0      IntPigeonBugApi.testIntFunction (package:pigeon_bug/messages.g.dart:46:7)
E/flutter (26398): <asynchronous suspension>
E/flutter (26398): #1      _MyAppState._testInt (package:pigeon_bug_example/main.dart:38:5)
E/flutter (26398): <asynchronous suspension>
E/flutter (26398):

Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.24.3, on macOS 15.0 24A335 darwin-x64, locale en-US)
    • Flutter version 3.24.3 on channel stable at /Users/vanyasem/fvm/versions/3.24.3
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 2663184aa7 (11 days ago), 2024-09-11 16:27:48 -0500
    • Engine revision 36335019a8
    • Dart version 3.5.3
    • DevTools version 2.37.3

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/vanyasem/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 16A242d
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2024.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)

[✓] VS Code (version 1.93.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.94.0

[✓] Connected device (6 available)
    -- Android/iOS devices redacted for privacy reasons --
    • macOS (desktop)        • macos                     • darwin-x64     • macOS 15.0 24A335 darwin-x64
    • Chrome (web)           • chrome                    • web-javascript • Google Chrome 129.0.6668.59

[✓] Network resources
    • All expected network resources are available.

4. Implement the generated interface from Messages.java in native plugin's kotlin source code

Is there a reason you are generating Java, rather than Kotlin, if the goal is to implement the plugin in Kotlin?
(Not to say that it shouldn't work, but it's not something we test extensively, so we would like to understand the use case.)

I've identified the issue and will have a fix up shortly

@stuartmorgan I don't really remember at that point. The decision to stick to Java was made a pretty long time ago
IIRC it was something related to namespace collisions when using Kotlin (?) in pigeon v10.1.5
We will reevaluate the decision some time later, I will get back to you with the results

(For posterity: this turned out to be unrelated to the mixture of Java and Kotlin, it was a generator bug that depended on not having any custom classes anywhere in the Pigeon definition file, which is why our tests missed it.)


No main generated

The following adoc-example will not be wrapped in a main:
[source,kotlin]
----
fun myMethod(notOptional: String, optional: String = "defaultValForOptionalString"){
    println("myMethod called with $notOptional and $optional")
}

//Possible Calls
myMethod("arg1", "arg2")
myMethod("arg1")
----

Trying to verify will lead to an error: No 'main' function in ...kt
Another example:
[source,kotlin]
----
data class Person(val name: String, val age: Int, val job: String)
----

Does it have to be excluded explicitly?
Kind regards
simon

Actually the README is incorrect. I'm going to update it.


Fluent Terminal Support?

I know you added open in terminal support but you could explain to me how I could possibly add Fluent Terminal as an option? Fluent Terminal is a UWP app just like this one. I don't know what path or argument I should put.

cc @lampenlampen

According to https://github.com/felixse/FluentTerminal/pull/194:
{
      "id": 5,
      "name": "Fluent Terminal",
      "path": "flute.exe",
      "arguments": "new \"{0\}",
      "icon": ""
}

@EmiyaBestBoy does it work?

It works. Tested it.

@lampenlampen Thank you for testing.
@EmiyaBestBoy I hope your issue has been resolved, feel free to reopen this issue or open a new one if you run into additional issues or if you have feedback for Files UWP.


feat: add web3modal for wallet connect support. disable metamask client.

TODO

[ ] Find a way to create provider from web3wallet hook inside the client functions - consider if that's needed at all?
[x] grab wc project id from env

@SergeyMushta cleaned up a bit, PR should be ready now


Make sure that 'source' isn't repeated twice for path provider

I believe this regression was introduced with e4480c587f86f59ca5bfa2de7d605fe636155f5b (but it was the right way to do stuff, so I'm just fixing the code around to adapt it to the correct behaviour).

Thanks Slavek!


feat: add protobuf 3.24.4 release

update protobuf with release 3.24.4

:tada: This PR is included in version 1.215.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Updated iOS image path

This plugin worked great for me in the past on a number of projects, but with a new project using ember-cordova the splash path in the iOS build seems to have changed? Updating the path worked.

It was indeed a regression. It should be solved with version 0.9.0 :)


[ui] Implement vehicle details component

When the user clicks on a vehicle marker, details of that vehicle should be displayed.
I have added the click handler and I think I have all the image assets I need. I just need to make the component and wire it up.

Closed with #7
It could still be enhanced but the basics are there: Distance from user, fuel level and an image of the vehicle model.
I got to flex my very modest image editing chops. Editing out the white background and making the shadow semi-transparent took me hours! I'm quite happy with the results, I think they both look great.

Closed with #7
It could still be enhanced but the basics are there: Distance from user, fuel level and an image of the vehicle model.
I got to flex my very modest image editing chops. Editing out the white background and making the shadow semi-transparent took me hours! I'm quite happy with the results, I think they both look great.


Why there are no switches?

Please give an switcher on this meta boxed on te place of checkbox instead

You can add checkbox using just a CSS syntex.
Please read this article https://alvarotrigo.com/blog/toggle-switch-css/


Feature/http worker pool

Reusable code for implementing a pooled HTTP client with optional rate-limiting.

Current coverage is 80.75% (diff: 100%)

Merging #16 into master will decrease coverage by 0.64%

@@             master        #16   diff @@
==========================================
  Files            42         42
  Lines          1081       1081
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits            880        873     -7
- Misses          173        179     +6
- Partials         28         29     +1

Powered by Codecov. Last update b861f60...92f74fa


[Gally]: master <- dev

Automatically created by Git-Ally

:tada: This PR is included in version 2.2.9 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


casting INF to int yield negative value

Tested versions
v4.2.1.stable.official [b09f793f5]
System information
Godot v4.2.1.stable - Windows 10.0.19045 - Vulkan (Forward+) - dedicated NVIDIA GeForce GTX 1070 Ti (NVIDIA; <IP_ADDRESS>86) - AMD Ryzen 7 2700X Eight-Core Processor (16 Threads)
Issue description
writing the following:
var i:int = INF
aka, making an explicit type cast from the INF which is of type float to i which is of type int causes the value to be negative, which is unexpected.
It correctly treats:
var j:int = -INF
i & j are equal, which is usually wrong. I would expect int(INF) to be a large positive integer,<PHONE_NUMBER>854775808 to be precise.
Steps to reproduce

paste the following into any GDScript file: print(int(INF))
Run said code
Get printout "-9223372036854775808"

Minimal reproduction project (MRP)
N/A

Thank you for your report, consolidating in:

https://github.com/godotengine/godot/issues/33017


端口测试不通，No connection adapters were found

dll显示已经注入成功，也能查到，但运行client.py，测试联通性时，报错    raise InvalidSchema(f"No connection adapters were found for {url!r}")
requests.exceptions.InvalidSchema: No connection adapters were found for '<IP_ADDRESS>:19088/api/?type=0'

<IP_ADDRESS>

上面是我测试过程中地址改错了，恢复成<IP_ADDRESS> 或者<IP_ADDRESS>，结果也是一样。

python的示例地址头部加上http://，可以联通，但是报另外一个错误{"code":200,"data":null,"msg":"not support url"}


WIP: Add Windows CI

Done

[x] Add Windows CI via Appveyor
[x] Pave the road for releases to be generated from Appveyor (Windows) or Travis (OSX / Linux)
[x] Provide a pathway for Node v4.2.1 / apm 1.5.x, but do not make them the default for internal Atom CI
[x] Make various improvements to script/bootstrap, script/cibuild, and build/Gruntfile.coffee (via build/Gruntfile-ci.coffee
[x] :arrow_up: Update Electron To 0.34.3

TODO

[ ] Suppress remoteReleases synchronization on anything but beta or stable
[ ] Add caching of electron, other heavy downloads
[ ] Create zip archive for Windows
[ ] Add artifacts explicitly instead of all of atom-build
[ ] Disable 0.10.35 builds on Windows (we now know they work)
[ ] Add signtool tool
[ ] Add encrypted ATOM_ACCESS_TOKEN to .travis.yml, with scopes necessary for release upload
[ ] Fix .deb package generation
[ ] Add encrypted signing keys for Travis and Appveyor, with corresponding encrypted variable

:rotating_light: WIP :rotating_light:

Nice. Appreciate the task list for status.

This would also probably be a good time to switch Travis's language to node_js.  Right now it's unset so it's defaulting to Ruby, which really isn't helping at all.  Switching to node_js should allow us to get rid of the manual nvm installation/Node configuration steps and let Travis handle it for us.
language: node_js
node_js:
  - "4.2.1" # or alternatively, "stable"
  - "0.10.35" # 0.10 should work as well

Appveyor also seems to support directly setting Node versions, though slightly differently from Travis: http://www.appveyor.com/docs/lang/nodejs-iojs

Appveyor also seems to support directly setting Node versions, though slightly differently from Travis: http://www.appveyor.com/docs/lang/nodejs-iojs

Already using the technique described.

This would also probably be a good time to switch Travis's language to node_js.

Done.

@50Wliu I have to back out use of the language: node_js support; it's not available on OSX yet: https://github.com/travis-ci/travis-ci/issues/2311.

it's not available on OSX yet: travis-ci/travis-ci#2311.

Ugh, whyyy

Kicked off a build on our internal CI infra so we can be sure we're still working there, too.

Kicked off another build after the force-push. Sorry, should have done this yesterday.

:zap: Thanks for your patience on this, both with the build and with us. Very excited about moving to more commodified build infra.

:clap: :moneybag: :chart_with_upwards_trend:

:rocket: :guitar: :clap:


Deprecated usage of vec_unchop() from {vctrs} 0.5.0

Description
{dataxray} makes use of code that is deprecated with {vctrs} 0.5.0.
Reproducible example
Using mostly the code from the README.
library(dplyr)
library(dataxray)
data(diamonds, package = "ggplot2")

diamonds <- diamonds %>%
  mutate(price = structure(price, label = 'price in US dollars'),
         carat = structure(carat, label = 'weight of the diamond'),
         cut = structure(cut, label = 'quality of the cut (Fair, Good, Very Good, Premium, Ideal)'),
         color = structure(color, label = 'diamond colour, from D (best) to J (worst)'),
         clarity = structure(clarity, label = 'a measurement of how clear the diamond is
                                               (I1 (worst), SI2, SI1, VS2, VS1, VVS2, VVS1, IF (best))'),
         x = structure(x, label = 'length in mm'),
         y = structure(y, label = 'width in mm'),
         z = structure(z, label = 'depth in mm'),
         depth = structure(depth, label = 'total depth percentage = z / mean(x, y) = 2 * z / (x + y)'),
         table = structure(table, label = 'width of top of diamond relative to widest point'))

withr::with_options(list(lifecycle_verbosity = "error"), {
  diamonds %>%
    report_xray(data_name = 'Diamonds', study = 'ggplot2')
})

Quitting from lines 51-54 (ggplot2_Diamonds_xray.rmd)
Error in `mutate()`:
! Problem while computing `attributes = list(...)`.
ℹ The error occurred in row 10.
Caused by error:
! `vec_unchop()` was deprecated in vctrs 0.5.0.
ℹ Please use `list_unchop()` instead.
Backtrace:
  1. dataxray::make_xray(data, by = params$by)
 14. dplyr:::mutate.data.frame(...)
 15. dplyr:::mutate_cols(.data, dplyr_quosures(...), caller_env = caller_env())
 17. vctrs::vec_unchop(chunks, rows)
 18. lifecycle::deprecate_soft("0.5.0", "vec_unchop()", "list_unchop()")
 19. lifecycle:::deprecate_stop0(msg)

Expected result
No error.
Session info

sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8
 [2] LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8
 [4] LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8
 [6] LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8
 [8] LC_NAME=C
 [9] LC_ADDRESS=C
[10] LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8
[12] LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils
[5] datasets  methods   base

other attached packages:
 [1] crosstalk_1.2.0 haven_2.5.1
 [3] forcats_0.5.2   stringr_1.4.1
 [5] purrr_0.3.5     readr_2.1.3
 [7] tidyr_1.2.1     tibble_3.1.8
 [9] ggplot2_3.4.0   tidyverse_1.3.2
[11] dataxray_0.1.0  dplyr_1.0.10

loaded via a namespace (and not attached):
 [1] fs_1.5.2            reactable_0.3.0
 [3] fontawesome_0.3.0   lubridate_1.8.0
 [5] RColorBrewer_1.1-3  httr_1.4.4
 [7] tools_4.2.1         backports_1.4.1
 [9] bslib_0.4.1         utf8_1.2.2
[11] R6_2.5.1            rpart_4.1.19
[13] Hmisc_4.7-1         DBI_1.1.3
[15] lazyeval_0.2.2      colorspace_2.0-3
[17] nnet_7.3-18         withr_2.5.0
[19] tidyselect_1.2.0    gridExtra_2.3
[21] curl_4.3.3          compiler_4.2.1
[23] rvest_1.0.3         cli_3.4.1
[25] htmlTable_2.4.1     xml2_1.3.3
[27] plotly_4.10.0       sass_0.4.2
[29] scales_1.2.1        checkmate_2.1.0
[31] digest_0.6.30       foreign_0.8-83
[33] rmarkdown_2.17      base64enc_0.1-3
[35] jpeg_0.1-9          pkgconfig_2.0.3
[37] htmltools_0.5.3     dbplyr_2.2.1
[39] fastmap_1.1.0       htmlwidgets_1.5.4
[41] rlang_1.0.6         readxl_1.4.1
[43] rstudioapi_0.14     shiny_1.7.3
[45] jquerylib_0.1.4     generics_0.1.3
[47] jsonlite_1.8.3      googlesheets4_1.0.1
[49] magrittr_2.0.3      reactablefmtr_2.0.0
[51] Formula_1.2-4       interp_1.1-3
[53] Matrix_1.5-1        Rcpp_1.0.9
[55] munsell_0.5.0       fansi_1.0.3
[57] lifecycle_1.0.3     stringi_1.7.8
[59] yaml_2.3.6          grid_4.2.1
[61] promises_<IP_ADDRESS>    crayon_1.5.2
[63] deldir_1.0-6        lattice_0.20-45
[65] splines_4.2.1       hms_1.1.2
[67] flexdashboard_0.6.0 knitr_1.40
[69] pillar_1.8.1        reprex_2.0.2
[71] glue_1.6.2          evaluate_0.17
[73] latticeExtra_0.6-30 data.table_1.14.4
[75] modelr_0.1.9        png_0.1-7
[77] vctrs_0.5.0         tzdb_0.3.0
[79] httpuv_1.6.6        cellranger_1.1.0
[81] gtable_0.3.1        reactR_0.4.4
[83] assertthat_0.2.1    cachem_1.0.6
[85] xfun_0.34           mime_0.12
[87] xtable_1.8-4        broom_1.0.1
[89] later_1.3.0         survival_3.4-0
[91] googledrive_2.0.0   viridisLite_0.4.1
[93] gargle_1.2.1        memoise_2.0.1
[95] cluster_2.1.4       ellipsis_0.3.2

diamonds %>%
report_xray(data_name = 'Diamonds', study = 'ggplot2')


Support unicode for paths on Python 2.

Also see discussion in issue #13.

Coverage decreased (-0.5%) to 86.203% when pulling f3bdf4bd3172600513282b669468a8f2e279f202 on feature/py2-unicode-paths into c6eb3c66bddb539c9e70dab4efa568ee6e8f4b32 on develop.

Coverage decreased (-0.5%) to 86.203% when pulling f3bdf4bd3172600513282b669468a8f2e279f202 on feature/py2-unicode-paths into c6eb3c66bddb539c9e70dab4efa568ee6e8f4b32 on develop.

Coverage increased (+0.03%) to 86.749% when pulling 5871bf9885af3cd8f1860951318191135ec90503 on feature/py2-unicode-paths into c6eb3c66bddb539c9e70dab4efa568ee6e8f4b32 on develop.

Coverage increased (+0.03%) to 86.749% when pulling 7bdeec78a1d559ba46d3e6f73396154a8fd140f8 on feature/py2-unicode-paths into c6eb3c66bddb539c9e70dab4efa568ee6e8f4b32 on develop.

Coverage increased (+0.03%) to 86.749% when pulling 7bdeec78a1d559ba46d3e6f73396154a8fd140f8 on feature/py2-unicode-paths into c6eb3c66bddb539c9e70dab4efa568ee6e8f4b32 on develop.

@native-api Can you test if this branch fixes the problem for you?

Coverage decreased (-0.03%) to 86.694% when pulling 04474c9a078a1bce8a228e274ce4a6b08647998c on feature/py2-unicode-paths into c6eb3c66bddb539c9e70dab4efa568ee6e8f4b32 on develop.

Coverage decreased (-0.03%) to 86.694% when pulling b2d002662e078f65c35054764e7807c4bb57978d on feature/py2-unicode-paths into c6eb3c66bddb539c9e70dab4efa568ee6e8f4b32 on develop.


[RUN-2829] exclude --pdf-renderer renderer processes from recording

We crash while recording the pdf renderer process when the print dialog comes up.  Turns out that is a --type=renderer process (with an additional --pdf-renderer flag.)
Turn off recording for --pdf-renderer renderers.
This might be the cause of RUN-2829.

rebased off current master :point_up:


Removed request_eu? helper method

Description
This PR removes the request_eu? helper helper method from this gem. In practice, it rolls back this PR.
The main reason to remove it, is that it seems to be unrelated to the purpose of this gem. It doesn't seem to belong here.
Additionally, what triggered me to open this PR: The gem depends on ActionController::Helpers being available in the ApplicationController, which is not the case for Rails api-only projects.
Library-Specific

[X] Increment the changelog (CHANGELOG.md)

Thanks. We use this helper as a convenience method for CloudFront headers. So it does feel like it somewhat belongs. Could it use more documentation or a change to avoid issues you were having?

Thanks for the quick response. My original message was a bit rushed, didn't want to come over as demanding or ungrateful, so allow me to clarify.
I disagree that the helper somewhat belongs here, at least if you consider this a general-purpose gem, based on the readme or the gem description.

Configure ActionDispatch::RemoteIp trusted proxies for Amazon CloudFront.

Adding a view helper method to the ApplicationController, to specifically detect whether a user is in the EU from CF headers is really something different, and feels very random and app-specific to me. It also assumes more about the app that adopts it than needed for its purpose, like ApplicationController being the root controller and ActionController::Helpers being used. So I think removing it, from that perspective, is the right decision.
I thought about solutions to solve it, like

Add some kind of parameter to make adding the helper optional
Use a different mechanism than a mixed in helper method to provide the functionality

But both solutions would require changes to your application (that's using the gem) anyway, which draws me back to the original conclusion that it would make more sense to make that code part of your application, or another gem, and remove it from this gem entirely.

If this PR is something you can't make work with the way you use the gem, or you disagree, that's your decision to make, and I totally understand and respect that. Feel free to close this. Thank you for providing the gem in the first place, it's saving us quite some work, even if we'll have to run with a fork!

didn't want to come over as demanding or ungrateful

Nah! Came across just fine!

But both solutions would require changes to your application

Well, the config could default to true and for your app you could set it to false. This would be important for us since we are using the method now. Then maybe in a future major version number we can default that the value to false. How does that sound?


Which parameter should we tweak to get a BkdTree without using Disk

Thanks for the library! Is there a way to use this BkdTree fully in memory for better performance skipping the disk overheads and also encoding/decoding overhead.
Any ways to tweak this like increasing t0M values or something?

The BkdTree data-structure is designed for external storage. This Golang implementation use mmap to speedup file read/write. (However mmap cost much RAM. It's better to make mmap optional.)
There are some parameters to tweak for performance: t0mCap, leafCap, intraCap.
t0m is an array of points in RAM.
leafCap is the capacity of a leaf node. Array under that size is not divided any more.
intraCap is the capacity of an internal node.
My benchmark doesn't show obvious difference of tweaking them.

Thanks for the reply.


test: fuzzer binaries should parse gmock flags.

Description: Parsing gmock flags is important for fuzzers that make use of NiceMock. The mocks log all interactions that are not expected. The volume of logging can be overwhelming. This change allows the use of the flag --gmock_verbose=error, which suppresses these logs.
Risk Level: Low
Testing: Run Fuzzers with the flag --gmock_verbose, see that they change their output level.
Docs Changes: None
Release Notes: None
Signed-off-by: Sam Kerner<EMAIL_ADDRESS>

/retest

Here is a presubmit error I do not understand:
./test/fuzz/fuzz_runner.h:11:10: error: 'compiler_rt/fuzzer/utils/FuzzedDataProvider.h' file not found [clang-diagnostic-error]
#include "compiler_rt/fuzzer/utils/FuzzedDataProvider.h"
            ^

Source
It appears that the presubmit can't read a file in Complier-RT.  This change does not add an include on the file in question.  Reviewers:  What am I missing?

It appears that the presubmit can't read a file in Complier-RT. This change does not add an include on the file in question. Reviewers: What am I missing?

I think there might be something up with clang-tidy... i'm looking in to it

I applied this diff to fix the clang-tidy warning, I'm not sure exactly why besides that other external_deps were working @lizan:
diff --git a/bazel/repositories.bzl b/bazel/repositories.bzl
index 415455e58..4986bba34 100644
--- a/bazel/repositories.bzl
+++ b/bazel/repositories.bzl
@@ -663,6 +663,10 @@ def _org_llvm_releases_compiler_rt():
         name = "org_llvm_releases_compiler_rt",
         build_file = "@envoy//bazel/external:compiler_rt.BUILD",
     )
+    native.bind(
+      name = "fuzzed_data_provider",
+      actual = "@org_llvm_releases_compiler_rt//:fuzzed_data_provider",
+    )

 def _fuzzit_linux():
     _repository_impl(
diff --git a/test/fuzz/BUILD b/test/fuzz/BUILD
index d46ae39a5..a9794393f 100644
--- a/test/fuzz/BUILD
+++ b/test/fuzz/BUILD
@@ -40,6 +40,9 @@ envoy_cc_test_library(
     name = "fuzz_runner_lib",
     srcs = ["fuzz_runner.cc"],
     hdrs = ["fuzz_runner.h"],
+    external_deps = [
+        "fuzzed_data_provider",
+    ],
     deps = [
         "//source/common/common:minimal_logger_lib",
         "//source/common/common:thread_lib",
@@ -49,7 +52,6 @@ envoy_cc_test_library(
         "//source/exe:process_wide_lib",
         "//test/test_common:environment_lib",
         "@com_github_google_libprotobuf_mutator//:libprotobuf_mutator",
-        "@org_llvm_releases_compiler_rt//:fuzzed_data_provider",
     ],
 )

diff --git a/test/fuzz/fuzz_runner.h b/test/fuzz/fuzz_runner.h
index e2279365c..3f3cc60aa 100644
--- a/test/fuzz/fuzz_runner.h
+++ b/test/fuzz/fuzz_runner.h
@@ -8,7 +8,7 @@
 #include "libprotobuf_mutator/src/libfuzzer/libfuzzer_macro.h"
 // Bring in FuzzedDataProvider, see
 // https://github.com/google/fuzzing/blob/master/docs/split-inputs.md#fuzzed-data-provider
-#include "compiler_rt/fuzzer/utils/FuzzedDataProvider.h"
+#include "utils/FuzzedDataProvider.h"
 #include "spdlog/spdlog.h"

 namespace Envoy {

Thank! Would you mind commenting a before/after execution speed on local runs with a mock-heavy target, maybe //test/common/http:conn_manager_impl_fuzz_test?

Build the binary:
$ bazel build //test/common/http:conn_manager_impl_fuzz_test_with_libfuzzer --config asan-fuzzer -c opt

Running with the old setting gets 7-11 executions per second:
$ bazel-bin/test/common/http/conn_manager_impl_fuzz_test_with_libfuzzer --gmock_verbose=warn |& grep pulse
#32     pulse  cov: 76319 ft: 84636 corp: 27/2557b lim: 4096 exec/s: 10 rss: 284Mb
#64     pulse  cov: 76340 ft: 85878 corp: 49/5093b lim: 4096 exec/s: 11 rss: 332Mb
#128    pulse  cov: 76386 ft: 91483 corp: 76/9900b lim: 4096 exec/s: 9 rss: 427Mb
#256    pulse  cov: 76395 ft: 92642 corp: 122/21Kb lim: 4096 exec/s: 7 rss: 620Mb

Running with the new default gets 13-16 executions per second:
$ bazel-bin/test/common/http/conn_manager_impl_fuzz_test_with_libfuzzer --gmock_verbose=error |& grep pulse
#32     pulse  cov: 73843 ft: 80455 corp: 28/5747b lim: 4096 exec/s: 16 rss: 250Mb
#64     pulse  cov: 74201 ft: 81716 corp: 41/10632b lim: 4096 exec/s: 16 rss: 297Mb
#128    pulse  cov: 76531 ft: 88285 corp: 72/19Kb lim: 4096 exec/s: 16 rss: 424Mb
#256    pulse  cov: 76667 ft: 90313 corp: 119/33Kb lim: 4096 exec/s: 13 rss: 613Mb

This seems like something we might want as a default when building fuzz targets, would you be able to add it to .bazelrc for the asan-fuzzer config? But then for our OSS-Fuzz builds, underlyingly we build the targets as cc_library, which can't take args in the build rule. Is it possible to set GMOCK_FLAG(verbose) to default error (GMOCK_FLAG(verbose) = "error")?

I set the default in code in fuzz_runner.cc.  This applies to fuzzing runs, but not to cases where the binary is used to replay a corpus of tests.  When fuzzing, we want speed.  When replaying, having logs show the details of what is happing is helpfull.

Re the clang-tidy warning:  Let me know if I should apply that patch, rebase to a point where it is in the tree, or just ignore the errors.

@sqkerner @asraa the bind part does nothing. Let's do this patch:
diff --git a/bazel/external/compiler_rt.BUILD b/bazel/external/compiler_rt.BUILD
index e75ac2ef26..96d90b46ab 100644
--- a/bazel/external/compiler_rt.BUILD
+++ b/bazel/external/compiler_rt.BUILD
@@ -2,8 +2,6 @@ licenses(["notice"])  # Apache 2

 cc_library(
     name = "fuzzed_data_provider",
-    hdrs = ["utils/FuzzedDataProvider.h"],
-    # This is moving from lib/fuzzer/utils to include/fuzzer after LLVM 9.0.
-    include_prefix = "compiler_rt/fuzzer",
+    hdrs = ["fuzzer/utils/FuzzedDataProvider.h"],
     visibility = ["//visibility:public"],
 )
diff --git a/bazel/repository_locations.bzl b/bazel/repository_locations.bzl
index cebb98e124..306bf7f291 100644
--- a/bazel/repository_locations.bzl
+++ b/bazel/repository_locations.bzl
@@ -302,7 +302,7 @@ REPOSITORY_LOCATIONS = dict(
     org_llvm_releases_compiler_rt = dict(
         sha256 = "56e4cd96dd1d8c346b07b4d6b255f976570c6f2389697347a6c3dcb9e820d10e",
         # Only allow peeking at fuzzer related files for now.
-        strip_prefix = "compiler-rt-9.0.0.src/lib/fuzzer",
+        strip_prefix = "compiler-rt-9.0.0.src/lib",
         urls = ["http://releases.llvm.org/9.0.0/compiler-rt-9.0.0.src.tar.xz"],
     ),
     fuzzit_linux = dict(
diff --git a/test/fuzz/fuzz_runner.h b/test/fuzz/fuzz_runner.h
index e2279365ca..31a317a220 100644
--- a/test/fuzz/fuzz_runner.h
+++ b/test/fuzz/fuzz_runner.h
@@ -8,7 +8,7 @@
 #include "libprotobuf_mutator/src/libfuzzer/libfuzzer_macro.h"
 // Bring in FuzzedDataProvider, see
 // https://github.com/google/fuzzing/blob/master/docs/split-inputs.md#fuzzed-data-provider
-#include "compiler_rt/fuzzer/utils/FuzzedDataProvider.h"
+#include "fuzzer/utils/FuzzedDataProvider.h"
 #include "spdlog/spdlog.h"

 namespace Envoy {

@lizan , thanks for the patch.  I applied it to this CL, and clang format works.  Let me know if it should be submitted separately, or if it should be squashed in to this commit.

Thank you! The changes look good to me (thanks for the speed up logs!) besides I think you may have applied my patch instead of Lizan's.

besides I think you may have applied my patch instead of Lizan's.

You are correct.  I altered the change to use Lizan's, as it avoids the external dep.  Also fixed some presubmit issues.


Browser specific delays for tests

I am running into timing issues when doing the screenshots especially with webkit compared to firefox and chrome. The delay parameter somewhat brings relief to this, but when using the delay for a test it would slow down all the tests in all other browsers aswell.
We are testing mobile/tablet/desktop resolutions in chrome/firefox/webkit - so we have 9 different versions of the same test. The delay (e.g. 2000 ms) would be a total delay of 18.000 ms (9 x 2.000ms) for us, which adds up significantly for our testing pipeline.
Would it be possible to delay the test only for specific browsers and introducing a syntax comparable to the skip parameter. E.g.
 parameters: {
    creevey: {
      delay: {
        in: ["webkit.mobile", "webkit.tablet", "webkit.desktop"],
        ms: 2000
      }
    }
  }

Yeah, it's possible to make it happen. I'll try to add this feature in the next week

thats great @wKich - many thanks!