Release Notes

I've added the here:
https://github.com/simlu/voxelshop/releases/tag/1.8.0
Anything missing or that should be changed? @MrTopom @funabab

What should be put up on twitter? This?
#VoxelShop 1.8 live! Download https://blackflux.com/node/11 and release notes https://github.com/simlu/voxelshop/releases/tag/1.8.0 Native installers to come next weekend.

You can use URL Shortener to short this urls (also you could see analytics about clicks on this shortened urls)

Release notes seems good. Closing this.


Prototype/fdb layer final rebase

Overview
This is prototype/fdb-layer rebased onto master. Lets merge it already.
Testing recommendations
make check
Checklist

[x] Code is written and works correctly
[x] Changes are covered by tests
[x] Any new configurable parameters are documented in rel/overlay/etc/default.ini
[ ] A PR for documentation changes has been made in https://github.com/apache/couchdb-documentation

The comparison that's most important is probably against prototype/fdb-layer which you can see here:
https://github.com/apache/couchdb/compare/prototype/fdb-layer...prototype/fdb-layer-final-rebase

Turns out a merge is much cleaner:
https://github.com/apache/couchdb/pull/3140


Please consider using LinkedHashMap as a default Map implementation

Currently the maps default implementations are a mix of HashMap and LinkedHashMap, this is not consistent, especially considering the keys order.
I'm aware that json objects provide no expectation of order.
However, having a somewhat consistent order is not only esthetically pleasing but also practical to :

write deterministic tests (HashMap spec does not preclude iteration order changes across java versions)
version swagger definitions nicely (adding an entry to an HashMap can completely reshuffle the iteration order, which makes diffing a pain)

If there are no outstanding reasons to prefer the HashMap implementation, I would change the remaining HashMaps to LinkedHashMap.

This has been added


[Request-For-External] Table 27 "Item" - Procedure from Local to Global

Hello, we need the following currently internal procedure to be set to external. Can you please check this? Thank you!
  procedure IsVariantMandatory(IsTypeItem: Boolean; ItemNo: Code[20]): Boolean

Already done in v23


Radius of zone loading

Hi,
is it possible to add a variable how many zones are loaded (radius)?
(If you have a better PC you can change it and have a greater view distance, else (maybe number 1) is like now)

Hello,
This system is for handcrafted zones, like for an FPS or 2D game.
What you are asking is for a chunk system like in Minecraft, this is a different system, and out of scope of this plugin.
So no, sorry.

I have thought about your loading system.
Is it possible to change the player area size ingame to change the view distance?

I have test it (I have scaled the ZoneTrigger area). It is working in your demo. With a .scale.x script.
If anyone wants to use diffrent view distances.
Gamemap

You are right, it should work lol. It has not been designed to be used this way, but it should actually work :)

Can you tell me what kind of game you are creating? Do you have zones of different sizes or terrain chunks like Minecraft?

I don't know the theme of my game yet (maybe airplane or open world RPG or both ;) ), but I thought to put related things in one zone (park with all objects, beach with all objects, etc.) So I have different zone sizes.
Later I want to add a multiplayer. Is this compatible with the zone loading?
#8 (Change the current camera or is it possible that both players load and unload zones at the same time?)

Yes, it possible, as the zone loading is mainly a client-side feature.
However you will need to synchronize the zones (objects and NPC) across clients that have it loaded, this part could be tricky.

Ok I will try this later with a server.
Thank you for this great addon!

thank you for using it :)


[Codelab Issue] Testing Codelab 5.#, Step # - Issue description

Describe the problem
A clear and concise description of what the problem is.
I am getting a compiler error when I try to run the code.
In which lesson and step of the codelab can this issue be found?
Lesson number + step number. (ex. 5.2, Step 7)
Task 4, step 1
How to reproduce?
What are the exact steps to reproduce the problem?
Clone code from git and try to run it.
Versions

What version of Android Studio are you using?
4.2.1

Additional information
Add any other context about the problem here.

codelab: advanced-android-kotlin

This has been fixed


Model not training with CUFED Dataset

Hi,
When I tried training the model with the CUFED dataset, the model did not show any significant learning, even after reducing the learning rate by a factor of 10 (from 1e-4 to 1e-5). Is there a recommended setup or any specific configuration needed for training?

Due to the model's complexity, training can be slow using a Swin Transformer. To optimize performance, you can remove the interpolated image added in the final layer. Additionally, tweaking the depth of the attention heads may help improve accuracy if resources allow. These tweaks help balance efficiency and model performance.


Error("invalid type: string "[...]", expected a borrowed string" error when deserializing a struct that's serialized with serde

It seems to only occur on items with \" in it. I find this really weird, because I have a response from a website, in JSON, i then deserialize that into struct A, i then save that struct A to a file, and then later i open that file, and when deserializing that file back into struct A i get the error:  Error("invalid type: string "{Something with \" in it]", expected a borrowed string". struct A is just ```
Struct A {     name: String }

Are you sure that you don't use the &str instead of String of your struct definition?

Here we are a couple years later. Same issue. Same cause. This chain was helpful to me.

My custom Deserialize was doing a match{} on map.next_key() (or something similar) and it implicitly coerced the key to &str instead of String. Doing map.next_key::() was the trick for me. Thanks you guys!


VS Code VARIABLES View does not show DFDL Variables

{Sev: minor}
The VS Code view called VARIABLES does not show DFDL variables. This could be perceived as misleading when working with DFDL files.

Does the Daffodil internal debugger provide this information for external consumption?
If not, is there a way to remove the variables display area from VS Code?

Does the Daffodil internal debugger provide this information for external consumption?

Yes, the debugger has access to ParserOrUnparseState, which has a variableMapForDebugger member that contains information about all the variables (both current and previous instances). The CLI "info variables" debugger command has an example of how to access this state information:
https://github.com/apache/daffodil/blob/main/daffodil-runtime1/src/main/scala/org/apache/daffodil/runtime1/debugger/InteractiveDebugger.scala#L1939-L1989
Note that it only displays the current instance of each variable--an enhancement might be to give the ability to display all instances instead.


Add comment to logic test methods

Codecov Report

Merging #306 (2e32d9d) into master (3d4a9f1) will increase coverage by 0.05%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #306      +/-   ##
============================================
+ Coverage     62.28%   62.33%   +0.05%
  Complexity      480      480
============================================
  Files            88       88
  Lines          1710     1710
  Branches        194      194
============================================
+ Hits           1065     1066       +1
+ Misses          561      560       -1
  Partials         84       84

Impacted Files
Coverage Δ

src/main/java/seedu/trackermon/AppParameters.java
75.00% <ø> (ø)

src/main/java/seedu/trackermon/Main.java
0.00% <ø> (ø)

src/main/java/seedu/trackermon/MainApp.java
0.00% <ø> (ø)

...ain/java/seedu/trackermon/commons/core/Config.java
69.23% <ø> (ø)

...ava/seedu/trackermon/commons/core/GuiSettings.java
80.76% <ø> (ø)

...n/java/seedu/trackermon/commons/core/JarTools.java
40.62% <ø> (ø)

...java/seedu/trackermon/commons/core/LogsCenter.java
75.00% <ø> (ø)

...in/java/seedu/trackermon/commons/core/Version.java
82.50% <ø> (ø)

...ava/seedu/trackermon/commons/core/index/Index.java
100.00% <ø> (ø)

...on/commons/exceptions/DataConversionException.java
100.00% <ø> (ø)

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f856299...2e32d9d. Read the comment docs.


api

Adding the api into the solution. Implemented the get products function.

Tested in PR environment. Looks good.


Meta: Infinite loops repeat lines in response

This meta issue tracks scenarios where chat requests are hitting an infinite loop until the request fails with a length error.
Error message:

Sorry, the response hit the length limit. Please rephrase your prompt.

In case you experience this, try to reduce or refine the code you provided as context; or rephrase the prompt.
We are working to reduce these cases, balancing the tendency of LLMs to hallucinate infinite loops while allowing valid use cases that have look like patterns of repetition.

In case you experience this, try to reduce or refine the code you provided as context; or rephrase the prompt.

This has been often a fix for me, frustrating at times when I keep telling the AI to keep their answers short or only tell me their additions or changes. it sometimes still tries to repeat my whole sourcefile.
Is there a nice prompt i could start with to keep copilot short in reply length, skip the whole repeating compile process?


What license is this code published under?

Hello,
I'm porting this code to ctmx to get familiarwith htmx.
I would like to publish my work after and since it's heavily inspired by this code I decided to ask for your permission.
I will put a link to this project.
And thank you for writing it.
https://github.com/whamtet/ctmx

I generally use MIT


after_install on windows does not work as expected

I have a template which has the npm install command in scripts.after_install. After running tmpo init and entering all the values the shell switched to cmd but no command is executed.

meta.json

{
  "kind": "template",
  "name": "lb4-module",
  "version": "0.1.0",
  "extend": ["lb4-default"],
  "scripts": {
    "before_install": "",
    "after_install": "npm install"
  }
}

should also be fixed in 1.5.2

the after_install script is now executed and running but there seems to be another issue related to copying the folder from the temp folder called .tmpw8PqPB in my case to the acutal location.
The error message is Access is denied. (os error 5)

as another sidenote: I noticed that husky failed to install because it seems like git is initialized after the after_install command is ran

as another sidenote: I noticed that husky failed to install because it seems like git is initialized after the after_install command is ran

Yes the git repository is initialized at the end. But should be no problem to do this before the templates are copied.

the after_install script is now executed and running but there seems to be another issue related to copying the folder from the temp folder called .tmpw8PqPB in my case to the acutal location.
The error message is Access is denied. (os error 5)

Mhhh thats strange on my machine thats working. Maybe you can have a look in the logs under /home/.tmpoe/logs/output.log. Disclaimer loggin is in an early phase xD

the log file does not provide any further details but there is another issue which might be related.
Maybe it has to do with the amount of files that are being copied

Mhh.. strange. I am actually thinking about how usefull the tempdir is. From one perspectiv its not bad to initialze everything in an temp directory and move after but the problem is that the os (linux/windows) dont suppot nativ temp dirs (only tempfile). That means that if the process is interrupted by e.g. a SIGTERM the temp dir is not deleted.

I tested now on two different windows systems with my default typescript template that worked. Can you maybe also have a test with this template.

I finalize a new release 1.5.3.
I add this lines in the init action to avoid workspace path like this /homes/thomas/./.tmp437832/2fa
My system had no problems with this but maybe this led to your problems.
  let dir = if workspace_directory != "." && workspace_directory != "./" {
    current_dir.join(workspace_directory)
  } else {
    current_dir
  };

I tested the default typescript template and it failed as well at first, then I tested the lb4-module template again and it suddenly worked...
I also updated to 1.5.3 and it worked again with the lb4-module template, I will let you know if I get the issue again and btw great updater 💯


In loop mode, duplicate slides are not being updated when the contents of the original slides change

This is a (multiple allowed):

[x] bug

[ ] enhancement

[ ] feature-discussion (RFC)

Swiper Version: 4.2.6

Platform/Target and Browser Versions: All browsers, but tested on macOS, CHROME

Live Link or JSFiddle/Codepen or website with issue: https://jsfiddle.net/knankv59/2/

What you did
Swiper config is at the bottom of the Javascript window in the jsfiddle (loop mode, loopedSlides set to 3, slidesPerView set to 1)
Click the right navigation arrow twice to swipe forward to slide 2.
Click the button above the slideshow to change the content of slide 2, which now reads "New content!"
Click the right navigation arrow six more times, going to the end of the slides and looping back around to slide 2.
Expected Behavior
Slide 2's text should still say "New content!"
Actual Behavior
Slide 2's text is still just the original "2".
_
Is there any way to refresh specific duplicate slides automatically when their original slide's content changes?

Same issue here. Swiper does not update duplicate slides when the original has changed. Any workarounds would be highly appreciated.

@jhowsick  One way is manually redoing the loop by calling methods on the swiper instance itself.
const swiper = this.swiper;
swiper.loopDestroy();
swiper.loopCreate();

This is hacky though. Hopefully an official fix will happen.

I had the same problem

me too

It is not supported and won't be, it should be done manually, for example like @brandonce recommended

How about  something similar with renderExternal() ?..  Sync React or Vue with loop mode is so painful.

I currently use the method that just update html of dupslides
updateDuplicateSlides = function() {   var swiper = this;   var $wrapperEl = swiper.$wrapperEl;   var params = swiper.params;   var slides = swiper.slides;   $wrapperEl.children(("." + (params.slideClass) + "." + (params.slideDuplicateClass)))   .each(function() {     var idx = this.getAttribute('data-swiper-slide-index');     this.innerHTML = $wrapperEl.children('.'+params.slideClass+'[data-swiper-slide-index="'+idx+'"]:not(.'+params.slideDuplicateClass+')').html(); })

Our swiper, here to note the onTransitionStart function of passing our function to trade the duplicates.
new Swiper( '.swiper-container', {
    slidesPerView: 1.25,
    spaceBetween: 15,
    centeredSlides: true,
    parallax: true,
    onTransitionStart: handleDuplicates.bind()
} )

Your slides with events we set:
let videos: [].slice.call( document.querySelectorAll( '.swiper-slide .videobox video' ) );

videos.forEach( function ( item, index ) {
    item.addEventListener( 'touchstart', playVideo.bind( this ) );
}.bind( this ) );

We handle the duplicates on transition start with this function handleDuplicates()
/**
 * Update, change or do what you want here with the duplicates
 */
handleDuplicates() {

    let swiperDuplicatesWithoutEvents = [].slice.call( document.querySelectorAll( '.swiper-slide-duplicate .dssVideobox video' ) );

    // Remove olds
    swiperDuplicatesWithoutEvents.forEach( function ( item, index ) {
        item.removeEventListener( 'touchstart', playVideo.bind( this ) );
    }.bind( this ) );

    // Add news
    swiperDuplicatesWithoutEvents.forEach( function ( item, index ) {
        item.addEventListener( 'touchstart', playVideo.bind( this ) );
    }.bind( this ) );
}

I was struggling with this behavior as well and since I could not find any solution out of the box for my problem I ended up with a mix of hacks that worked perfectly to me. In my case some slides have a counter and also some buttons, both were not working on duplicated slides. So I used the solution proposed by @golee to copy the dom at the moment of slide transition start (slideChangeTransitionStart) and then on (slideChangeTransitionEnd) I slide it to the realIndex but with speed 0 so it's not noticed by the user.
slideChangeTransitionStart() {
            let $wrapperEl = this.swiper.$wrapperEl;
            let params = this.swiper.params;
            $wrapperEl.children(('.' + (params.slideClass) + '.' + (params.slideDuplicateClass)))
                .each(function () {
                    let idx = this.getAttribute('data-swiper-slide-index');
                    this.innerHTML = $wrapperEl.children('.' + params.slideClass + '[data-swiper-slide-index="' + idx + '"]:not(.' + params.slideDuplicateClass + ')').html();
                });
    }

slideChangeTransitionEnd() {
            this.swiper.slideToLoop(this.swiper.realIndex, 0, false);
    }

@nolimits4web I follow @brandonce described, then found if I have 2 slides in swiper,  I only can change the first slide value and can't change the second one. swiper.loopDestroy() will rollback the value of second slide.

//this will update the duplicates only once if you slides update
this.mySwiper.once('transitionStart',function updateDuplicates() {
this.loopDestroy()
this.loopCreate()
})

//this will update the duplicates only once if you slides update
this.mySwiper.once('transitionStart',function updateDuplicates() {
this.loopDestroy()
this.loopCreate()
})

Hello, I'm having the same problem and can't fix it. Is possible for you to give me a real example, probably i'm pasting your code in a wrong place.
Thanks a lot.

@jhowsick One way is manually redoing the loop by calling methods on the swiper instance itself.
const swiper = this.swiper;
swiper.loopDestroy();
swiper.loopCreate();

This is hacky though. Hopefully an official fix will happen.

Hello..
Can you please tell me were should i put the code?
Thanks a lot

const swiper = this.swiper;
swiper.loopDestroy();
swiper.loopCreate();

@JoseCigarro you should run these methods whenever the original slides change.
this.swiper points at your Swiper instance. Here is an example of how you can access it through Swiper events or swiper's HTMLElement - http://idangero.us/swiper/api/#initialize

I was struggling with this behavior as well and since I could not find any solution out of the box for my problem I ended up with a mix of hacks that worked perfectly to me. In my case some slides have a counter and also some buttons, both were not working on duplicated slides. So I used the solution proposed by @golee to copy the dom at the moment of slide transition start (slideChangeTransitionStart) and then on (slideChangeTransitionEnd) I slide it to the realIndex but with speed 0 so it's not noticed by the user.
slideChangeTransitionStart() {
            let $wrapperEl = this.swiper.$wrapperEl;
            let params = this.swiper.params;
            $wrapperEl.children(('.' + (params.slideClass) + '.' + (params.slideDuplicateClass)))
                .each(function () {
                    let idx = this.getAttribute('data-swiper-slide-index');
                    this.innerHTML = $wrapperEl.children('.' + params.slideClass + '[data-swiper-slide-index="' + idx + '"]:not(.' + params.slideDuplicateClass + ')').html();
                });
    }

slideChangeTransitionEnd() {
            this.swiper.slideToLoop(this.swiper.realIndex, 0, false);
    }

This one is super and awesome fix. You saved my day. Thank you very much.

I use Swiper in Vue with a loop and need change slides data. Of course, the loop doesn't change duplicates data so I need to use this.loopDestroy () and this.loopCreate () but ... when to run them? I came to the point that I have to run them after creating DOM Swiper but how to detect this? Because I use Vue I can fire it on "update", but without it, I don't have any good idea. Of course, I can use interval and check something in DOM but it sucks in my opinion.

hmm this fix worked for me
before adding a slider destroy the loop
swiper.loopDestroy();
Add the slider setTimeout for a sec  and update the swiper  and create the loop again
setTimeout({
directiveRef.update or swiper.update()
swiper.loopCreate();}, 1000);
the loop doesnt work properly on slide update so manually updating it!  hope it works for you as well!

Thank you so much @victorcamargos for the perfect solution! Here is how I used it on my code if anyone needs it:
let productOptions = {
    slidesPerView: 1,
    autoHeight: true,
    loop: true,

    on: {

        slideChangeTransitionStart: function(swiper) {
            let $wrapperEl = swiper.$wrapperEl;
            let params = swiper.params;
            $wrapperEl.children(('.' + (params.slideClass) + '.' + (params.slideDuplicateClass)))
                .each(function() {
                    let idx = this.getAttribute('data-swiper-slide-index');
                    this.innerHTML = $wrapperEl.children('.' + params.slideClass + '[data-swiper-slide-index="' + idx + '"]:not(.' + params.slideDuplicateClass + ')').html();
                });
        },

        slideChangeTransitionEnd: function(swiper) {
            swiper.slideToLoop(swiper.realIndex, 0, false);
        }

    }

}

const product = new Swiper(".swiper-image-container", productOptions);

I am using Vue, not sure if this helps to someone but in my case I just destroyed the slider and after 100 milliseconds initialized it again:
this.swiper.destroy()
_.delay(() => {
        this.swiper = new Swiper(this.$refs.swiper, this.swiperOptions)
 }, 100)

setTimeout(()=>{
            that.swiper.loopDestroy();
            that.swiper.loopCreate();
            console.log('reborn!')
          })

this is just alright and clean !

setTimeout(()=>{
            that.swiper.loopDestroy();
            that.swiper.loopCreate();
            console.log('reborn!')
          })

this is just alright and clean !

tahnks a lot !!!  yesterday i saw it at 3:00 am, saved me~

no solution works for me
setTimeout(()=>{
that.swiper.loopDestroy();
that.swiper.loopCreate();
console.log('reborn!')
})
      this make the same behaviour come back again

Thank you so much @victorcamargos for the perfect solution! Here is how I used it on my code if anyone needs it:
let productOptions = {
    slidesPerView: 1,
    autoHeight: true,
    loop: true,

    on: {

        slideChangeTransitionStart: function(swiper) {
            let $wrapperEl = swiper.$wrapperEl;
            let params = swiper.params;
            $wrapperEl.children(('.' + (params.slideClass) + '.' + (params.slideDuplicateClass)))
                .each(function() {
                    let idx = this.getAttribute('data-swiper-slide-index');
                    this.innerHTML = $wrapperEl.children('.' + params.slideClass + '[data-swiper-slide-index="' + idx + '"]:not(.' + params.slideDuplicateClass + ')').html();
                });
        },

        slideChangeTransitionEnd: function(swiper) {
            swiper.slideToLoop(swiper.realIndex, 0, false);
        }

    }

}

const product = new Swiper(".swiper-image-container", productOptions);

thank you! it works!

I just use loopFix to fixed this problem.
setTimeout(() => {         swiperInstance.loopFix();       });
dont need loopDestroyandloopCreate`
maybe this can help u guys.


Update for Swift 4.1

Can you please update this for Swift 4.1?

Please rebuild the library, after upgrading Swift.

@keygx
I've tried cleaning and rebuilding multiple times with the same issue.
BTW: I'm not using CocoaPods, but rather dragging the source files and framework into my project.
There was no issue with this before the migration from Swift 4.0 -> 4.1

@seenoevo
Please set config.build_settings ['SWIFT_VERSION'] to 4.1
--Podfile
config.build_settings['SWIFT_VERSION'] = '4.1'

@seenoevo
I'm sorry. I was mistaken.
If you have successfully installed GradientCircularProgress.framework in your project, the error disappears when you rebuild the project.


convert ghostdriver cookie tests to python

should_be_able_to_create_cookie_via_javascript and should_allow_to_delete_cookies_even_if_not_set refactored so they do not depend upon an external website

Merged, thank you.

(FYI, we don't use the "merge" button because we want to keep the history as linear as practical.)

cool.  i'll work on some of the other tests when i get the time.

Much appreciated.


Intro to Hyrax

Description needed

About the workshop:
In this workshop we will walk through a local installation of Hyrax 2.x, generate a work type, and get a local test suite running. This will be an abbreviated version of a module usually run at Samvera Camp. Because of the limited time, this will mostly be a demo with audience participation. However, we will offer some support if you want to bring your laptop and follow along.
About Bess Sadler, Senior Developer, Data Curation Experts:
Bess has been building digital repositories for over fifteen years. She is a co-founder of both the Blacklight and Samvera software projects, and is a passionate advocate for open source software and DevOps culture.


[swift-3.1-branch] Fix + and - operations on DispatchTime and DispatchWallTime

Explanation: Fixing an incorrect behavior for non-finite values of Double paramter
Scope of Issue: Enables more safe ways to use the Dispatch overlay
Origination: The original code contained this bug
Risk: minimal
Reviewed By: Kim Topley (@ktopley-apple )
Testing: Existing test suite + an extra test
Directions for QA: N/A

rdar://problem/30276292

@swift-ci Please test

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 4a37368643a7a7e27ca15438e9c5b7d20b7dc8bc
Test requested by - @moiseev

Git connection failure.
@swift-ci Please test Linux platform


Feature/datamodel

Metanorma PR checklist

[ ] Breaking changes (list related PRs)
[ ] Documentation update required (create task for this)
[ ] External dependency introduced (documentation update need)
[ ] Gem with native library introduced

Merging first, but this blog post needs some clean up because we now have LutaML for data modelling.


BitLocker-protected volumes

Under "Supported drives or volumes for backup", BitLocker-protected volumes are mentioned.
Do you more specifically mean that BitLocker-protected volumes which are LOCKED are not supported?
That's what the Details seem to imply.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b566a3bb-80d9-151e-cc27-7bc94d48f363
Version Independent ID: 0f8ca0eb-0153-13b6-5929-0e5996eb418e
Content: Support matrix for the Microsoft Azure Recovery Services (MARS) agent
Content Source: articles/backup/backup-support-matrix-mars-agent.md
Service: backup
GitHub Login: @dcurwin
Microsoft Alias: dacurwin

#assign:saurabhsensharma

Thanks for your comment. The feedback has been shared with the content owner for further review. Thanks for your patience.

absolutely right! Bitlocker protected volumes that are in locked-state during backup are not supported. We will update documentation to make this more clear. Thanks for the feedback.

#please-close


PR: Improved dependency dialog

Implements #5652
Following are screenshots of the new dialog.
Regular Spyder session:

Test case with some errors:

Hello @timhoffm! Thanks for submitting the PR.

In the file spyder/widgets/dependencies.py, following are the PEP8 issues :

Line 76:80: E501 line too long (87 > 79 characters)

I really like it too!! Thanks for this @timhoffm!
I only have one suggestion: the text about optional dependencies should not be in a smaller font size, but in the same size as the text in the dialog.

@ccordoba12 The font size issue is already present in the current solution. It happens with a 125% scaling.

It looks great @timhoffm, I don't mind that much the current font size.

@rlaverde Basically this pull request is just another way of visualizing the same data. The current test_dependencies() does only test that a dialog exists.
I don't have much experience in UI testing. Could you give me a hint what to test and how to do that?

I think something simple, adding an optional dependency to the existing test_dependencies() is enough:
dependencies.add("sympy", "Symbolic Mathematics", ">=10.0", optional=True)

The font size issue is already present in the current solution

Ok, didn't remember that! Let's leave things as they are then.

I did the change that I suggested, so this is ready to be merged

@ccordoba12 merge?

Yep. Thanks a lot @timhoffm! This is a great improvement!!


Add typescript server

Hello!
So, in a new TypescriptServer folder,  I converted the server code into typescript, and merged all of the secondary files (constructors/, custom/ and static/) into a single networking file.
So, uh, tell me what you think of it I guess

Yup, now that you mention it, I don't really know why I put it to string, it would be much better if it was any


[e2e] - increase coverage on homepage (indexui)

Current Behavior
Currently, Meshery's e2e tests for the homepage only check that the Left navigation panel is visible. Let's add some more visibility checks for this panel
Test Steps

Verify these menu items are visible on landing on the homepage:
- [ ]  Dashboard
- [ ] Lifecycle
- [ ] Configuration
- [ ] Performance
- [ ] Extensions

Implementation
The test cases will be made in this file
Acceptance Tests
Increase e2e test coverage on the left navigation panel
Mockups

Contributor Guides and Handbook

🛠 Contributing to Meshery's End-to-End Tests
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

assign this issue to me. I'd love to work on it!

@sahilshityalkar go for it! :)
I'm unable to assign, but go ahead and create a PR and then link it to this issue.

Hey @vishalvivekm ! Can i work on this issue?

@sahilshityalkar   do you have updates on this task that you'd like to share in today's Meshery Dev call? Is it something you're still working on?
Please let us know if you're not, we can try to hand it over to @THEBOSS0369

Hey @vishalvivekm ! shall i go ahead?

Thankyou @sahilshityalkar for raising your hand, considering no response in past 3 weeks we are reassigning this issue to @THEBOSS0369
In case if you are still interested, you are very welcome to participate with reviewing PR on this issue or taking up new one

Hey @AmTryingMyBest @vishalvivekm @Yashsharma1911 I have created PR for this issue here #12708 . Please review it whenever you get time. thanks!

@stalebot PR's has been created already


Implement unionTypes(a, b *types.Type) *types.Type

This is a variant of the algorithm we have now for simplifying types. And when we add optional struct fields, it will become the algorithm for simplifying types.
The difference between union and simplify is that when simplify encounters two structs and a field is present in only one, it drops it. Union must keep it.

@aboodman: Given @rafael-atticlabs's suggestion that this ultimately replace makeSimplifiedUnion, do you still have reservations about implementing this in noms rather than attic?

I think it might be easier to iterate at first with this in attic closer to
the calling code. But not a huge deal.
On Tue, Feb 21, 2017 at 5:38 PM Eric Halpern<EMAIL_ADDRESS>wrote:

@aboodman https://github.com/aboodman: Given @rafael-atticlabs
https://github.com/rafael-atticlabs's suggestion that this ultimately
replace makeSimplifiedUnion, do you still have reservations about
implementing this in noms rather than attic?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/attic-labs/noms/issues/3204#issuecomment-281540579,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAE6BFxm2iWGLzfN8ZRW6ifnDsMKnlDRks5re5GxgaJpZM4MGv3z
.


tiledlayout not supported

The new tiledlayout option (instead of subplots) is not yet supported, see https://de.mathworks.com/help/matlab/ref/tiledlayout.html.

Please see README.md

Currently unmaintained


Async record

Hi,
I have started using the record in Kapacitor and it looks great, but waiting for the duration only to get the record id isn't nice..
For example, consider that I want to wait for 50 minutes, I would like to do something like this:
> kapacitor record stream -name cool_alert -duration 50m
{some record id}
> kapacitor record status {record id}
Recording

.. after 50 minutes ..
> kapacitor record status {record id}
Finished!

What do you think?

What are you going to do with the ID before the recording has finished? I agree that having a long running task can be difficult to manage but the current workflow prevents invalid behavior, of trying to use a partially completed ID. Are you trying to programatically interact with recordings?

I am trying to interact programatically, I want to create recording of 1 hour and setting request timeout limit to 1h isn't acceptable.
What I am suggesting is:

Create recording returns id
I will check the status of this id and there will be two stats: recording or finished

If this is still recording, I will try again later (5 minutes later for example)

If this is finished I will go to /api/download_recording/{recording id} for example

@yosiat The underlying API that the kapacitor cli tool uses does what you want. At the moment the HTTP API is undocumented because it is still changing. Depending on your requirements you could use the HTTP API directly for now, although you may have to update your code if the API changes. If you want to see how it works look at this code snippet for how to perform a recording and check on its state later. https://github.com/influxdata/kapacitor/blob/master/cmd/kapacitor/main.go#L268-L325

I checked the api and it doesn't work asynchronously,
I sent the next API call:
POST http://localhost:9092/api/v1/record?type=stream&name=cpu_alert&duration=120m

And I waited for 2 minutes and didn't got no response, I expected to get response immediately.

Looks like you are using an older version. Please upgrade to the newest version 0.10.1 and you should see the behavior you expect.

Thanks :+1:


Fake PR for courses

So it looks like actions/labeler always pulls the file down via the API rather than using the local file. This probably won't be viable.


fix appleboy/gofight/v2 dependency

Hello!
Recently i've started to receive following error on go build in packages that include ginprom
go<EMAIL_ADDRESS>go.mod has post-v2 module path "github.com/appleboy/gofight/v2" at revision v2.0.0
go: error loading module requirements

adding /v2 to github.com/appleboy/gofight @ go.mod & prom_test.go fixes this issue

Codecov Report

Merging #7 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master     #7   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           2      2
  Lines         131    131
=====================================
  Hits          131    131

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d43ab6...8be321f. Read the comment docs.

Hey!
Thanks a lot for your PR!
I'll draft a release ASAP with your changes.

v1.1.2 is released, thanks a lot :)

thanks!


win-1 is dead.

Type of Request
bug
Infrastructure Environment
LUCI, Github, Cocoon scheduler, Autosubmit, etc...
What is happening?
win-1 is dead.

Steps to reproduce
Step 1:
Step 2:
..
Step n:
Expected results
I expect to see X when Y is finished.

win-1 is out of sync with Salt master, run command "salt-call state.apply" to re-sync with Salt master again.


Screen recording not blocked in Android 11

So I started using this package around last week, and it works fine: screenshot is blocked, and the screen turned black when screen recording. However, today I tried it again and my app's screen no longer turns black when being recorded. The only difference I had in my phone is that it was updated to Android 11, even though I'm not sure if that's the case. Anyone else have the same issue? The device I use is Samsung Galaxy S10.

Can you confirm the Android version? I will see if I can reproduce with a system image using the same version.


Weight updates for branch heads

Hi, I noticed that for every training sample the network outputs predictions for all 5 output branches, but the loss is then (correctly) calculated using the output from the branch that corresponds to that sample's high-level command and summing those losses for all samples in the batch to get the total_loss tensor. Is this total loss value then used to update all 5 branches? Or is an individual loss for each branch calculated somewhere only using the samples that they are supposed to predict on given the high-level command?
Hopefully the question is clear, I can try to rephrase if it isn't!
Thanks a lot for this repo it has been very useful!

Hi, I noticed that for every training sample the network outputs predictions for all 5 output branches, but the loss is then (correctly) calculated using the output from the branch that corresponds to that sample's high-level command and summing those losses for all samples in the batch to get the total_loss tensor.

You are right. The loss is accumulated adding the loss of separate branches exactly as you describe it.

Is this total loss value then used to update all 5 branches?

Through the gradients (derivate of loss with respect to the data) the weight will influences the branches that were present in the training batch.

So if for example we have a batch of 20 images, 10 of them are Right and 10 of them are Straight, does this mean the Left and Follow branch heads are not updated at all for this batch? This would make sense to me but it doesn't seem reflected in the code, since only 1 total loss value is calculated and the entire network is trained based on this value.

So if for example we have a batch of 20 images, 10 of them are Right and 10 of them are Straight, does this mean the Left and Follow branch heads are not updated at all for this batch?

I think this is correct. In my opinion the code https://github.com/merantix/imitation-learning/blob/master/imitation/models/conditional_il_model.py#L53-L81 does that.
You could make an experiment: Train one batch only with data of just one condition, and see what happens to other conditions heads.


credit imput as this app relies on the cobalt API

this app relies almost entirely on the cobalt.tools api, and this is never mentioned. You should credit imput for the cobalt API

Thank you for pointing that out! I’ll make sure to add proper attribution in the documentation.


Snake-case version of your library

A few weeks ago I found your library and I realy like it and I want to use in my project. But I have the camelCase itch which means I don't like camelCase at all.
So I've forked your library to use it with PySide6 snake-case feature. Here's the link: https://github.com/uselessvevo/snakeribbon
And I really hope you don't mind that I forked it so hard.
Yeah.

Also, I don't want to use QtPy, and I've made some tweaks to use different icons and themes.


Uio syllabification

[x] This pull request has been linted by running npm run lint without errors
[x] This pull request has been tested by running npm run start and reviewing affected routes
[x] This pull request has been built by running npm run build without errors
[x] This isn't a duplicate of an existing pull request

Description
Implemented UIO Syllables Panel.

Steps to test

Go to deploy preview
Click on UIO tab in top right corner of webpage
Turn on syllables panel

Expected behavior:
Syllabification should be activated.
Additional information

N/A
Related issues

closes #246

@TeddTech, your "add-uio" pull request has been merged into the dev branch. Thanks for the excellent work. Could you update this and all other dependent pull requests by merging in the dev branch? Thanks.


added config copy only icons

Summary

Enhancement of the Configuration File (blade-fontawesome.php)
The configuration file has been refined with the addition of an empty icons array to each key. This helps to maintain better structure and clarity in the file.

Major upgrade to the SyncIconsCommand.php File
In a bid to improve synchronization of icons:

A new option (--only) has been introduced. This option enables users to sync only the icons they need from the npm package, instead of a full package sync, fostering time and resource optimization.
The sync process has been streamlined with the removal of a key named custom from the $family array, which makes the array less complicated.
The functionality to delete existing icon directories within the destination path has been added - this ensures that outdated versions of the same icons get replaced with their newest versions.
The addition of logic to create icon directories in the destination path has been made only if there are necessary icons to copy. This prevents the creation of empty directories and saves space.
Now each icon file is copied neatly from the source path to its intended destination. This allows for smoother and more accurate icon files management.
An added layer of fault tolerance with the inclusion of a try-catch block during the sync process. This ensures that expected and unexpected exceptions that may occur during the process are handled gracefully, thereby preventing abrupt termination of the process.

Problem
I am using steam laravel to deploy changes to aws through its steam cli, and this in turn compiles and compresses the project for deployment, what happens is that the project is too heavy due to the blade-fontawesome package copies all the svgs even without these are used

My solution
Therefore, I updated the sync of the package so that it only copies the files that I am using in the template
Other posible solution
when deploying list all blade icons in the template and then purge them

You could add an option to include all icons of a set in the config, like: 'icons' => '*',
In the sync command you could add:
      // skip if icons are not specified
      if (! isset($item['icons'])) {
          continue;
      }

      // load all files if * is specified
      if ($item['icons'] === '*') {
          $item['icons'] = collect(scandir($assetSourcePath))
              ->filter(fn ($file) => $file !== '.'
                   && $file !== '..'
                   && str_ends_with($file, '.svg')
              )
              ->map(fn ($file) => str_replace('.svg', '', $file))
              ->toArray();
      }


feat: Allow clearKey configuration in base64 or hex

One thing you should still change is the comments on the clearKeys config, to indicate that either format is accepted.

Done!


Alanren/extension install

use opener service instead of window.open. also found out azure account feature is also using window.open. updating it as well. tested both feature are working fine now.

#6917


getPluralIndex return value bug fixed

getPluralIndex should return index, not boolean

In theory, the plural function should return an integer, and not always is 0 or 1, but in some languages there's more plurals. Are you using a correct plural funcion?

更新 %s 条: ["%s item updated", "%s items updated"]
This is the plural translation of one item in json file, it's no working when i call n__(original, plural, value)

What I want to see is the plural function that returns a boolean instead an integer, or in other words, the content of plurals[domain].code

When plural form is
Plural-Forms: nplurals=2; plural=n != 1;
The code will be
return n != 1;
The doc from GNU gettext

Two forms, singular used for one only
This is the form used in most existing programs since it is what English is using. A header entry would look like this:

Plural-Forms: nplurals=2; plural=n != 1;
(Note: this uses the feature of C expressions that boolean expressions have to value zero or one.)

Ok, understood.
Thank you! 👍


MultiMeshInstance / Gridmap not working properly on some Android devices (Exynos CPU, Samsung S22 Series)

Godot version
3.5.2
System information
Android 13, GLES2, SM-S901B
Issue description
MultiMeshInstance and implicitly Gridmap (which I think uses multimeshinstance behind the scenes) do not work properly on some Android devices.
More specifically regardless of how many items are being placed only 1-2 end up being rendered / shown.
This seems to happen on all Samsung S22 series with Exynos soc:
"SM-S901B", "SM-S901B/DS", "SM-S908B", "SM-S908B/DS", "SM-S906B", "SM-S906B/DS"

Steps to reproduce

Create GLES2 3D scene
Add a Gridmap or multi mesh items with some items
Try on any Samsung S22 series phone (plus, ultra, etc) with the Exynos soc.

Minimal reproduction project
testGodot.zip

Please note that CPUParticles are also broken as reported here on issue #63261. So there seems to be multiple issues at hand here.

Might also be a shader failing to compile due to use of unsupported gl_FragDepth.

Might also be a shader failing to compile due to use of unsupported gl_FragDepth.

It can be reproduced by using just a basic spatial material with an albedo color set.

Thanks for the report! Consolidating in https://github.com/godotengine/godot/issues/63261, as this is due to the same cause.


Bug: Unable to connect to console

Haven't done anything with the app or console since last night when it worked

Same is happening to me. Made no changes.

Restart your xbox. The console sometimes loses connection to the xbox network and fails to connect because of this.


cannot read property 'download' of undefined

var RNFileDownload = NativeModules.RNFileDownload;
var promisify = require("es6-promisify")
let progressEventName = 'RNFileDownloadProgress'
let progressEventTable = {}
var _download = promisify(RNFileDownload.download)
var _error = (err) => {
throw err
}
in index.ios.js file of package line number 12
I am using
"react": "16.0.0-alpha.12",
"react-native": "0.47.2",
xcode-8.3.2

I think it's because this package doesn't work with rn>0.39 on iOS. But also note that this library is marked as DEPRECATED. So I'm not going to fix this and please use other libraries for your use case, like react-native-fs


net/http: readLoopPeekFailLocked: <nil> from client when server sends response immediately

What version of Go are you using (go version)?

$ go version
go version go1.12.1 linux/amd64

Does this issue reproduce with the latest release?
Yep. 1.14.1
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/go"
GOPROXY=""
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="/usr/bin/gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build705707104=/tmp/go-build -gno-record-gcc-switches"

What did you do?

100% reproducible on Linux x64, and play.golang.org. The behavior is unpredictable and the code (see below) to reproduce should be executed few times to got the error and the log.
The error is:
Get http://<IP_ADDRESS>:9101/path: readLoopPeekFailLocked: <nil>

The log is:
transport.go:1829: Unsolicited response received on idle HTTP channel starting with "HTTP/1.1 502 Bad Gateway\r\n\r\n"; err=<nil>

What happens?
HTTP server sends response without reading request and without a delay. This approach is good for proxy servers where a client determined by his IP (i.e. using white-list). Thus, we haven't to read request, if the user is not white-listed.
Reproducing
(try few times to got error)
play.golang.org
package main

import (
	"fmt"
	"log"
	"net"
	"net/http"
	"net/url"
	"sync"
	"time"
)

func mustParseURL(uri string) *url.URL {
	x, err := url.Parse(uri)
	if err != nil {
		panic(err)
	}
	return x
}

func main() {

	l, err := net.Listen("tcp", "<IP_ADDRESS>:9100")
	if err != nil {
		log.Fatal(err)
	}
	go func(l net.Listener) {
		for {
			c, err := l.Accept()
			if err != nil {
				log.Fatal(err)
			}
			go func(c net.Conn) {
				defer c.Close()
				fmt.Fprint(c, "HTTP/1.1 502 Bad Gateway\r\n\r\n")
			}(c)
		}
	}(l)
	time.Sleep(100 * time.Millisecond)

	var cl http.Client
	cl.Transport = &http.Transport{
		Proxy:           http.ProxyURL(mustParseURL("http://<IP_ADDRESS>:9100/")),
		MaxConnsPerHost: 100,
	}

	const N = 100

	var wg sync.WaitGroup
	wg.Add(N)
	for i := 0; i < N; i++ {
		go func() {
			defer wg.Done()
			resp, err := cl.Get("http://<IP_ADDRESS>:9101/path")
			if err != nil {
				log.Print("error:", err)
				return
			}
			if resp.StatusCode != 502 {
				log.Println("status:", resp.StatusCode)
			}
			resp.Body.Close()
		}()
	}

	wg.Wait()

}

Also, playing with the N constant I've got this error for N=10, N=100 and N=1000. For N=1000 this error occurs every time. For N=10 rarely. Seems, it depends on goroutines scheduling.
What did you expect to see?
Got this error every time or never.
What did you see instead?
It's unpredictable. That makes testing a bit harder.
Also
Probably related issues #15253, #15446

CC @bradfitz

Go version
go version go1.13 linux/amd64
Behavior
Running the following code with go test -race fails most of the times, but not always.
package race

import (
	"bytes"
	"context"
	"net"
	h "net/http"
	ht "net/http/httptest"
	"testing"
	"time"
)

func TestDataRace(t *testing.T) {
	d := &dialer{content: "HTTP/1.1 502 Bad Gateway\r\n\r\n"}
	tr := &h.Transport{
		DialContext: d.dial,
	}
	req := ht.NewRequest(h.MethodGet, "http://example.com", nil)
	resp, e := tr.RoundTrip(req)
	if e != nil {
		t.Error(e)
	} else if resp.StatusCode != h.StatusBadGateway {
		t.Error("status:", resp.Status)
	}
}

type dialer struct {
	content string
}

func (d *dialer) dial(ctx context.Context, network,
	addr string) (n net.Conn, e error) {
	n = newMockConn(d.content)
	return
}

type mockConn struct {
	read  *bytes.Buffer
	write *bytes.Buffer
}

func newMockConn(content string) (m *mockConn) {
	m = &mockConn{
		read:  bytes.NewBufferString(content),
		write: new(bytes.Buffer),
	}
	return
}

func (m *mockConn) Read(p []byte) (n int, e error) {
	n, e = m.read.Read(p)
	return
}

func (m *mockConn) Write(p []byte) (n int, e error) {
	n, e = m.write.Write(p)
	return
}

func (m *mockConn) Close() (e error) { return }

func (m *mockConn) LocalAddr() (a net.Addr) { return }

func (m *mockConn) RemoteAddr() (a net.Addr) { return }

func (m *mockConn) SetDeadline(t time.Time) (e error) { return }

func (m *mockConn) SetReadDeadline(t time.Time) (e error) {
	return
}

func (m *mockConn) SetWriteDeadline(t time.Time) (e error) {
	return
}

Output when failing
2019/10/11 17:19:50 Unsolicited response received on idle HTTP channel starting with "HTTP/1.1 502 Bad Gateway\r\n\r\n"; err=<nil>
--- FAIL: TestDataRace (0.00s)
    race_test.go:21: readLoopPeekFailLocked: <nil>
FAIL
exit status 1

Defining in the previous code mockConn, newMockConn, mockConn.Read and mockConn.Write the following way, the error never appeared so far, with go test or go test -race:
type mockConn struct {
	read    *bytes.Buffer
	write   *bytes.Buffer
	writeOk chan bool
}

func newMockConn(content string) (m *mockConn) {
	m = &mockConn{
		read:    bytes.NewBufferString(content),
		write:   new(bytes.Buffer),
		writeOk: make(chan bool, 0),
	}
	return
}

func (m *mockConn) Read(p []byte) (n int, e error) {
	<-m.writeOk
	n, e = m.read.Read(p)
	return
}

func (m *mockConn) Write(p []byte) (n int, e error) {
	n, e = m.write.Write(p)
	m.writeOk <- true
	return
}

Maybe related https://github.com/golang/go/issues/34066

@logrusorgru seeing this happen as well. Can you reproduce it on go 1.11 as well or just 1.12++ ?

@cyx, note that per the release policy, neither Go 1.11 nor 1.12 is supported at this point.
(@sebglon's report of a repro using Go 1.13 is helpful, though.)

@cyx, I just have checked out go version go1.11.13 linux/amd64.  I can say that this is reproducible in the entire go version go1.11.13 linux/amd64 - go version go1.14.1 linux/amd64 range.
I don't think this issue is a real problem, but anyone is working with HTTP "manually" may run into it.
I just use (let's say it's app.go)
var isTesting bool

// [...]

// handler function
    if isTesting {
       time.Sleep(100*time.Microsecond)
    }

(let's say it's app_test.go)
var _ = func() struct{} {
    isTesting = true
    return struct{}{}
}()

To get my tests 100% stable.
Because of the 100µs, I think it's about goroutines scheduling.

Thanks! That makes sense. I'm seeing this in production with the clients
primarily being node http clients -- although the interesting thing is that
I only see it on higher throughput environments

I can report that we're seeing this issue in production at Let's Encrypt, using https://github.com/letsencrypt/boulder/. Specifically, there are some "hasty" HTTP servers out there that will send an HTTP response as soon as the TCP connection completes. When we make multiple concurrent requests to such a server, we get Unsolicited response in our stdout logs.
The reason we'd like to see this fixed: In general we try to route all log output through our logging system, and this is the one place in net/http that regularly prints to stdout. I have a minimized repro case below that is a bit shorter than ones posted to this thread. Reproduces 100% on Go 1.14.2 (latest as of 2020-04-23).
package main

import (
	"log"
	"net"
	"net/http"
	"sync"
	"time"
)

const port = ":8081"

var wg sync.WaitGroup

func main() {
	go hastyHTTPServer()
	time.Sleep(100 * time.Millisecond)
	for i := 0; i < 2; i++ {
		wg.Add(1)
		go fetchLoop()
	}
	wg.Wait()
}

func fetchLoop() {
	for i := 0; i < 1e3; i++ {
		resp, err := http.Get("http://localhost" + port)
		if err != nil {
			log.Println(err)
			continue
		}
		resp.Body.Close()
	}
	wg.Done()
}

// hastyHTTPServer runs an HTTP server that sends a response and closes the
// connection as soon as a client makes a TCP connection, rather than waiting
// for the client to actually make an HTTP request.
// A real-world example of this type of server can be seen at 8849777.com
func hastyHTTPServer() {
	ln, err := net.Listen("tcp", port)
	if err != nil {
		log.Fatal(err)
	}
	for {
		conn, err := ln.Accept()
		if err != nil {
			log.Fatal(err)
		}
		go func() {
			conn.Write([]byte("HTTP/1.0 200 OK\r\n\r\nQuick response!"))
			conn.Close()
		}()
	}
}

The output is like below (the number of errors varies from run to run, but there's almost always at least one error):
$ go run hastyserver.go
2020/04/23 16:26:37 Unsolicited response received on idle HTTP channel starting with "HTTP/1.0 200 OK\r\n\r\nQuick response!"; err=<nil>
2020/04/23 16:26:37 Get "http://localhost:8081": readLoopPeekFailLocked: <nil>
2020/04/23 16:26:37 Unsolicited response received on idle HTTP channel starting with "HTTP/1.0 200 OK\r\n\r\nQuick response!"; err=<nil>
2020/04/23 16:26:37 Get "http://localhost:8081": readLoopPeekFailLocked: <nil>
2020/04/23 16:26:37 Unsolicited response received on idle HTTP channel starting with "HTTP/1.0 200 OK\r\n\r\nQuick response!"; err=<nil>
2020/04/23 16:26:37 Get "http://localhost:8081": readLoopPeekFailLocked: <nil>

Expected behavior:
No errors output.
Additional detail:
This test case also reproduces when I set http.DisableKeepAlives, and when I set Connection:Close on the response from the hasty server. Also, each connection yields at most 1 HTTP response, so this is not really an issue about an unsolicited response on an idle channel as the message suggests, but seems like a race between roundTrip incrementing numExpectedResponses and readLoop checking whether numExpectedResponses is 0. I don't know this code very well but I wonder if it makes sense to treat every connection as starting with numExpectedResponses at 1?
I've tried to figure out whether hasty responses are allowed or disallowed by RFC 7230. The closest language I can find is:

HTTP only presumes a reliable
transport with in-order delivery of requests and the corresponding
in-order delivery of responses.  The mapping of HTTP request and
response structures onto the data units of an underlying transport
protocol is outside the scope of this specification.

Which seems to implicitly allow them. At any rate it's not clear that it would be possible to consistently reject hasty responses, since there's an unavoidable network race between writing a request and reading a response. Given that, I think it probably makes sense to make the current behavior (accepting hasty responses) more consistent.

I can confirm that this patch, which starts persistConns with numExpectedResponses = 1, causes the repro code above to run without errors.
0001-Start-persistConns-with-numExpectedResponses-1.patch.txt

move <-pc.reqch to the front of  Peek may solve this problem
--- net/http/transport.go.bk    2020-04-24 10:06:11.253338231 +0800
+++ net/http/transport.go       2020-04-24 10:07:03.676671394 +0800
@@ -1893,6 +1893,7 @@

        alive := true
        for alive {
+               rc := <-pc.reqch
                pc.readLimit = pc.maxHeaderResponseSize()
                _, err := pc.br.Peek(1)

@@ -1904,7 +1905,6 @@
                }
                pc.mu.Unlock()

-               rc := <-pc.reqch
                trace := httptrace.ContextClientTrace(rc.req.Context())

                var resp *Response

hi i'm receiving the same error. is there a way I can try this patch out without having to rebuild a fork fo go?

Yep, it's still actual with go version go1.16.6 linux/amd64. I've improved the code to get detailed stat https://play.golang.org/p/V3YkMeJ__4F (result can't be given on the play.golang.org). For example, for  100 000 total requests, only 329 ends with the error.

Specifically, there are some "hasty" HTTP servers out there that will send an HTTP response as soon as the TCP connection completes.

... but seems like a race between roundTrip incrementing numExpectedResponses and readLoop checking whether numExpectedResponses is 0.

@jsha Thanks for your repro code!
As far as I understand, I think the race you mentioned above may also happen for "non-hasty" HTTP servers, in which case pc.numExpectedResponses++ does not necessarily happen before if pc.numExpectedResponses == 0 {...} either.
However, I can not reproduce the same error by replacing hastyHTTPServer() with normalHTTPServer() as below:
package main

import (
	"fmt"
	"log"
	"net"
	"net/http"
	"sync"
	"time"
)

const port = ":8081"

var wg sync.WaitGroup

func init() {
	tr := http.DefaultTransport.(*http.Transport)
	tr.DisableKeepAlives = true
}

func main() {
	//go hastyHTTPServer()
	go normalHTTPServer()
	time.Sleep(100 * time.Millisecond)
	for i := 0; i < 2; i++ {
		wg.Add(1)
		go fetchLoop()
	}
	wg.Wait()
}

func fetchLoop() {
	for i := 0; i < 1e3; i++ {
		resp, err := http.Get("http://localhost" + port)
		if err != nil {
			log.Println(err)
			continue
		}
		resp.Body.Close()
	}
	wg.Done()
}

// hastyHTTPServer runs an HTTP server that sends a response and closes the
// connection as soon as a client makes a TCP connection, rather than waiting
// for the client to actually make an HTTP request.
// A real-world example of this type of server can be seen at 8849777.com
func hastyHTTPServer() {
	ln, err := net.Listen("tcp", port)
	if err != nil {
		log.Fatal(err)
	}
	for {
		conn, err := ln.Accept()
		if err != nil {
			log.Fatal(err)
		}
		go func() {
			conn.Write([]byte("HTTP/1.0 200 OK\r\n\r\nQuick response!"))
			conn.Close()
		}()
	}
}

func normalHTTPServer() {
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "HTTP/1.0 200 OK\r\n\r\nQuick response!")
	})
	log.Fatal(http.ListenAndServe(port, nil))
}

For those who are interested, I have finally figured out the reason for the readLoopPeekFailLocked: <nil> errors occurred in the production environment.
TL;DR: An abnormal 204 response, which includes a body, will cause the readLoopPeekFailLocked: <nil> error.
Background
We have deployed two types of backends behind Caddy (a web server in Go):

Client -> Caddy -> Go Backend
Client -> Caddy -> Erlang Backend (Cowboy based)

But only HTTP requests in the 2nd case (i.e. Caddy <-> Erlang Backend) are encountering the readLoopPeekFailLocked: <nil> errors occasionally.
Root Cause
The Erlang Backend is using Cowboy 2.7.0, which unfortunately allows developers to construct a 204 response with a body. And the fact is that our Erlang developer does do this!
After correcting these abnormal 204 responses, all readLoopPeekFailLocked: <nil> errors disappear.
NOTE: Cowboy 2.8.0 has fixed this issue (commit).
Reproducing Code
package main

import (
	"log"
	"net/http"
	"time"
)

const port = ":8081"

func main() {
	go abnormalHTTPServer()
	time.Sleep(100 * time.Millisecond)

	fetchLoop()
}

func fetchLoop() {
	client := &http.Client{Timeout: 2 * time.Second}
	for i := 0; i < 2; i++ {
		resp, err := client.Get("http://localhost" + port)
		if err != nil {
			log.Println(err)
			continue
		}
		resp.Body.Close()
	}
}

// abnormalHTTPServer runs an HTTP server that sends a 204 response with a body.
func abnormalHTTPServer() {
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		hj, ok := w.(http.Hijacker)
		if !ok {
			panic("impossible for HTTP/1.x")
		}

		conn, _, err := hj.Hijack()
		if err != nil {
			panic(err)
		}

		// Mimic the abnormal 204 response from Cowboy 2.7.0.
		conn.Write([]byte("HTTP/1.1 204 No Content\r\n\r\n{}"))
		//conn.Close()
	})
	log.Fatal(http.ListenAndServe(port, nil))
}

The same error still can be reproduced with https://github.com/golang/go/issues/31259#issuecomment-541229021 on go1.19.5

Related to #59310

FYI, a new RFC 9112 was published in June 2022 adding more clarity to unsolicited responses. This spec supersedes RFC 7230 in that regard:

If a client receives data on a connection that doesn't have outstanding requests, the client MUST NOT consider that data to be a valid response; the client SHOULD close the connection, since message delimitation is now ambiguous, unless the data consists only of one or more CRLF

That's very interesting @domdom82 ! Thanks for sharing.
Strangely, even though RFC 9112 seems to make this more clear, I think it winds up being less clear - and to some extent, introduces a race condition into protocol handling.
Traditional single-threaded HTTP implementations would do something like:

connect
write the request
read the socket for a response

If you do that, a response from a "hasty" HTTP server will look normal - it will look as if it was a response to the request.
However, my understanding is that Go winds up in a different situation:

connect
fire off a goroutine to read from the socket and process results
write the request

So depending on the speed of the server and the speed of the network, a "hasty" response could arrive before or after the request is written. So the RFC 9112 requirement to reject responses on idle channels seems to require the current racy behavior - but only if the client happens to be reading concurrently.
Perhaps a more useful fix on the HTTP RFC side would have been to specify that servers MUST NOT be hasty. That would make it somewhat easier to say that the client is free to do whatever it likes (including logging an error to stdout) when a hasty server is encountered.

I get err like Get "https://localhost:8777": readLoopPeekFailLocked: %!w(<nil>)
(that %!w if anything is a bug, right?)


Nested DllCall conversion

v1
DllCall("oleacc\AccessibleChildren", "Ptr", ComObjValue(Acc), "Int", 0, "Int", cChildren, "Ptr", VarSetCapacity(varChildren,cChildren*(8+2*A_PtrSize),0)*0+&varChildren, "Int*", cChildren)
v1→v2 converter
DllCall("oleacc\AccessibleChildren", "Ptr", ComObjValue(Acc), "Int", 0, "Int", cChildren, "Ptr", varChildren := Buffer(cChildren*(8+2*A_PtrSize), 0)*0+&varChildren, "Int*", &cChildren := 0)
Shouldn't
varChildren := Buffer(cChildren*(8+2*A_PtrSize), 0)*0 +&varChildren instead be
(varChildren:= Buffer(cChildren*(8+2*A_PtrSize), 0))*0+&varChildren?

Yes I believe it should be.
If you want to take a crack at fixing that, feel free. Lol

Sorry, I barely understand this DllCalls and Buffer etc syntax as is and struggle if my copy&pasted examlpes misbehave, unlikely would be able to help much in writing a translation code to fix it :(
I might take a look at it as it might be as simple as adding *0 to a copy of VarSetCapacity(TargetVar,RequestedCapacity,FillByte) | *_VarSterCapacity and copy&pasting the _VarSterCapacity function to put the parenthesis in the correct places
Just to understand your testing framework — is there currently a way to test the files directly, e.g. VarSetCapacity_ex1.ah1 and VarSetCapacity_ex1.ah2 (that would be extremely helpful as it's easy to delete everything, modify these two files and see if I can change the rules to make it work, then get the other tests and see if anything is broken)
or is the only way to work is to paste the content to the Tests.ahk file?

The original way I did the testing framework was to use the Tests.ahk file
But @dmtr99 added his new way of testing using the ah1 and ah2 files with the QuickConvertor script. You can try running that and playing around with it. I'm not exactly sure how it works to be honest

I've finished the tests that adds extra parenthesis to match the example, they passed, but then setting this.test_exec := true revealed that the suggested "fixed" conversion is wrong — you can't multiply a buffer object by 0, you get an Expected a Number, but got a Buffer. runtime error, so the proper way to fix this is something that should've been done in the original code to make it at least somewhat readable — you need to set the var outside the DllCall and then use it as a parameter
So I guess there is no point in submitting the fix, this example should just manually be rewritten
Let me know if you feel the fix is still useful for something and I'll submit it anyway, just disabling the FileAppend part so that it wouldn't be actually executed in tests

I saw your attempted fix in your branch, I added a comment there.
The proper fix would be complicated, and that would be to check if the replacement is inside another expression (which i have no idea how we'd determine that), and then if so, add parens around the whole replacement.
Otherwise, maybe a simpler fix that would work for this specific case, would be to change this line:
https://github.com/mmikeww/AHK-v2-script-converter/blob/11a0d14472844396e8e165451acd573f0b42d786/ConvertFuncs.ahk#L3162
Return Format("{1} := Buffer({2}, {3})", p*)
to
Return Format("({1} := Buffer({2}, {3}))", p*)
but that will end up always putting parens around all of these replacements, even if they are on a line by themselves

but that will end up always putting parens around all of these replacements, even if they are on a line by themselves

Yeah, that's what my "fix" actually does, and I haven't noticed that since there was not a unit test for the non-*0 version of the VarSetCapacity function call and thought your parser did some magic by matching ()*0 separately from () in the FunctionsToConvert

check if the replacement is inside another expression (which i have no idea how we'd determine that), and then if so, add parens around the whole replacement.

maybe that's the proper general way, but for this can't we "just" change the parser not to finish at the ending ), but also match everything after that (within the given string VarSetCapacity(TargetVar,RequestedCapacity,FillByte)*0) and pass it as a separate {4} parameter to the _VarSterCapacity function so that the function can know it's not blank and you need to do something different

If you don't mind another opinion in the thread...
I think this case could be generalized to other functions.
Probably a proper check to add wrapping parenthesis would go just before this line:
https://github.com/mmikeww/AHK-v2-script-converter/blob/master/ConvertFuncs.ahk#L604
My general idea is that if "NewFunction" came back with an assignment ":=", and creating some more checks to the context inside oResult.Pre and oResult.Post there would be some clue if it's worth to wrap it in parenthesis.
But I haven't thought that much on what the context checks would be.

Check out this new hack, that should force function(p)_extra into submission!
(also moved some data to separate files as the core file is huge enough and converted into an easier2parse OrderedMap format)
Now the () seem to guard only the multiplied functions, not all of them like before

But I haven't thought that much on what the context checks would be.

yeah, that's the hard part ;) maybe @mmikeww knows AHK syntax enough to set the proper rules

(also moved some data to separate files as the core file is huge enough and converted into an easier2parse OrderedMap format)

Do we really need the OrderedMap? A quick look and it doesn't seem to offer any real benefits. You could've just moved the strings to other files

Check out this new hack, that should force function(p)_extra into submission!

This is great work. As long as all the existing unit tests pass, I'm fine with merging it.  That's why the unit tests are so valuable, because we can easily see if we broke something that used to work.
Only suggestion is I think you should merge the tests commit with the implementation commit, so that they are one distinct commit.

Now the () seem to guard only the multiplied functions, not all of them like before

Could this:
VarSetCapacity(TargetVar,RequestedCapacity,FillByte)\*\d*
be changed to:
VarSetCapacity(TargetVar,RequestedCapacity,FillByte)[\+\-\*\\/]\d*
to cover +,-,*,/

If you don't mind another opinion in the thread... I think this case could be generalized to other functions. Probably a proper check to add wrapping parenthesis would go just before this line: https://github.com/mmikeww/AHK-v2-script-converter/blob/master/ConvertFuncs.ahk#L604 My general idea is that if "NewFunction" came back with an assignment ":=", and creating some more checks to the context inside oResult.Pre and oResult.Post there would be some clue if it's worth to wrap it in parenthesis. But I haven't thought that much on what the context checks would be.

Yes very quickly, we can start to question, can this work for more expressions? I think you are on the right path with your idea. Whereas eugene just tried to solve a very specific case, your plan could be generalized for many more.
Sadly, I do not really have the time currently to revisit this project. If you guys are capable of implementing this stuff, along with associated unit tests, and all existing tests pass, then I'm happy to merge the changes in.

Do we really need the OrderedMap? A quick look and it doesn't seem to offer any real benefits. You could've just moved the strings to other files

Well, then we'd have to worry about properly splittng on "|" and whether some regex rule broke it silently by adding it's own |, so strings aren't a proper data structure for this, why bother splitting after if we already know the proper split before?
We could use Maps(1, ["NameV1","NameV2"]) to avoid the dependency, but then not sure the order of this map is guaranteed by AHK, so might be a bit brittle

As long as all the existing unit tests pass ... That's why the unit tests are so valuable, because we can easily see if we broke something that used to work.

Fully  agree on the value of unit tests. I only have one broken test StringReplace_All_NoReplaceText, but it was broken for me on the master as well, so not relate to these changes

Only suggestion is I think you should merge the tests commit with the implementation commit, so that they are one distinct commit.

Ok

Could this: VarSetCapacity(TargetVar,RequestedCapacity,FillByte)\*\d* be changed to: VarSetCapacity(TargetVar,RequestedCapacity,FillByte)[\+\-\*\\/]\d* to cover +,-,*,/

Maybe? In general, this doesn't make much sense due to the change in types:
in v1 the output of was GrantedCapacity integer, so you could potentially + or - to this, but
in v2 the output is a buffer, for which these operations don't make sense.
But then this is no different from the original * :) and they seem to both raise a runtime error on use, so not sure which ways is less bad ;)

Do we really need the OrderedMap? A quick look and it doesn't seem to offer any real benefits. You could've just moved the strings to other files

Well, then we'd have to worry about properly splittng on "|" and whether some regex rule broke it silently by adding it's own |, so strings aren't a proper data structure for this, why bother splitting after if we already know the proper split before? We could use Maps(1, ["NameV1","NameV2"]) to avoid the dependency, but then not sure the order of this map is guaranteed by AHK, so might be a bit brittle

True. Before we weren't using regexmatches I don't believe, so there was no concern about the stray pipe. I think your solution was the first to use a regex.

Could this: VarSetCapacity(TargetVar,RequestedCapacity,FillByte)\*\d* be changed to: VarSetCapacity(TargetVar,RequestedCapacity,FillByte)[\+\-\*\\/]\d* to cover +,-,*,/

Maybe? In general, this doesn't make much sense due to the change in types: in v1 the output of was GrantedCapacity integer, so you could potentially + or - to this, but in v2 the output is a buffer, for which these operations don't make sense. But then this is no different from the original * :) and they seem to both raise a runtime error on use, so not sure which ways is less bad ;)

Good point

Good point, but then, if the conversion is invalid, why should I converter give this as a result? In other instances, we would insert a comment line with some explanation instead of giving false conversion

Well, the conversion is valid (in the case *0), it just can't be inlined like before, needs to be moved before (and not zeroed out anymore). By the way, is it actually possible with this parser to move the result to a previous separate line (then we could also remove the *0)?
Otherwise, could you please point to an example of a function where a comment is inserted on a separate line, might be a good idea to add to this?
Re. the other + / -, wouldn't those only be used with a 2-parameter function for strings, which would be converted to VarSetStrCapacity and not suffer from a :=
And in they are used with a 3-parameter VarSetCapacity for some reason then let's also just add a comment

Well, the conversion is valid (in the case *0)

Should we only convert this valid case then?

By the way, is it actually possible with this parser to move the result to a previous separate line (then we could also remove the *0)?

Initially there was no support for moving replacements on previous/next lines, but I don't recall if @dmtr99 added this feature when he was overhauling the converter. However....

Otherwise, could you please point to an example of a function where a comment is inserted on a separate line, might be a good idea to add to this?

There are cases where we simply replace multiple lines instead of one. The output string just includes `n chars. Example of comment:
https://github.com/mmikeww/AHK-v2-script-converter/blob/11a0d14472844396e8e165451acd573f0b42d786/ConvertFuncs.ahk#L3026
Example of multi line replace:
https://github.com/mmikeww/AHK-v2-script-converter/blob/11a0d14472844396e8e165451acd573f0b42d786/ConvertFuncs.ahk#L3177
I believe these have to go in the custom replacement funcs

Re. the other + / -, wouldn't those only be used with a 2-parameter function for strings, which would be converted to VarSetStrCapacity and not suffer from a :=
And in they are used with a 3-parameter VarSetCapacity for some reason then let's also just add a comment

I don't know, I haven't kept up with this project and won't have time in the near future. If you want to take some initiative, I can give you write access to the repository as well, like i did with dmtr99

Should we only convert this valid case then?

That was the my idea behind only the * support, but I've recently encountered varsetcap()//4 where the granted capacity result of varsetcapactiy was actually used in a loop in a legitimate way to get the whole number of 4-byte vars, so your idea of adding more operators makes more sense, so instead of trying to guess, which of those operators might be valid or not (unless you know, so please do tell :) ), I guess we should have a proper generalized conversion, though this should require us to:

realize that we don't really know its a buffer even with 3 args (I've seen strings for WinAPIs that should be strings that had 3 args in v1, don't know enough whether you really needed to zero them out, but at any rate that's a valid v1 use case that shouldn't be buffered in v2)
remove the all-too-smart inlining and set var as a buffer/string on a previous line
use buffer.Size in place of where varsetcapacity was (unless its a string, then just use the granted capacity from the previous line)

So the rule should be something like this:
v1 VarSetCapacity(vTarget , RequestedCap, FillByte) +,-,/,*,// X
v2

line − 3: ; Warning that there are two options: (usually) a buffer and (sometimes) a string, it depends on which API the variable is used in and we don't have the capacity to guess it
line − 2: vTarget := Buffer(RequestedCap, FillByte) ; option 1
line − 1: vTargetCap_v1v2 := VarSetStrCapacity(&vTarget, RequestedCap) ; option 2, replace vTarget with vTargetCap_v1v2 below
conversion line: vTarget + - / * // X

OR in the absence of a previous-line-comment feature we can use this hack:
v1 VarSetCapacity(vTarget , RequestedCap, FillByte) +,-,/,*,// X
v2 (vTarget:=Buffer(16,0)).Size +,-,/,*,// X
This also sets the variable and uses its size inline (but doesn't prevent mistakes when vTarget was supposed to be a string)
So I'll implement this hack then and leave the proper commented conversion for when there is a previous-line-comment feature

There are cases where we simply replace multiple lines instead of one.

Yeah, unfortunately, this wouldn't work in complex inline cases as we don't really know all the contexts in which they can appear (it's not just DllCalls, but also loops, but then also I have no clue what else) :(

Re. the other + / -, wouldn't those only be used with a 2-parameter function for strings

Likely nope as I've seen with the // example mentioned above :(

If you want to take some initiative, I can give you write access to the repository as well, like i did with dmtr99

Thanks, though given my limited knowledge of the syntax and ahk language, I'd prefer if someone reviews the changes

@dmtr99 have you by any chanced added a feature to display additional commens on a new previous line during conversion so that we can signal some warnings to the user?

@dmtr99 have you by any chanced added a feature to display additional commens on a new previous line during conversion so that we can signal some warnings to the user?

No, you could add it to the line itself, or adding an extra global variable in the line

add it to the line itself

hm, but I'm not operating on a line level, but an argument level
Let's say there is this
DllCall(...,VarSetCapacity(vTarget, reqCap, Fill)*0+&vTarget,...) ; some existing comment
The update/test I've done converts the VarSetCapacity argument (with a hack it'll also be working inside this DllCall)
Question: is it possible to ask the converter to ADD a new line BEFORE this DllCall based on parsing VarSetCapacity, can I change this EOLComment inside a _VarSetCapacity(p) {} output construction function or something?
(In this case it'll just be a user warning that the conversion might have another alternative)

Inside the custom func replacements themselves (that we've prefixed with _ ), the whole line is not available, only the parameters that we parsed earlier which is passed in with the array p*
But you can preceed the output line with a comment, by just doing something like:
return "comment here`r`n" . Format(replacement output line)

Inside the custom func replacements themselves (that we've prefixed with _ ), the whole line is not available, only the parameters that we parsed earlier which is passed in with the array p*

I think I can partially get around that by passing oResult as an argument, then at least I can get everything except for the comment in e.g. oResult.Post
Line comments for some reason are not part of some oResult property, might need to add them as then I'll definitely be certain that it's safe to add another comment there

But you can preceed the output line with a comment, by just doing something like:

I don't think I can since my Format() isn't the full line, it can be a function argument.
But I can do what dmtr99 suggested and change a global var

Ok, think I've done everything structural related to VarSet, now it can add new buffers to new previous lines, zero out with ().Size where possible, warn that any conversion can be wrong :) and suggest the alternative in an end-of-line comment etc., and now DllCall is even a bit smarter and knows how to guard the resulting VarSetStr in a StrPtr()!
I've added a warning than if there is an un{guarded} control flow statement and there is a new line(s) created so that the user should guard it manually, but can the parser track it?
That is, can I know that the previous line was a control flow statement and it ended without {?
Then I'd be able to add {} myself

That is, can I know that the previous line was a control flow statement and it ended without {?

I don't believe our parser can track that

Can the parser convey this info about the current line?
Then I guess we could save it in some wasPreviousLineControlNoBlock :-) global variable and act accordingly in parsing the current line

I don't believe we even track control flow statements as a single group
The if statements are all handled near the top of the conversion function and detected with regexes
The IfCommands and Loop statements are handled separately in the long list of Command parsing


Why self.first_pass = True in Scaler method

Hello, in utils.py 29 line, there is self.first_pass = True, as far as what I know, this will not lead to a running mean method, isn't it? Plz point out what I misunderstood, thanks.

my wrong.

my wrong, sorry.


Kubernetes service discovery doesn't work along with simple discovery service

Describe the bug
I want to have 2 layers of discovery service: Kubernetes discovery client and simple discovery client. It's working with eureka (local dev env) and it was working before 3.0 spring-cloud-kubernetes.
Sample
Here's my configs:
spring:
  config:
    import: "kubernetes:"
  cloud:
    kubernetes:
      discovery:
        service-labels:
          "[app.kubernetes.io/part-of]": asdf
      loadbalancer:
        mode: SERVICE

spring:
  cloud:
    discovery:
      client:
        simple:
          instances:
            xyz:
              - uri: http://xyz/
            xyz-custom:
              - uri: http://xyz/

and I've got:
Load balancer does not contain an instance for the service xyz-custom

Can you provide a complete, minimal, verifiable sample that reproduces the problem? It should be available as a GitHub (or similar) project or attached to this issue as a zip file.

I'm attaching spring boot project. I don't attach K8s yaml?
When you run with profile local it should connect to eureka and it's working. When you deploy to K8s it doesn't work.
Endpoint:
GET app_address/weather
mvp.zip

I just imported your sample, and what am I supposed to see there? How do I understand what problem are you facing?
You need to tell us, how to deploy this (yes, k8s yaml files) and requests you make, what you expect, etc.
I assume you build an image our of that project, have a deployment. So after it ends up in k8s cluster (again, show us how you do that), what are you doing with it? You make some requests? How is eureka related to this and where is it in your project set-up?

Local without K8s (everything working fine)

download zip
unzip
run: mvn spring-boot:run
make GET request to: http://localhost:8080/weather
get response, something like this: {"current_weather":{"time":"2023-04-13T09:00:00","temperature":9.8,"weathercode":3,"windspeed":11.1,"winddirection":205.0}}
everything works fine

On K8s (not working)

download yaml file (attached to this post)
install app on K8s (I have pushed image already to docker hub)
run kubectl port-forward YOU_POD_NAME 8080:8080
make GET request to: http://localhost:8080/weather
you've got an error:

feign.FeignException$ServiceUnavailable: [503] during [GET] to [http://openmeteo/v1/forecast?latitude=52.52&longitude=13.41&current_weather=true&hourly=temperature_2m&hourly=relativehumidity_2m&hourly=windspeed_10m] [OpenMeteoFeignClient#getWeather()]: [Load balancer does not contain an instance for the service openmeteo]

It isn't working when I'm using simple discovery client combined with K8s service discovery. It was working on version 2.x. Also when i combine simple discovery client with eureka server (without any K8s) it's also working.

Much, much better now! Thank you.
I can re-produce and I think I know where the problem is... I'll update this post once I have all the proper conclusions.

This is the code to be blamed:
List<V1Service> services = properties.allNamespaces()
				? serviceLister.list().stream().filter(svc -> serviceId.equals(svc.getMetadata().getName())).toList()
				: List.of(serviceLister.namespace(namespace).get(serviceId));

Specifically this part: List.of(serviceLister.namespace(namespace).get(serviceId)). Because CompositeDiscoveryClient kicks in, it will call KubernetesInformerDiscoveryClient::getInstances with a value of openmeteo. In turn this leads to serviceLister.namespace(namespace).get(serviceId) being null, and a null in an ImmutableCollection triggers a NPE.
That is a bug.
The good part, is that because of recent simplifications, there is no fix needed here (I will still add a minor integration test), that code was removed.

What I did is test this against our current 3.0.3-SNAPSHOT:
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-kubernetes-client-all</artifactId>
            <version>3.0.3-SNAPSHOT</version>
        </dependency>

in your dependency, deploy locally in k8s and :
root@kind-control-plane:/# curl <IP_ADDRESS>:8080/weather
{"current_weather":{"time":"2023-04-13T17:00:00","temperature":12.7,"weathercode":3,"windspeed":7.5,"winddirection":235.0}}

@ryanjbaxter I'll add a minor integration test case here, so please let's keep it open until then. Thank you.

perfect, thank you. Have you any estimate time when you release next version?

Tentative release dates can be found here https://github.com/spring-cloud/spring-cloud-release/milestones


Update tests

All the tests are unmodified and taken as it is from github:tantale/deprecated, we need to change them to suit the new features such as deprecated_args

temporarily this works as a live test, but I need to make proper tests soon, https://deprecat.readthedocs.io/en/latest/source/api.html#examples

done!


Disclosure of Supply Chain Vulnerability

Vulnerability type:   Leaked access token
Domain:   Github Source Code and NPM Packages
How:   Leaked sensitive github code in docker stage build logs
Description
The workflow files for GitHub Actions are currently poorly designed, using sensitive data passed through environment variables which leaks in the build logs.
Otomi DockerHub repository has several public images using multiple build stages, which are exposing NPM_TOKEN variable.
NPM_TOKEN is defined as a GitHub secret and should be hidden from public access.
Considering Docker BuildKit secrets mounting features, all private env vars should be passed in secrets to the build.
Reproduction
Navigate to https://hub.docker.com/layers/otomi/console-stages/1/images/sha256-c30a8523f3a396a07beab8ba31929762f7af9c5ef63a170050522fe4c2b82e7a using unauthenticated browser session.
Click on Image Layer 11.
This will show the underlying command which exposes NPM_TOKEN sensitive value. ( NPM_TOKEN=a160adxxxxxxxxxxxxxxxxxxxxxxx )
Exploitability
Using this token gives unauthorized actors access to Private Github Repositories and Resources.
Attack vector:

Gain visibility to all repositories
clone/fork/create/delete private repositories
View Private Issues and Discussions
Access to private npm packages
Gain access to underlying private info and possibly escalate priviledges to other services

Demo:
$ export GITHUB_TOKEN=$NPM_TOKEN                                  (impersonating token)
$ gh api graphql -f query='MALICIOUS_QUERY_TO_VIEW_ALL_REPOS'     (view all PRIVATE repos)
$ git clone PRIVATE_REPO                                          (clone private repo)
$ gh issue list                                                   (view private issues & discussions)
$ gitleaks -r PRIVATE_REPO -v --pretty --access-token=$NPM_TOKEN  (gain additional access from repo leaks scanning)

Impact
All Private GitHub repositories using compromised token code.
Other services exposing sensitive information under private repos.
Recommended
Refresh leaked access token value
Patching of all Workflow Pipelines and Dockerfiles to use BUILDKIT secrets mounting features and pass sensitive variables as docker build secrets.
Possibly Remove logs and exposed build views

Workflow files:

set DOCKER_BUILDKIT=1 environment variable for docker action steps

use following argument in build stages:
build_extra_args: "--secret=id=NPM_TOKEN,src=/local/npm-token"
Dockerfiles:

use following format for RUN directives
RUN --mount=type=secret,id=NPM_TOKEN,dst=/npm-token cat /npm-token

[x] Create and use unprivileged NPM_TOKEN to allow just READ on public packages, thus allowing downloads from the build image

[x] Use Private Github CR for build pipelines stages

Reference
https://docs.docker.com/develop/develop-images/build_enhancements
https://docs.github.com/en/free-pro-team@latest/actions/reference/encrypted-secrets
https://docs.github.com/en/free-pro-team@latest/actions/learn-github-actions/security-hardening-for-github-actions
Better P.S. than a postmortem
I know we are in the process of open-sourcing but using secrets should be mandatory across layers and private things should be private for all intents and purposes.

Fixed and closed.


Cannot use search or scroll in menu on Openlayers website API page for v4.6.5

Describe the bug
On de API v4.6.5 page of the Openlayers website the search bar does not work and I cannot scroll at all in the menu. In the console I found an error stating that "currentVersion" has been redeclared. One is declared on line 56 and then redeclared on line 59. I'm guessing this stops javascript execution. Its hard to navigate the website atm and its been like this for a few weeks now. Not sure where I should have reported this bug but I thought I should atleast let you know somewhere.
To Reproduce
Steps to reproduce the behavior:

Go to 'https://openlayers.org/en/v4.6.5/apidoc/'
Try to use the search bar, on the left side of the screen, or try to scroll the menu beneath it
See error in console

Expected behavior
The search bar to filter the menu & the being able to scroll in the menu

Thanks for reporting this. Fixed with https://github.com/openlayers/openlayers.github.io/commit/b58470f00a7f36ca1d1c974e3ddf364f0c0c138d.


Add ToggleEntity save and restore state mechanism

Breaking Change:
Now ToggleEntity state can be saved and restored.
Description:
It can be useful when you temporary need to change state of the switch, light or fan and restore it later, i.e. temporary turn on light when motion was detected and schedule restore previous state in 2 minutes.
Example automation entry for automation.yaml:
- id: 'motion_detected'
  alias: Motion detected
  trigger:
  - platform: state
    entity_id: binary_sensor.hallway_motion
    from: 'off'
    to: 'on'
  action:
  - service: light.save_state
    data:
      entity_id: light.hallway
  - service: light.restore_state
    data:
      entity_id: light.hallway
      delay: 00:02:00

Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.
[x] I have followed the development checklist

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] The manifest file has all fields filled out correctly. Update and include derived files by running python3 -m script.hassfest.
[ ] New or updated dependencies have been added to requirements_all.txt by running python3 -m script.gen_requirements_all.
[ ] Untested files have been added to .coveragerc.

If the code does not interact with devices:

[ ] Tests have been added to verify that the new code works.

The standard development process is to open an issue here: https://github.com/home-assistant/architecture. If they are approved there, you can implement/open such functionality here.
For now, you can open an architecture issue, and if that will be approved, someone will review this code.
Thanks for your contribution.

Will close this until this is discussed and approved in the architecture repo.


[Question] Current support for cassandra persistence module

Hi there,
This might be somewhat related or a continuation of issue #819. We're looking to use/extend cassandra db option as our execution data layer. It looks like there're quite a few methods in ExecutionDAO that the cassandra impl didn't implement. I understand that the cassandra module is only providing partial implementation, what I want to find out is that what functionally are we missing(or considered not critical) from using/running conductor with the cassandra impl as is?
Is there planned support for cassandra now or in the near future?
Thanks in advance,
Chao

+1

@cshen200 As @apanicker-nflx mentioned in the other thread, at the moment, Cassandra module supports ExecutionDAO persistence. We're planning to add MetadataDAO persistence sooner. We're not sure about adding QueueDAO implementation though. We've been using Cassandra in production for the past year at Netflix and definitely plan to support it.

Thanks for the response @kishorebanala!
So if I'm understanding this correctly, the current Cassandra module which implemented some functionality of ExecutionDAO persistence(Looking at code in master branch) is sufficient enough to support your production needs?
And that the other unimplemented methods are not used in Prod workflow execution?
I'm curious what your setup is in Production, do you use Cassandra as your backing DB for execution data while using different DB(s) for other DAOs?
Thanks.
Chao

@cshen200

So if I'm understanding this correctly, the current Cassandra module which implemented some functionality of ExecutionDAO persistence(Looking at code in master branch) is sufficient enough to support your production needs? And that the other unimplemented methods are not used in Prod workflow execution?
This is correct. Current implementations are sufficient enough to support workflow execution requirements.

I'm curious what your setup is in Production, do you use Cassandra as your backing DB for execution data while using different DB(s) for other DAOs?
In production, we are using Cassandra for storing all the workflow and task execution data. We are using dynomite/dyno-queues for the metadata DAO and the queue DAO respectively. We plan to migrate the metadata DAO to Cassandra and we will be implementing this in Conductor soon.

As for other methods in the execution DAO, all of the methods are providing a search functionality and do not impact workflow/task execution, and we are utilizing the indexDAO to surface this information to the APIs.
For the methods related to event execution, we are continuing to use dynomite for this purpose. These will also be implemented in Cassandra soon.
However, if you choose to use Cassandra as your execution store, you would still need to use another dependency for your queueDAO implementation, since we do not plan to support/implement a queueing layer built on top of Cassandra.

Thanks much for the detailed information @apanicker-nflx, really appreciate it! That answers all my questions above.
We'll move forward with using redis for our execution and queue backing DB, while using aurora as our metadata DB.
While I've got your attention, a couple more questions if you don't mind.

I'm sure you've considered this, but any concern on the volatile nature of using in-mem cache to serve metadata? Curious what your redundancy level is for your dynomite clusters for metadata?

Is the additional wire up and setup for supporting different DB for different data types(as mentioned above) checked in to this repo?

Thanks again,
Chao

I'm sure you've considered this, but any concern on the volatile nature of using in-mem cache to serve metadata? Curious what your redundancy level is for your dynomite clusters for metadata?

Our dynomite cluster is set up with cross-region replication with in-region replication factor of 3. Additionally, the in-memory redis data is persisted to disk every few minutes and a snapshot of the data is taken every day. Since the metadata (workflow and task definitions) do not change frequently, this setup has been working fine for our use-case.

Is the additional wire up and setup for supporting different DB for different data types(as mentioned above) checked in to this repo?

It is not checked in, since this is a very specific setup for our internal deployment, which was born out of live migration from dynomite to C*. The additional wire-up is a very thin layer similar to the ExecutionDAOFacade but implements the ExecutionDAO interface. This implementation is a simple orchestrator of the respective apis to the corresponding DAO layers (C*, dynomite, index).