theme download gets stuck at 97%

Hi there,
I recently updated themekit to latest version (1.1.1) and since then it cannot download all the files...
I tried with 3 different themes and all of them get stuck around 97-98%
[development] Warning, this is the live theme on XXXXX-THEME.
[development] 389|401 [===================================================================>--]  97 %
Any thoughts on this?
Thanks
OS: Mac
Themekit: v1.1.1

Hey @alexissel Does it get stuck forever? Since v1.0 themekit, there are 2 things that would make it pause

Throttling pausing, generally for 2 seconds for each file in flight
Retrying if there was an error.

If these things might be combined as well as the backoff in the retrying it might take about 10 seconds to finish or fail completely

Also @alexissel, can you try running running in verbose mode (-v) to help determine which files it's 'stuck' on?

Same issue here. Since the latest update I have been struggling to pull any themes.
Now it seems to always stick at 97%.
-v shows it sticking at a standard .liquid file in the sections folder.
runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x145c8f2]
Thanks

Thanks, that's very helpful. It seems that when in non-verbose mode the error is being hidden so I've logged that as a separate issue #802.
Are you able to determine which particular liquid files are not being uploaded? If so, can you check if you can upload any of them individually, e.g. theme upload myfile.liquid?

Also, are you seeing a stacktrace below that error, e.g. with a reference to any .go files?

@tanema , yes, it gets stuck forever.
@andyw8 , I just run the command in verbose mode (theme download -v) and for the first 17 files it says "No Change", and for the rest of them, says "Successfully wrote". No error message.
The last file is:
[development] Successfully wrote snippets/catelog_mod.liquid to disk (Local: e816ef6ce58bdd25a2374b1c2e6c0f48)

See below for a couple of recent fails:
`[development] Successfully wrote sections/password-template.liquid to disk (Remote: b4a6f5b07911860fd00fd632cbfc1ea7, Local: )
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x145c8f2]
goroutine 300 [running]:
github.com/Shopify/themekit/src/httpify.(*HTTPClient).doWithRetry(0xc000206280, 0xc0007d6900, 0x0, 0x0, 0x1, 0xc00036a740, 0x0)
/Users/andyw8/src/github.com/Shopify/themekit/src/httpify/client.go:130 +0x102
github.com/Shopify/themekit/src/httpify.(*HTTPClient).do(0xc000206280, 0x1577561, 0x3, 0xc0008000c0, 0x56, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/andyw8/src/github.com/Shopify/themekit/src/httpify/client.go:105 +0x5d1
github.com/Shopify/themekit/src/httpify.(*HTTPClient).Get(0xc000206280, 0xc0008000c0, 0x56, 0x0, 0x19dd860, 0xd, 0xd)
/Users/andyw8/src/github.com/Shopify/themekit/src/httpify/client.go:70 +0x6d
github.com/Shopify/themekit/src/shopify.Client.GetAsset(0x7ffeefbffcd9, 0xb, 0xc0001f83c0, 0x18, 0x19dd860, 0xd, 0xd, 0x19fa1e0, 0x0, 0x0, ...)
/Users/andyw8/src/github.com/Shopify/themekit/src/shopify/theme_client.go:263 +0x178
github.com/Shopify/themekit/cmd.download.func1(0xc00022c900, 0xc0004adcf0, 0xc0002da340, 0x12, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/andyw8/src/github.com/Shopify/themekit/cmd/download.go:62 +0x4a4
created by github.com/Shopify/themekit/cmd.download
/Users/andyw8/src/github.com/Shopify/themekit/cmd/download.go:52 +0x1bb`
`[development] Successfully wrote assets/swatch-get-fierce-leopard-print-coloured-insert-mug-green.png to disk (Remote: 2e0337daf78ca40484e5309f81c6e7e8, Local: )
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x145c8f2]
goroutine 283 [running]:
github.com/Shopify/themekit/src/httpify.(*HTTPClient).doWithRetry(0xc0000ce780, 0xc000888600, 0x0, 0x0, 0x1, 0xc0003b6cc0, 0x0)
/Users/andyw8/src/github.com/Shopify/themekit/src/httpify/client.go:130 +0x102
github.com/Shopify/themekit/src/httpify.(*HTTPClient).do(0xc0000ce780, 0x1577561, 0x3, 0xc0003fca20, 0x58, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/andyw8/src/github.com/Shopify/themekit/src/httpify/client.go:105 +0x5d1
github.com/Shopify/themekit/src/httpify.(*HTTPClient).Get(0xc0000ce780, 0xc0003fca20, 0x58, 0x0, 0x19dd860, 0xd, 0xd)
/Users/andyw8/src/github.com/Shopify/themekit/src/httpify/client.go:70 +0x6d
github.com/Shopify/themekit/src/shopify.Client.GetAsset(0x7ffeefbffcd9, 0xb, 0xc000496c60, 0x18, 0x19dd860, 0xd, 0xd, 0x19fa1e0, 0x0, 0x0, ...)
/Users/andyw8/src/github.com/Shopify/themekit/src/shopify/theme_client.go:263 +0x178
github.com/Shopify/themekit/cmd.download.func1(0xc0000aa900, 0xc0003288a0, 0xc0000d2220, 0x14, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/Users/andyw8/src/github.com/Shopify/themekit/cmd/download.go:62 +0x4a4
created by github.com/Shopify/themekit/cmd.download
/Users/andyw8/src/github.com/Shopify/themekit/cmd/download.go:52 +0x1bb`
Realised that perhaps I'm wrong to say that it 'failed' on a certain file - the most I can specify is the last successful transfer.
Also, after one of these fails, the rest of the download seem to 'pick up' where it left off by running the same theme get command.
Cheers

Thanks, this looks like the same issue as #794 so we'll aim to prioritize that fix.

Thanks @andyw8 !
Hope you'll fix this soon :)

@andyw8 , until you'll release a fix for this, how can I downgrade themekit to v1.0.3 (latest release which works properly)?

@alexissel You can run theme update --version=1.0.3

I have the same problem, I use themekit for many projects, but a couple of them get stuck at advanced download state.
This is usually 98% or 99%, downgrading themekit will fix it.
The problem occurs without any type of error, even the verbose mode does not throw any exception.

I am also having this issue with Themekit 1.1.0 and 1.1.0 there are no error when I run it in verbose mode, it just hangs between 97% and 99%. I have tried running the command multiple times to see if it could complete occasionally, but it seems on some themes it never completes.
OS: Ubuntu 18
Themekit: v1.1.0 and v1.1.1

@alexissel We've fixed the issue and will have a new release of Theme Kit out very soon.

@alexissel You can now run theme update to update to v1.1.2. Please re-open the issue if are running having problems.

@andyw8 , the issue is still there... nothing changed on my end.
I run theme download -v and didn't get any error messages. It still gets stucks after 90%

@alexissel Thanks for reporting, we'll continue to look into.

Also, still having an issue with this. No errors with verbose on, just stuck at 99% and two files remaining to download. I think it may actually be completing the whole theme download based on my git diff when I try to re-download with v1.0.3, but the visual representation of complete download or in verbose mode shows that it never finished and just hangs on v1.1.0, 1.1.1, and 1.1.2

Still having this issue as well on 1.1.2. tried downgrading to 1.0.3 and it seems to work.

I have been looking into this and it only seems to happen on the first download of some themes. I just did it with one theme and it got stuck on a locales file for some reason locales/tr.json
Is it only happening on download for everyone?

Okay I have put up a draft PR with more information on where I think the issues lay. You can see it here https://github.com/Shopify/themekit/pull/819
And you can try this patch out by running theme update --version=v1.1.3-issue801 and validate my approach.

I have had this issue for a few times so far, here are my observations:
I noticed it happens with larger themes rather than to smaller ones.
Generally stuck on around ~96%-99%.
No errors in verbose mode.
Once I left it hanging and files downloaded eventually, but it took hours.
@tanema, I tried your patch, unfortunately, didn't help.

Can anyone else test this to see if it fixes it for you? I am fairly certain about where the problem lies, I just need to figure out the correct tweaks/settings.

I have updated the v1.1.3-issue801 version with some more connection stability configurations, please try it again.

Hey @tanema ,
I updated themekit to your version (v1.1.3-issue801) and I did the following:

run theme download for theme X = got stuck at 56% (after 2 min of waiting I pressed Ctrl+C)
run theme download -v for same theme X = Downloaded: 546, No Changes: 29, Errored: 0 - I think it worked
run theme download for theme Y = download goes to 66% and it stays there for ~3 min, then it goes again to 87% (gets stuck for 2 min) and then it goes to 100%
run theme download for theme Z = download goes to 73% and it stays for ~2 min, then it goes to 100%

So... I think it works?!
I'll try again tomorrow, maybe I have some issues with my internet connection...

Perhaps we could add something in the output to indicate when throttling is happening?

I faced the same problem in v1.0.3. but when I downgraded the version to 1.0.3, the problem was gone.

Tested v1.1.3-issue801 and still experiencing stalls around 90-99%. Theme contains 193 files.
After it stalls, if I leave it downloading, sometimes it will slowly move along 1-2% more, but never finishes. No errors reported with verbose flag.
If API throttling is the issue here, wouldn't all users be affected?
Happy to help test further PRs!

I also have this issue with 1.1.2, stuck on 99%

The working version 1.0.3
sudo theme update --version=1.0.3

I tried v.1.1.3-issue801 with verbose output and it worked. TBH I'm not sure if it was the branch that worked or just patience since I could track progress with the verbose output.
It does seem that theme download now is getting all files again rather than getting only the changed files

@tanema Does this mean this issue is now fixed? If so, how can we get the updated version?

Sorry, I cannot confirm it has. I marked the PR as will-fix and now I am surprised that it auto-closed it

I have been running into this problem with hanging downloads for weeks or months.  Like others, I tried updating to 1.1.2 and that did not fix the issue.  I am on Ubuntu 20.04.
I just updated to your version v1.1.3-issue801 and ran theme get -v -p=<password> -s=<shopname.myshopify.com> -t=<themeid> against an empty download folder.  On the first execution, it ran into a lot of errors like these:
[development] error downloading locales/en.default.json: DNS problem while connecting to Shopify, this indicates a problem with your internet connection
and
[development] error downloading assets/base-comment_form_bg.png: received a malformed response from shopify, this usually indicates a problem with your connection
FWIW, my Internet connection didn't seem unstable.  I am sitting here streaming music and loading other web sites just fine.
I ended up hitting Ctrl-C on the download.  I then cleared out the download folder and re-ran the command.  This time, it hung for periods of time.  There were a lot of errors like this one:
[development] error writing assets/lightblue-h2_underline.png: open /home/derrick/Scratch//theme/assets/lightblue-h2_underline.png: too many open files
It eventually completed with "Downloaded: 300, No Changes: 0, Errored: 2".
I hit the up arrow and re-ran the same command in an attempt to download the errored files.  It again ran into errors on a couple files. It told me:
[development] error writing assets/lightblue-h2_underline.png: open /home/derrick/Scratch//theme/assets/lightblue-h2_underline.png: too many open files
[development] error downloading assets/purple-sort.png: DNS problem while connecting to Shopify, this indicates a problem with your internet connection
This time it completed with "Downloaded: 245, No Changes: 55, Errored: 2".
I couldn't get it to work reliably so I eventually rolled back to 1.0.3.
Hopefully that helps at least a little bit.

I just did theme update --version=v1.1.3-issue801 and ran theme download -v and it still hangs unfortunately. :/
The theme has 517 files.
I also got these kinds of errors (along with the DNS ones) on subsequential runs:
[development] error writing assets/home5_slide3.png: open /absolute/path/to/dev/dir/assets/home5_slide3.png: too many open files

I do not see freezing anymore (v1.1.2) but there is definitely excessive throttling

I have a theme that freezes every time. I let it run the whole night. It still didn't finish downloading.
@tanema if you need the theme so you can repro this issue consistently, email me at viktor.fonic[at]gmail.com.
This is a regression that is not present in 1.0.2. Would it be possible to revert the changes responsible for this issue?
That would help me a lot, as I'd be able to use much faster theme download from 1.1.2, after theme download finally passes.

Would it be possible to revert the changes responsible for this issue?

This would mean just reverting to 1.0.2. There are some foundational changes since them and reverting them would just mean going back to that version number. If we knew the exact issues for this problem we would have fixed them of course.

That would help me a lot, as I'd be able to use much faster theme download from 1.1.2, after theme download finally passes.

That would not be the case if the changes were reverted

Wouldn't it be better if the changes were reverted until proven that they work?

No, since developers are able to roll back themekit to an earlier version this means that they are not blocked from continuing their work. Without developers using this version and being able to identify which instances that it does work and which it does not, I have no recourse to fixing this issue and making themekit better. This change is actually working and improving development for a lot of developers but there is some edge case that we have not identified yet that is effecting a not-small sample of developers and we continue to work on this.

Thanks, @tanema!
I understand now. I thought there were two separate pieces of functionality that were added to last couple of versions:

running multiple requests in parallel
downloading files based on if their checksum changed

If we could revert 1 and keep 2, that would be perfect. But seems like that's not possible...

Without developers using this version and being able to identify which instances that it does work and which it does not, I have no recourse to fixing this issue and making themekit better.

I can send you the full theme file if you want. So far, it hangs 100% of the times.

running multiple requests in parallel

This has been implemented since version 0.0.1
I will be in contact to get theme information for you, that will be helpful, thank you.

Hi, just wanted to comment that version 1.1.2 is still hanging for me at 99% on a basic Debut theme (hardly any customization as it is my dev store). I downgraded to 1.0.2 and that resolved it.

Okay some things I have learned.
While running theme download on @vfonic I found

They are not a plus merchant
hit throttling limit block new requests
inflight requests respond with 429 responses
inflight requests then start responding with 443 responses

Is anyone having issue with any requests on themekit after their command pauses? For instance if you try to run theme download again, does it pause again? I ask because this looks like it is getting squashed by Shopify's bot detection. In trying to optimize themekit's functionality, we have made it a little bit too fast 😅
I may just simply have to limit the amount of inflight requests at a single time so that when a 429 response is received, another 20 requests are not still in flight causing our algorithms to keep counting requests.

I noticed that if I run theme download and, once it reaches ~90% (it's about to or just started to fail new requests), if I kill that process and run theme download again, I will get much more failures. You can see this if you add --verbose which shows more requests failing.

It happens to often for me.

Not sure if my issue is related but I have been working on a theme that when I downloaded it got stuck on 99% and crashed my terminal and VScode. I only ran into problems after making a dynamic section. The dynamic section won't show up in the admin editor.
Link to thread: https://community.shopify.com/c/Shopify-Design/Dynamic-section-not-showing-up-in-admin-editor/m-p/950736#M239145
I will try creating a new store with everything fresh and downloading the theme with the older version of themekit to see if that resolves it.
If it does fix it, it might point to a potential source of this bug being data that is connected to dynamic sections.

Has anybody been able to solve this? I've tried with v1.1.2 and v1.1.3-issue801 to download a theme with no luck.

Yep, v1.0.3 is stable and is working perfectly. It doesn't make sense to update until this issue is fixed.

I thought I'd comment here as I was having the same issue when using @shopify/themekit where the download would freeze at 99% when getting a theme from a store.
I was using @shopify/themekit (v1.1.6) which was using themekit v1.1.1 as the binary under the hood.
Downgrading @shopify/themekit to v1.1.4 fixed this for me as it's the most recent version to have themekit v1.0.3 which seems to be stable at the moment

Same issue on 1.1.2. Got stuck at 174/175 - 99%
Rolling back to 1.0.3 worked

Okay I have opened a PR #839 to fix this. Thanks to @vfonic for lending me his theme to test. Please try the update again theme update --version=v1.1.3-issue801 and see if it works for you. I have tested with @vfonic theme and what was breaking before hand, is now fixed.
An unrelated note on the on the recent optimizations as well since I have seen the same confusion from people here. For the content checking (checksums) there has been some confusion on how that works. If you are only running download right now, you will not see it optimized. We were unable to backfill the (100s of) terrabytes of assets checksums so they will only be in effect if you run a deploy so that every asset will get updated with a checksum and you will see the optimization after that.
Please let me know how the update goes. Since we are easing back on a lot of our internal work in preparation for BFCM I am hoping to ship a lot of my pending work in the next few weeks.

Hey @tanema! Thanks for the time you put into this!
I upgraded to --version=v1.1.3-issue801 and ran theme download on the same theme. Unfortunately, it actually didn't finish successfully. :/
I also tried couple of more times and I also setup a new Shopify store, uploaded the theme there and ran into the same issue.
I've got couple of such errors:
[development] error downloading assets/home_slide1.png: received a malformed response from shopify, this usually indicates a problem with your connection
[development] error downloading snippets/product-footer.liquid: received a malformed response from shopify, this usually indicates a problem with your connection

And then the very last line was:
finished command with errors

Another issue with this last line is that the program finished with exit code 0 (which means successful).
So if this was on a CI, the build would seem like it's passing!
Repro steps:

Export the theme zip file
Extract and add all files to new git repo and commit the changes
Upload the theme to a new store by uploading exported zip file or by using themekit 1.0.3 (if I'm not mistaken: upload the theme w/o checksums)
Run theme update --version=v1.1.3-issue801
Delete some files just to have them in the git status: rm assets/* snippets/* sections/* layouts/* (better do this through file explorer)
Run theme download

Expected:
git status -s displays no output (no changes)
Actual:
git status -s shows the list of previously deleted files that were not downloaded successfully

@vfonic by not finishing successfully do you mean finished with errors or hanging still?

@tanema

@vfonic by not finishing successfully do you mean finished with errors or hanging still?

It finishes with the above errors, but finishes. Maybe the errors are unrelated, but some of the files were not downloaded, perhaps after failing several retries.

It finishes with the above errors, but finishes. Maybe the errors are unrelated, but some of the files were not downloaded, perhaps after failing several retries.

Thank you so much, you are absolutely right and this is very good new. I can address your other problems in separate issues.

I agree with you. If you think that #839 PR fixes this issue good enough (my go skills are pretty bad) and that these other issues could be addressed in the following PRs, I think it's the best way forward, fixing one thing at a time.
Thank you very much, Tim!

Since we will not be merging anything around Black Friday/Cyber Monday, I have created a prerelease with the additions for this issue included in it. You can install the prerelease by running theme update --version=v1.1.3-pre, please share any feedback that you have on this.

@tanema thank you for your efforts in resolving this, much appreciated. I will test and give feedback soon 👍

@tanema I upgraded from ThemeKit v1.1.1 by running theme update --version=v1.1.3-pre and can confirm that it successfully downloads the theme from the store without any errors.
I also tested other parts of the API (watching for file changes when developing, downloading a single file, listing themes and deploying) and everything seems to be working as expected.
If anyone wants to test ThemeKit v1.1.3-pre using the node wrapper @shopify/themekit, I forked the project here so it automatically updates the binary to ThemeKit v1.1.3-pre after install.
This could be useful for testing a pre release of the binary and testing CI/CD workflows where your project is dependent on using the node-wrapper instead of using the ThemeKit binary itself.

I've run theme download for 4 different themes and this is what I've got:
Test 1:
[development] 583|583 [======================================================]  100 %
[development] 583 files, Downloaded: 546, No Change: 37
Test 2:
[development] 542|542 [======================================================]  100 %
[development] 542 files, Downloaded: 539, No Change: 3, Errored: 1
[development] Errors encountered:
[development] error downloading config/settings_data.json: request failed after 5 retries finished command with errors
Test 3:
[development] 543|543 [======================================================]  100 %
[development] 543 files, Downloaded: 540, No Change: 3, Errored: 1
[development] Errors encountered:
[development] error downloading config/settings_data.json: request failed after 5 retries finished command with errors
Test 4:
[development] 255|255 [======================================================]  100 %
[development] 255 files, Downloaded: 237, No Change: 18, Errored: 1
[development] Errors encountered:
[development] error downloading config/settings_data.json: request failed after 5 retries finished command with errors
OS: MacOS Big Sur 11.0.1
ThemeKit: v1.1.3-pre

Here's another test:
[development] 353|353 [======================================================]  100 %
[development] 353 files, Downloaded: 332, No Change: 21, Errored: 4
[development] Errors encountered:
[development] error downloading assets/about_us_parallax.jpg: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: 218ec5d6-25fd-497e-b6a7-a45773de010a
[development] error downloading assets/mailing_list_bg.jpg: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: c326a64d-17e2-4c9d-9ae5-a9025f06dc4f
[development] error downloading assets/banner_on_top_right.jpg: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: c3908fd5-d29d-4053-aa57-19f7633c12d5
[development] error downloading assets/block_collection_img.jpg: could not read response body
This may mean that the request was not able to finish successfully
Http Response Status: 200
Request ID: 189b0b86-83a3-4c5c-b1f2-6e6d94e8fc4e
finished command with errors

@alexissel Thanks so much! The new error output is really helpful, though not that helpful because this tells me that the request succeeded but themekit could not read it for some reason. The request IDs showed me that the requests completed successfully on Shopify's side as well. This is helpful for #631 as it at least narrows down where the issues are stemming from. Thanks again for all the detail that you provided!

@tanema at least in my case I had issues with large files failing to download. Looking at the file names (images, settings_data.json), I'd assume these are large as well.
BTW Thank you @tanema! I've been using themekit v1.1.3-pre and it works much better! I didn't know I could do theme deploy to add checksums that were missing. My theme deploys are much faster now! Thanks again!

@tanema I started my own investigation about the config/settings_data.json to find out why it doesn't download the file...
So, I opened the file with Shopify's code editor and saw these 3 errors. All 3 images have encrypted URLs.

v1.1.3 has been released with the fix in it. Please reopen this issue if the problem persists


cutorch test segfaults if ran 2 times

Doing this
cutorch.test()
cutorch.test()

produces a segfault (both with CUDA 6.5 and CUDA 7)
th> cutorch.test()
Running 104 tests
_____|__________________________________________________________________________________________________  ==> multi_gpu_copy_noncontig/opt/rocks/cutorch/lib/THC/THCStorage.c(111) : cuda runtime error : an illegal memory access was encountered
stack traceback:
	[C]: at 0x7f24ee186300
	[C]: in function 'collectgarbage'
	/opt/rocks/distro/install/share/lua/5.1/torch/Tester.lua:187: in function 'run'
	/opt/rocks/distro/install/share/lua/5.1/cutorch/test.lua:1555: in function 'f'
	[string "local f = function() return cutorch.test() en..."]:1: in main chunk
	[C]: in function 'xpcall'
	/opt/rocks/distro/install/share/lua/5.1/trepl/init.lua:620: in function 'repl'
	...cks/distro/install/lib/luarocks/rocks/trepl/scm-1/bin/th:185: in main chunk
	[C]: at 0x00406170
Segmentation fault (core dumped)

fixed via https://github.com/torch/cutorch/pull/131


Staring this

Hello, i've just installed the mat-elements packages. How do i use your custom elements? I've tried to look at your website to see if there are any guides but unable to find any. typical link rel doesnt work? am i missing something? i typed the below...
link rel="import" href="expandjs.html"

Hello!
Importing expandjs.html will just import the expand.js library.
To use mat/xp elements you need to import them as needed. Demos here and here
<link rel="import" href="../bower_components/mat-elements/mat-item.html">

Cheers


Add time exceeded flow for fip

#211 https://github.com/ovn-org/ovn/issues/211 Add "ip4.src == <external_ip>" to match for distinguish between external router gateway and floatingIP to handle time exceeded packet.

@ts170710 Thanks for your PR!
I guess I'm not sure I understand how this is supposed to be useful. We already have flows in IP_INPUT stage that check for ip.ttl == {0, 1} && !ip.later_frag packets and reply with Time Exceeded for those. Why isn't that enough?

@ts170710 just in case you missed my question above.
Thanks,
Dumitru

@ts170710 Thanks for your PR!
I guess I'm not sure I understand how this is supposed to be useful. We already have flows in IP_INPUT stage that check for ip.ttl == {0, 1} && !ip.later_frag packets and reply with Time Exceeded for those. Why isn't that enough?

@ts170710 just in case you missed my question above.
Thanks, Dumitru

@dceara Thank you for your comment!
As described in #211, ip.ttl == {0, 1}  treats floatingIPs and router gateway IPs together. When traceroute to cloud instances associated with the router, the router gateway IP address gets replaced by  ip4.dst <-> ip4.src,  resulting in the inability to display the router gateway IP address in the traceroute results.


Issue #203 - Adding ability to set Header and other Config for Axios …

I changed the function from
axios[this.httpMethod]
to
axios.request
This opens up the ability to do more than post though, should we limit it?
I also added a test to test/unit/specs/Vuetable.spec.js and ran it with success while keeping the other tests successful as well (however, the expects had to change because the arguments are different now)
I think you can put the rest on Axios because you are essentially just passing the httpOptions through to it.
Let me know your thoughts.

@tsexton6689 Thanks for the PR. Can you please fork the PR from develop branch and resubmit again?
Using axios.request is nicer. 👍
Right now, the httpMethod is limited to only get and post because Vuetable itself does not really modify the data, it's just needed for display. However, some people prefer to use POST instead of GET because they might feel that it is more secure. That's why the httpMethod is only support for get and post verbs.

I would like to have the ability to set Axios header while using vuetable-2. Looks like there are other people having similar problems. @ratiw - any chance you could just merge this PR directly into the develop branch anyway?
I think it is possible to do it using Github's web interface

Hi, @hanxue and @ratiw, I have been busy with work and haven't had a chance to do this I will try and get it forked from develop today.

@ratiw it looks like this has been done in develop here
      this.fetch(this.apiUrl, this.httpOptions).then(
          success,
          failed
      ).catch(() => failed())
    },
    fetch (apiUrl, httpOptions) {
      return this.httpFetch
          ? this.httpFetch(apiUrl, httpOptions)
          : axios[this.httpMethod](apiUrl, httpOptions)
    },

can we close this PR?

@tsexton6689 That was done by me to close another somewhat related issue.
Is that workable for you as well? If so, I'll just close this PR.

@hanxue The version in the develop branch has a new prop http-fetch. Please see brief description about it here.

Also, please try a new http-fetch prop in the develop branch. See here for a brief description.


vm: implementation that supports account abstraction model (includes signle key wallet)

related https://github.com/spacemeshos/go-spacemesh/issues/3220

state

account extend with Template and State fields.
stub accounts will have both Template and State as nil. When account is spawned with template - Template will store the address of the template, and State will store mutable and immutable state encoded using scale.

address

when account is spawned address is computed as sha256(template_address, spawn_args). this must be taken into an account when configuring the coinbase for a node.

validation and mempool

tx id is computed as a sha256(raw_tx), note that raw_tx is a full body of the transaction including signature.
tx header can't be decoded without executing template code (specifically parse_payload and max_spend methods).
in the upcoming pr - mempool will call into vm to initialize a validation flow (which includes parsing of the tx header). validation api is a 2 step process:

parse. returns transaction header that can be validated against conservative state

verify. executes templates verify method. it is assumed that this method costs much more than parse, so it makes sense to call it only if conservative validation succeeded.

applying transaction

vm.Apply API method accepts layer id, transactions and rewards. Every transaction loads state, executes parse payload, initializes state and executed. Template/Handler API is specified in genvm/core/types.go.
Transaction can be skipped for 3 reasons:

transaction is malformed (expectation is that such transactions should not be added to the block at all)
account is not spawned (same as a nonce mismatch)
nonce doesn't match the expected nonce of the account
can't cover max gas with the account balance

If transaction is skipped it is returned back to the caller (presumably conservative state and the caller is free to do anything about it). Otherwise transaction is consumed and will pay fee to smeshers (in the followup pr for every consumed transaction we will store execution result). Logic for the fee distribution is not changed.

some questions:

address
when account is spawned address is computed as sha256(template_address, spawn_args). this must be taken into an account when configuring the coinbase for a node.

what do you mean by this must be taken into an account when configuring the coinbase for a node
do you expect the coinbase addresses to be stubs?

verify. executes templates verify method. it is assumed that this method costs much more than parse, so it makes sense to call it only if conservative validation succeeded.

so in the future PR, the happy flow is expected to be

conservative state (CS) passes tx to vm for parse()
CS checks balance/nonce+bitfield/[min|max]Layer
CS passes tx to vm for verify()
CS adds tx to mempool

do i understand this correctly?

Transaction can be skipped for 3 reasons:

transaction is malformed (expectation is that such transactions should not be added to the block at all)
account is not spawned (same as a nonce mismatch)
nonce doesn't match the expected nonce of the account
can't cover max gas with the account balance

how about the min/max layer check?

what do you mean by this must be taken into an account when configuring the coinbase for a node
do you expect the coinbase addresses to be stubs?

yes, if they want to spend funds they need to be able to self-spawn themself at some point

so in the future PR, the happy flow is expected to be

in general yes. but i after looking into code i want to keep verify change as a future improvement. i don't want to change conservative cache flow too much

how about the min/max layer check?

i think that the block should be invalid if there is such transaction. why would honest smeshers include such transaction? but it looks like we won't have them atleast for genesis

bors merge


Makers (tsc, tslint) on Windows with custom '_exe' path gives error: the filename directory name or volume label syntax is incorrect

Expected behavior
Maker with let g:neomake_typescript_tslint_exe = <some path> works without error.
Environment
OS: Windows 8.1
nvim --verstion:
NVIM v0.2.2
Build type: RelWithDebInfo
LuaJIT 2.0.5
Compilation: C:/msys64/mingw64/bin/gcc.exe -Wconversion -DNVIM_MSGPACK_HAS_FLOAT32 -DNVIM_UNIBI_HAS_VAR_FROM -O2 -g -DMIN_LOG_LEVEL=3 -Og -g -Wall -Wextra -pedantic -Wno-unused-parameter -Wstrict-prototypes -std=gnu99 -Wimplicit-fallthrough -D__USE_MINGW_ANSI_STDIO -D_WIN32_WINNT=0x0600 -Wvla -fdiagnostics-color=auto -DINCLUDE_GENERATED_DECLARATIONS -IC:/projects/neovim/build/config -IC:/projects/neovim/src -IC:/projects/neovim/.deps/usr/include -IC:/projects/neovim/.deps/usr/include -IC:/msys64/mingw64/include -IC:/projects/neovim/.deps/usr/include -IC:/projects/neovim/.deps/usr/include -IC:/projects/neovim/.deps/usr/include -IC:/msys64/mingw64/include -IC:/msys64/mingw64/include -IC:/projects/neovim/build/src/nvim/auto -IC:/projects/neovim/build/include
Compiled by appveyor@APPVYR-WIN

Features: -acl +iconv -jemalloc +tui
See ":help feature-compile"

   system vimrc file: "$VIM\sysinit.vim"
  fall-back for $VIM: "C:/Program Files (x86)/nvim/share/nvim"

Run :checkhealth for more info

Steps to reproduce
init.vim:
call plug#begin('~/.local/share/nvim/plugged')
Plug 'neomake/neomake'
call plug#end()

let g:neomake_logfile='e:\work\neomake.log'
let g:neomake_typescript_enabled_makers = ['tslint']
let g:neomake_typescript_tslint_exe = 'e:\work\work\project-ui\node_modules\.bin\tslint'

:e e:\work\work\project-ui\Components\Models.ts
:Neomake
In neomake.log you can see error The filename, directory name, or volume label syntax is incorrect.
I've checked that tslint and tsc are actually located at  e:\work\work\project-ui\node_modules\.bin.
I've found this error message in #720, but still don't figured out what from there can help me.

neomake.log
14:52:19 [D      ] [1.-:1:1] Running makers: tslint.
14:52:19 [D +0.01] Defining sign: sign define neomake_file_err texthl=NeomakeErrorSign text=тЬЦ.
14:52:19 [D +0.01] Defining sign: sign define neomake_project_err texthl=NeomakeErrorSign text=тЬЦ.
14:52:19 [D      ] Defining sign: sign define neomake_file_warn texthl=NeomakeWarningSign text=тЪа.
14:52:19 [D      ] Defining sign: sign define neomake_project_warn texthl=NeomakeWarningSign text=тЪа.
14:52:19 [D      ] Defining sign: sign define neomake_file_info texthl=NeomakeInfoSign text=тД╣.
14:52:19 [D      ] Defining sign: sign define neomake_project_info texthl=NeomakeInfoSign text=тД╣.
14:52:19 [D      ] Defining sign: sign define neomake_file_msg texthl=NeomakeMessageSign text=тЮд.
14:52:19 [D      ] Defining sign: sign define neomake_project_msg texthl=NeomakeMessageSign text=тЮд.
14:52:19 [D      ] [1.1:1:1] Setting neomake_automake_tick.
14:52:19 [V      ] [1.1:1:1] Starting async job [string]: "e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\src\tsconfig.json" "e:\work\work\project-ui\src\Components\Models.ts".
14:52:19 [D      ] [1.1:1:1] cwd: e:\work\work\project-ui\src (changed).
14:52:19 [D +0.04] [1.1:1:1] stderr: tslint: ['The filename, directory name, or volume label syntax is incorrect.
', ''].
14:52:19 [D      ] [1.1:1:1] exit: tslint: 1.
14:52:19 [D      ] [1.1:1:1] tslint: processing 1 lines of output.
14:52:19 [D      ] [1.1:1:1] Creating location list.
14:52:19 [D      ] [1.1:1:1] Processing 1 entries.
14:52:19 [D      ] [1.-:1:1] File-level errors cleaned.
14:52:19 [D      ] [1.1:1:1] Skipped 1 entries without bufnr: [{'lnum': 0, 'bufnr': 0, 'col': 0, 'valid': 0, 'vcol': 0, 'nr': -1, 'type': 'W', 'maker_name': 'tslint', 'pattern': '', 'text': 'The filename, directory name, or volume label syntax is incorrect.'}].
14:52:19 [D      ] [1.1:1:1] tslint: completed with exit code 1.
14:52:19 [D      ] [1.1:1:1] Cleaning jobinfo.
14:52:19 [D      ] [1.-:1:1] Cleaning make info.
14:53:01 [D +1644] Calling VimLeave.
15:03:28 [D  +668] Calling VimLeave.
15:03:37 [D      ] [1.-:1:1] Running makers: tslint.
15:03:37 [D      ] Defining sign: sign define neomake_file_err texthl=NeomakeErrorSign text=тЬЦ.
15:03:37 [D      ] Defining sign: sign define neomake_project_err texthl=NeomakeErrorSign text=тЬЦ.
15:03:37 [D      ] Defining sign: sign define neomake_file_warn texthl=NeomakeWarningSign text=тЪа.
15:03:37 [D      ] Defining sign: sign define neomake_project_warn texthl=NeomakeWarningSign text=тЪа.
15:03:37 [D      ] Defining sign: sign define neomake_file_info texthl=NeomakeInfoSign text=тД╣.
15:03:37 [D      ] Defining sign: sign define neomake_project_info texthl=NeomakeInfoSign text=тД╣.
15:03:37 [D      ] Defining sign: sign define neomake_file_msg texthl=NeomakeMessageSign text=тЮд.
15:03:37 [D      ] Defining sign: sign define neomake_project_msg texthl=NeomakeMessageSign text=тЮд.
15:03:37 [D      ] [1.1:1:1] Setting neomake_automake_tick.
15:03:37 [V      ] [1.1:1:1] Starting async job [string]: "e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\src\tsconfig.json" "e:\work\work\project-ui\src\Components\Models.ts".
15:03:37 [D      ] [1.1:1:1] cwd: e:\work\work\project-ui\src (changed).
15:03:37 [D +0.03] [1.1:1:1] stderr: tslint: ['The filename, directory name, or volume label syntax is incorrect.
', ''].
15:03:37 [D      ] [1.1:1:1] exit: tslint: 1.
15:03:37 [D      ] [1.1:1:1] tslint: processing 1 lines of output.
15:03:37 [D      ] [1.1:1:1] Creating location list.
15:03:37 [D +0.01] [1.1:1:1] Processing 1 entries.
15:03:37 [D      ] [1.-:1:1] File-level errors cleaned.
15:03:37 [D      ] [1.1:1:1] Skipped 1 entries without bufnr: [{'lnum': 0, 'bufnr': 0, 'col': 0, 'valid': 0, 'vcol': 0, 'nr': -1, 'type': 'W', 'maker_name': 'tslint', 'pattern': '', 'text': 'The filename, directory name, or volume label syntax is incorrect.'}].
15:03:37 [D      ] [1.1:1:1] tslint: completed with exit code 1.
15:03:37 [D      ] [1.1:1:1] Cleaning jobinfo.
15:03:37 [D      ] [1.-:1:1] Cleaning make info.

Same error with tsc
...
let g:neomake_typescript_enabled_makers = ['tsc']
let g:neomake_typescript_tsc_exe = 'e:\work\work\project-ui\node_modules\.bin\tsc'
...

15:13:42 [D      ] [1.-:1:1] Running makers: tsc.
15:13:42 [D      ] Defining sign: sign define neomake_file_err texthl=NeomakeErrorSign text=тЬЦ.
15:13:42 [D      ] Defining sign: sign define neomake_project_err texthl=NeomakeErrorSign text=тЬЦ.
15:13:42 [D      ] Defining sign: sign define neomake_file_warn texthl=NeomakeWarningSign text=тЪа.
15:13:42 [D      ] Defining sign: sign define neomake_project_warn texthl=NeomakeWarningSign text=тЪа.
15:13:42 [D      ] Defining sign: sign define neomake_file_info texthl=NeomakeInfoSign text=тД╣.
15:13:42 [D      ] Defining sign: sign define neomake_project_info texthl=NeomakeInfoSign text=тД╣.
15:13:42 [D      ] Defining sign: sign define neomake_file_msg texthl=NeomakeMessageSign text=тЮд.
15:13:42 [D      ] Defining sign: sign define neomake_project_msg texthl=NeomakeMessageSign text=тЮд.
15:13:42 [V      ] [1.1:1:1] Starting async job [string]: "e:\work\work\project-ui\node_modules\.bin\tsc" --noEmit --watch false --pretty false --project "e:\work\work\project-ui\src\tsconfig.json".
15:13:42 [D      ] [1.1:1:1] cwd: e:\work.
15:13:42 [D +0.03] [1.1:1:1] stderr: tsc: ['The filename, directory name, or volume label syntax is incorrect.
', ''].
15:13:42 [D      ] [1.1:1:1] exit: tsc: 1.
15:13:42 [D      ] [1.1:1:1] tsc: processing 1 lines of output.
15:13:42 [D      ] [1.1:1:1] Creating location list.
15:13:42 [D      ] [1.1:1:1] Processing 1 entries.
15:13:42 [D      ] [1.-:1:1] File-level errors cleaned.
15:13:42 [D      ] [1.1:1:1] Skipped 1 entries without bufnr: [{'lnum': 0, 'bufnr': 0, 'col': 0, 'valid': 0, 'vcol': 0, 'nr': -1, 'type': 'W', 'maker_name': 'tsc', 'pattern': '', 'text': 'The filename, directory name, or volume label syntax is incorrect.'}].
15:13:42 [D      ] [1.1:1:1] tsc: completed with exit code 1.
15:13:42 [D      ] [1.1:1:1] Cleaning jobinfo.
15:13:42 [D      ] [1.-:1:1] Cleaning make info.

Output from (verbose) NeomakeInfo

Neomake information (use ":verbose NeomakeInfo" for extra output)
Enabled makers
For the current filetype ("typescript", used with :Neomake):

tslint

args: ['--project', 'e:\work\work\project-ui\src\tsconfig.json']
auto_enabled: 0
cwd: 'e:\work\work\project-ui\src'
errorformat: '%-G,%EERROR: %f[%l, %c]: %m,%E%f[%l, %c]: %m'
exe: 'e:\work\work\project-ui\node_modules.bin\tslint'
NOTE: you can define g:neomake_typescript_enabled_makers to configure it (or b:neomake_typescript_enabled_makers).

Default maker settings:

output_stream: 'both'
remove_invalid_entries: 0
buffer_output: 1

For the project (used with :Neomake!):
None.
NOTE: you can define g:neomake_enabled_makers to configure it.
Settings
New-style (dict, overrides old-style)
g:neomake: unset
b:neomake: unset

Old-style
g:neomake_logfile = 'e:\work\neomake.log'
g:neomake_place_signs = 1
g:neomake_typescript_enabled_makers = ['tslint']
g:neomake_typescript_tslint_exe = 'e:\work\work\project-ui\node_modules\.bin\tslint'

makeprg=make

Does running the command manually work?
"e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\src\tsconfig.json" "e:\work\work\project-ui\src\Components\Models.ts"
IIRC this might be related to special things with NodeJS on Windows etc.

@blueyed,
it works.

The log says "Starting async job [string]:" - I do not remember currently if this ("string") is expected on Windows.
Relevant code: https://github.com/neomake/neomake/blob/b64402dcc4eaaf6e88accc2d9a73cec37872e490/autoload/neomake/compat.vim#L165-L174
For Vim it wraps it into a shell (below).
Can you experiment with the following?
Save it to a .vim file and then :source %:
call jobstart('"e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\src\tsconfig.json" "e:\work\work\project-ui\src\Components\Models.ts"',
\ {'on_stdout': {->execute('echom string(a:000)', 1)},
\  'on_stderr': {->execute('echom string(a:000)', 1)}})
messages

And then change the first arg to use a list only, and wrapped in a shell.
Testing if Neomake works on Vim might also be helpful.

Also: what is the output of :echom string([&shell, &shellcmdflag, &shellslash])? (missing from the issue template)

I think jobstart('string') behaves as system(), so you could also try that to figure out what quoting / wrapping in a shell is needed (it does not require a callback).

And then change the first arg to use a list only, and wrapped in a shell.

How to do that?
call jobstart('"e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\src\tsconfig.json" "e:\work\work\project-ui\src\Components\Models.ts"',
\ {'on_stdout': {->execute('echom string(a:000)', 1)},
\  'on_stderr': {->execute('echom string(a:000)', 1)}})
messages

this gives me:
[1, ['<91>⠪᪠ 訡   䠩,    ⪥ ⮬.^M', ''], 'stderr']
Actually it's looks like (github hiding most of the symbols):

Changing encoding to Windows 866 (it's cmd.exe default encoding) gives some russian symbols:
[1, ['<91>такска шиб   фай,    тке том.^M', ''], 'stderr']
I can guess they are part of the 'Синтаксическая ошибка в имени файла, имени папки или метке тома.'
What means The filename, directory name, or volume label syntax is incorrect - this is the same error message from top post.
And by the way in the top post there was that error message in russian, I just translated it.
:echom string([&shell, &shellcmdflag, &shellslash]) gives:
['cmd.exe', '/c', 0]
I've tried to remove quoting ", that surrounds the commands in string:
call jobstart('e:\work\work\project-ui\node_modules\.bin\tslint --project e:\work\work\project-ui\src\tsconfig.json e:\work\work\project-ui\src\Components\Models.ts',
\ {'on_stdout': {->execute('echom string(a:000)', 1)},
\  'on_stderr': {->execute('echom string(a:000)', 1)}})
messages

And it gives tslint working output:
[2, ['', 'ERROR: e:/work/work/project-ui/src/Components/Models.ts[...ui/src/Components/Models.ts[30, 5]: Missing semicolon', '', ''], 'stdout']

Tried with Vim 8 and with same steps to reproduce that described above in top post.
neovim.log:
15:09:08 [D      ] [1.-:3:1] Running makers: tslint.
15:09:08 [D +0.02] Defining sign: sign define neomake_file_err texthl=NeomakeErrorSign text=?.
15:09:08 [D      ] Defining sign: sign define neomake_project_err texthl=NeomakeErrorSign text=?.
15:09:08 [D      ] Defining sign: sign define neomake_file_warn texthl=NeomakeWarningSign text=?.
15:09:08 [D +0.04] Defining sign: sign define neomake_project_warn texthl=NeomakeWarningSign text=?.
15:09:08 [D      ] Defining sign: sign define neomake_file_info texthl=NeomakeInfoSign text=?.
15:09:08 [D      ] Defining sign: sign define neomake_project_info texthl=NeomakeInfoSign text=?.
15:09:08 [D      ] Defining sign: sign define neomake_file_msg texthl=NeomakeMessageSign text=?.
15:09:08 [D      ] Defining sign: sign define neomake_project_msg texthl=NeomakeMessageSign text=?.
15:09:08 [D      ] [1.1:3:1] Setting neomake_automake_tick.
15:09:08 [V +0.05] [1.1:3:1] Starting async job [string]: C:\Windows\system32\cmd.exe /c "e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\src\tsconfig.json" "e:\work\work\project-ui\src\Components\Models.ts".
15:09:08 [D      ] [1.1:3:1] cwd: e:\work\work\project-ui (changed).
15:09:08 [D +0.03] [1.1:3:1] Vim job: {'status': 'run', 'stoponexit': 'term', 'exitval': 0, 'exit_cb': 0, 'channel': channel 1 open, 'process': 5764}.
15:09:08 [D      ] [1.1:3:1] Vim channel: {'status': 'open', 'id': 1, 'in_io': 'pipe', 'in_timeout': 2000, 'err_mode': 'RAW', 'in_status': 'open', 'out_io': 'pipe', 'err_io': 'pipe', 'err_status': 'open', 'out_timeout': 2000, 'out_status': 'open', 'out_mode': 'RAW', 'in_mode': 'RAW', 'err_timeout': 2000}.
15:09:08 [D +0.11] [1.1:3:1] stderr: tslint: ['The filename, directory name, or volume label syntax is incorrect.', ''].
15:09:08 [D +0.11] [1.1:3:1] exit: tslint: 1.
15:09:08 [D +0.02] [1.1:3:1] tslint: processing 1 lines of output.
15:09:08 [D      ] [1.1:3:1] Creating location list.
15:09:08 [D +0.02] [1.1:3:1] Processing 1 entries.
15:09:08 [D      ] [1.-:3:1] File-level errors cleaned.
15:09:08 [D      ] [1.1:3:1] Skipped 1 entries without bufnr: [{'lnum': 0, 'bufnr': 0, 'col': 0, 'valid': 0, 'vcol': 0, 'nr': -1, 'type': 'W', 'maker_name': 'tslint', 'pattern': '', 'text': 'The filename, directory name, or volume label syntax is incorrect.'}].
15:09:08 [D      ] [1.1:3:1] tslint: completed with exit code 1.
15:09:08 [D      ] [1.1:3:1] Cleaning jobinfo.
15:09:08 [D      ] [1.-:3:1] Cleaning make info.

And it make sense. I've tried to perform directly in cmd.exe:
C:\Windows\system32\cmd.exe /c "e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\src\tsconfig.json" "e:\work\work\project-ui\src\Components\Models.ts"
What gives me The filename, directory name, or volume label syntax is incorrect..
But with:
C:\Windows\system32\cmd.exe /c "e:\work\work\project-ui\node_modules\.bin\tslint --project e:\work\work\project-ui\src\tsconfig.json e:\work\work\project-ui\src\Components\Models.ts"
or with:
C:\Windows\system32\cmd.exe /c ""e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\src\tsconfig.json" "e:\work\work\project-ui\src\Components\Models.ts""
I get:
ERROR: e:/work/work/project-ui/src/Components/Models.ts[21, 4]: Missing semicolon
ERROR: e:/work/work/project-ui/src/Components/Models[30, 5]: Missing semicolon

Thanks for the detailed info - have to look into it more in depth.
For now some quotes from justinmk on IRC:

quoting on windows is completely up to the program being called
also if you are invoking nodejs, most likely you want a shell since IIRC nodejs is invoked via a *.cmd script
:h jobstart has a note about windows
using a list is definitely preferred where possible, but for .cmd stuff you would need [&shell, &scf, 'foo.cmd'] (and I don't remember if it searches $PATH)
string is probably best for this case, where "shell magic" is desired

How to wrap in a shell?

Like you did, by prepending &shell / &shellcmdflag - but there are two variants: using a list or a string.

"gives [5, ['', '', ''], 'stdout']
call jobstart([&shell, &shellcmdflag, 'e:\work\work\project-ui\node_modules.bin\tslint --project e:\work\work\project-ui\src\tsconfig.json e:\work\work\project-ui\src\Components\Models.ts'],

From quickly skimming it sounds like the following should be used with both Neovim/Vim on Windows:
    function! neomake#compat#get_argv(exe, args, args_is_list) abort
        let prefix = &shell.' '.&shellcmdflag.' '
        if a:args_is_list
            let args = neomake#utils#ExpandArgs(a:args)
            if a:exe ==# &shell && get(args, 0) ==# &shellcmdflag
                " Remove already existing &shell/&shellcmdflag from e.g. NeomakeSh.
                let argv = join(map(copy(args[1:]), 'neomake#utils#shellescape(v:val)'))
            else
                let argv = join(map(copy([a:exe] + args), 'neomake#utils#shellescape(v:val)'))
            endif
            let argv = neomake#utils#shellescape(argv)
        else
            let argv = a:exe . (empty(a:args) ? '' : ' '.a:args)
            if argv[0:len(prefix)-1] ==# prefix
                return argv
            endif
        endif
        return prefix.argv
    endfunction

Can you check this (by editing autoload/neomake/compat.vim)?

So as I understand, the command parameter for cmd.exe have to be wrapped to quote ".

Looks like it, yes - and makes sense.

I placed you code like this:
diff --git a/autoload/neomake/compat.vim b/autoload/neomake/compat.vim
index 18d422f..50b3ba4 100644
--- a/autoload/neomake/compat.vim
+++ b/autoload/neomake/compat.vim
@@ -160,67 +160,25 @@ function! neomake#compat#globpath_list(path, pattern, suf) abort
     return split(globpath(a:path, a:pattern, a:suf), '\n')
 endfunction

-if has('nvim')
-    if neomake#utils#IsRunningWindows()
-        function! neomake#compat#get_argv(exe, args, args_is_list) abort
-            if a:args_is_list
-                let args = neomake#utils#ExpandArgs(a:args)
-                " Convert it to a string to handle PATHEXT (e.g. .cmd files).
-                " This might be skipped when `exepath(a:exe)[-4:] == '.exe'`,
-                " but not worth it probably (and more fragile in the end?!).
-                return join(map(copy([a:exe] + args), 'neomake#utils#shellescape(v:val)'))
-            endif
-            return a:exe . (empty(a:args) ? '' : ' '.a:args)
-        endfunction
-    else
-        function! neomake#compat#get_argv(exe, args, args_is_list) abort
-            if a:args_is_list
-                return [a:exe] + neomake#utils#ExpandArgs(a:args)
-            endif
-            return a:exe . (empty(a:args) ? '' : ' '.a:args)
-        endfunction
-    endif
-elseif neomake#has_async_support()  " Vim-async.
-    if neomake#utils#IsRunningWindows()
-        " Windows needs a shell to handle PATH/%PATHEXT% etc.
-        function! neomake#compat#get_argv(exe, args, args_is_list) abort
-            let prefix = &shell.' '.&shellcmdflag.' '
-            if a:args_is_list
-                let args = neomake#utils#ExpandArgs(a:args)
-                if a:exe ==# &shell && get(args, 0) ==# &shellcmdflag
-                    " Remove already existing &shell/&shellcmdflag from e.g. NeomakeSh.
-                    let argv = join(map(copy(args[1:]), 'neomake#utils#shellescape(v:val)'))
-                else
-                    let argv = join(map(copy([a:exe] + args), 'neomake#utils#shellescape(v:val)'))
-                endif
-            else
-                let argv = a:exe . (empty(a:args) ? '' : ' '.a:args)
-                if argv[0:len(prefix)-1] ==# prefix
-                    return argv
-                endif
-            endif
-            return prefix.argv
-        endfunction
+function! neomake#compat#get_argv(exe, args, args_is_list) abort
+    let prefix = &shell.' '.&shellcmdflag.' '
+    if a:args_is_list
+        let args = neomake#utils#ExpandArgs(a:args)
+        if a:exe ==# &shell && get(args, 0) ==# &shellcmdflag
+            " Remove already existing &shell/&shellcmdflag from e.g. NeomakeSh.
+            let argv = join(copy(args[1:]))
+        else
+            let argv = join(copy([a:exe] + args))
+        endif
+        "let argv = neomake#utils#shellescape(argv)
     else
-        function! neomake#compat#get_argv(exe, args, args_is_list) abort
-            if a:args_is_list
-                return [a:exe] + neomake#utils#ExpandArgs(a:args)
-            endif
-            " Use a shell to handle argv properly (Vim splits at spaces).
-            let argv = a:exe . (empty(a:args) ? '' : ' '.a:args)
-            return [&shell, &shellcmdflag, argv]
-        endfunction
-    endif
-else
-    " Vim (synchronously), via system().
-    function! neomake#compat#get_argv(exe, args, args_is_list) abort
-        if a:args_is_list
-            let args = neomake#utils#ExpandArgs(a:args)
-            return join(map(copy([a:exe] + args), 'neomake#utils#shellescape(v:val)'))
+        let argv = a:exe . (empty(a:args) ? '' : ' '.a:args)
+        if argv[0:len(prefix)-1] ==# prefix
+            return argv
         endif
-        return a:exe . (empty(a:args) ? '' : ' '.a:args)
-    endfunction
-endif
+    endif
+    return prefix.argv
+endfunction

 if v:version >= 704 || (v:version == 703 && has('patch831'))
     function! neomake#compat#gettabwinvar(t, w, v, d) abort

When running :Neomake your code gives:
18:50:04 [D +73.9] [6.-:1:1] Running makers: tslint.
18:50:04 [D +0.01] [6.6:1:1] Setting neomake_automake_tick.
18:50:04 [V      ] [6.6:1:1] Starting async job [string]: cmd.exe /c """e:\work\work\project-ui\node_modules\.bin\tslint"" --project ""e:\work\work\project-ui\src\tsconfig.json"" ""e:\work\work\project-ui\src\Components\Models.ts""".
18:50:04 [D      ] [6.6:1:1] cwd: e:\work\work\project-ui(changed).
18:50:04 [D +0.07] [6.6:1:1] stderr: tslint: ['Ошибка в синтаксисе команды.
', ''].
18:50:04 [D +0.02] [6.6:1:1] stdout: tslint: ['
', 'e:\work\work\project-ui>@IF EXIST "e:\work\work\project-ui\"e:\work\work\project-ui\node_modules\.bin\\node.exe" (
', ''].
18:50:04 [D      ] [6.6:1:1] exit: tslint: 255.
18:50:04 [D      ] [6.6:1:1] tslint: processing 2 lines of output.
18:50:04 [D      ] [6.6:1:1] Creating location list.
18:50:04 [D      ] [6.6:1:1] Processing 1 entries.
18:50:04 [D      ] [6.-:1:1] File-level errors cleaned.
18:50:04 [D      ] [6.6:1:1] Skipped 1 entries without bufnr: [{'lnum': 0, 'bufnr': 0, 'col': 0, 'valid': 0, 'vcol': 0, 'nr': -1, 'type': 'W', 'maker_name': 'tslint', 'pattern': '', 'text': 'e:\work\work\project-ui>@IF EXIST "e:\work\work\project-ui\"e:\work\work\project-ui\node_modules\.bin\\node.exe" ('}].
18:50:04 [D      ] [6.6:1:1] tslint: processing 1 lines of output.
18:50:04 [D      ] [6.6:1:1] Processing 1 entries.
18:50:04 [D      ] [6.6:1:1] Skipped 1 entries without bufnr: [{'lnum': 0, 'bufnr': 0, 'col': 0, 'valid': 0, 'vcol': 0, 'nr': -1, 'type': 'W', 'maker_name': 'tslint', 'pattern': '', 'text': 'Ошибка в синтаксисе команды.'}].
18:50:04 [D      ] [6.6:1:1] tslint: completed with exit code 255.
18:50:04 [D      ] [6.6:1:1] Cleaning jobinfo.
18:50:04 [D      ] [6.-:1:1] Cleaning make info.
18:50:13 [D +9.14] Calling VimLeave.
18:50:24 [D      ] [1.-:2:1] Running makers: tslint.
18:50:24 [D +0.01] Defining sign: sign define neomake_file_err texthl=NeomakeErrorSign text=тЬЦ.
18:50:24 [D      ] Defining sign: sign define neomake_project_err texthl=NeomakeErrorSign text=тЬЦ.
18:50:24 [D      ] Defining sign: sign define neomake_file_warn texthl=NeomakeWarningSign text=тЪа.
18:50:24 [D      ] Defining sign: sign define neomake_project_warn texthl=NeomakeWarningSign text=тЪа.
18:50:24 [D      ] Defining sign: sign define neomake_file_info texthl=NeomakeInfoSign text=тД╣.
18:50:24 [D      ] Defining sign: sign define neomake_project_info texthl=NeomakeInfoSign text=тД╣.
18:50:24 [D      ] Defining sign: sign define neomake_file_msg texthl=NeomakeMessageSign text=тЮд.
18:50:24 [D      ] Defining sign: sign define neomake_project_msg texthl=NeomakeMessageSign text=тЮд.
18:50:24 [D      ] [1.1:2:1] Setting neomake_automake_tick.
18:50:53 [D +29.6] Calling VimLeave.
18:50:53 [D      ] [-.-:-:1] Cancelling make.
18:50:53 [D      ] [1.-:2:1] Cleaning make info.

Looks like extra wrapping with " quote for arguments.
I just tried to remove neomake#utils#shellescape:
And got:
18:51:00 [D      ] [1.-:1:1] Running makers: tslint.
18:51:00 [D      ] Defining sign: sign define neomake_file_err texthl=NeomakeErrorSign text=тЬЦ.
18:51:00 [D      ] Defining sign: sign define neomake_project_err texthl=NeomakeErrorSign text=тЬЦ.
18:51:00 [D      ] Defining sign: sign define neomake_file_warn texthl=NeomakeWarningSign text=тЪа.
18:51:00 [D      ] Defining sign: sign define neomake_project_warn texthl=NeomakeWarningSign text=тЪа.
18:51:00 [D      ] Defining sign: sign define neomake_file_info texthl=NeomakeInfoSign text=тД╣.
18:51:00 [D      ] Defining sign: sign define neomake_project_info texthl=NeomakeInfoSign text=тД╣.
18:51:00 [D      ] Defining sign: sign define neomake_file_msg texthl=NeomakeMessageSign text=тЮд.
18:51:00 [D      ] Defining sign: sign define neomake_project_msg texthl=NeomakeMessageSign text=тЮд.
18:51:00 [D      ] [1.1:1:1] Setting neomake_automake_tick.
18:51:00 [V      ] [1.1:1:1] Starting async job [string]: cmd.exe /c e:\work\work\project-ui\node_modules\.bin\tslint --project e:\work\work\project-ui\src\tsconfig.json e:\work\work\project-ui\src\Components\Models.ts.
18:51:00 [D      ] [1.1:1:1] cwd: e:\work\work\project-ui (changed).
18:51:13 [D +13.6] [1.1:1:1] stdout: tslint: ['', 'ERROR: src/Components/Models.ts[21, 4]: Missing semicolon', 'ERROR: src/Components/Models.ts[30, 5]: Missing semicolon', ''].
18:51:13 [D +0.02] [1.1:1:1] exit: tslint: 2.
18:51:13 [D      ] [1.1:1:1] tslint: processing 3 lines of output.
18:51:13 [D      ] [1.1:1:1] Creating location list.
18:51:13 [D      ] [1.1:1:1] Processing 2 entries.
18:51:13 [D      ] [1.-:1:1] File-level errors cleaned.
18:51:13 [D      ] [-.-:1:1] Placing sign: sign place 5000 line=21 name=neomake_file_err buffer=1.
18:51:13 [D      ] [-.-:1:1] Placing sign: sign place 5001 line=30 name=neomake_file_err buffer=1.
18:51:13 [D +0.03] [1.1:1:1] tslint: completed with exit code 2.
18:51:13 [D      ] [1.1:1:1] Cleaning jobinfo.
18:51:13 [D      ] [1.-:1:1] Cleaning make info.

And I can see (red signs at the left side of window and loclist is populated) that neomake working with that my addition, but it's just some guess editing. I mean I don't know how it can affect to plugin in general.

What does "Ошибка в синтаксисе команды." mean?

Does cmd.exe /c """"e:\work\work\project-ui\node_modules\.bin\tslint"" --project ""e:\work\work\project-ui\src\tsconfig.json"" ""e:\work\work\project-ui\src\Components\Models.ts"""" work?
Does cmd.exe /c "e:\path with space\work\work\project-ui\node_modules\.bin\tslint" --help work?

Maybe the quotes for the exe are bad in case they are not needed - but it works when there are spaces in the path.
The issue is that quoting should be used (since paths might contain spaces), and currently it gets quoted since \ is not skipped (but only /).

What does "Ошибка в синтаксисе команды." mean?

Forgot about that. It means "Error in command syntax."

No, it doesn't.
It gives The filename, directory name, or volume label syntax is incorrect.
No, it doesn't.
cmd.exe /c "e:\work\work\project ui\node_modules\.bin\tslint" --help gives
"e:\work\work\project" is not recognized as an internal or external command, program or batch file executable.

What about `cmd.exe /c """e:\work\work\project ui\node_modules.bin\tslint"" --help" ?

cmd.exe /c """e:\work\work\project ui\node_modules.bin\tslint"" --help"
gives almost the same
"""e:\work\work\project" is not recognized as an internal or external command, program or batch file executable.

I don't get why here are double quotes for paths?

I think the inner double quotes need to be escaped by another quote, i.e. inside of quotes "" is needed for a single quote.
But that's wrong apparently - got that from :h shellescape(), but it only applies to args, not what follows "/c".
cmd /c """echo"" 1" works for me on Windows 7 (yes, you made me boot a windows from the "recovery"/kept-for-BIOS-updates partition.. ;))
cmd /c "dir "with space"" works (full listing), but cmd /c ""dir" "with space""also works (but only displays file names) - similar to usingdirvs"dir"` directly.
"with space/test.cmd" can be run as "with space/test.cmd", cmd /c "with space/test.cmd" and `cmd  /c ""with space/test.cmd"".
"with space/test,cmd" "1 2" works, but cmd /c "with space" "1 2" does not ('the command "with" is wrong).
This needs cmd /c ""with space/test.cmd" "1 2"" (and will echo %1 as "1 2").
I will create a PR to test/debug/fix this - thanks for all your input so far!

btw: maybe you are interested in / can help with https://github.com/neomake/neomake/issues/724, too?

Please try #1894.

With updated neomake (from master) I'me getting the same error The filename, directory name, or volume label syntax is incorrect..
I've tried cmd.exe /c "e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\tsconfig.json" "e:\work\work\project-ui\src\actions\index.ts". directly in cmd.exe and got same error.
As I understand " quotes for entire cmd.exe argument is miising.
This is working:
cmd.exe /c ""e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\tsconfig.json" "e:\work\work\project-ui\src\actions\index.ts"".
14:01:22 [D      ] [1.-:1:1] Running makers: tslint.
14:01:22 [D +0.01] Defining sign: sign define neomake_file_err texthl=NeomakeErrorSign text=тЬЦ.
14:01:22 [D      ] Defining sign: sign define neomake_project_err texthl=NeomakeErrorSign text=тЬЦ.
14:01:22 [D      ] Defining sign: sign define neomake_file_warn texthl=NeomakeWarningSign text=тЪа.
14:01:22 [D      ] Defining sign: sign define neomake_project_warn texthl=NeomakeWarningSign text=тЪа.
14:01:22 [D      ] Defining sign: sign define neomake_file_info texthl=NeomakeInfoSign text=тД╣.
14:01:22 [D      ] Defining sign: sign define neomake_project_info texthl=NeomakeInfoSign text=тД╣.
14:01:22 [D      ] Defining sign: sign define neomake_file_msg texthl=NeomakeMessageSign text=тЮд.
14:01:22 [D      ] Defining sign: sign define neomake_project_msg texthl=NeomakeMessageSign text=тЮд.
14:01:22 [D +0.10] [1.1:1:1] Setting neomake_automake_tick.
14:01:22 [V      ] [1.1:1:1] Starting async job [string]: cmd.exe /c "e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\tsconfig.json" "e:\work\work\project-ui\src\actions\index.ts".
14:01:22 [D      ] [1.1:1:1] cwd: e:\work\work\project-ui (changed).
14:01:22 [D +0.21] [1.1:1:1] stderr: tslint: ['The filename, directory name, or volume label syntax is incorrect.
', ''].
14:01:22 [D +0.01] [1.1:1:1] exit: tslint: 1.
14:01:22 [D      ] [1.1:1:1] tslint: processing 1 lines of output.
14:01:22 [D      ] [1.1:1:1] Creating location list.
14:01:22 [D      ] [1.1:1:1] Processing 1 entries.
14:01:22 [D      ] [1.-:1:1] File-level errors cleaned.
14:01:22 [D      ] [1.1:1:1] Skipped 1 entries without bufnr: [{'lnum': 0, 'bufnr': 0, 'col': 0, 'valid': 0, 'vcol': 0, 'nr': -1, 'type': 'W', 'maker_name': 'tslint', 'pattern': '', 'text': 'The filename, directory name, or volume label syntax is incorrect.'}].
14:01:22 [D +0.01] [1.1:1:1] tslint: completed with exit code 1.
14:01:22 [D      ] [1.1:1:1] Cleaning jobinfo.
14:01:22 [D      ] [1.-:1:1] Cleaning make info.

@kreedz
Sorry, missed the last part.
Can you test https://github.com/neomake/neomake/pull/1896, please?

I've tested it with:
Plug 'neomake/neomake', {'branch': 'win2'}

I've checked that plugin switched on win2 branch and that it was updated.
And got same error, now it looks like that extra quotes around paths is the issue.
18:01:11 [D +22.2] [3.-:2:1] Running makers: tslint.
18:01:11 [D      ] [3.3:2:1] Setting neomake_automake_tick.
18:01:11 [V      ] [3.3:2:1] Starting async job: cmd.exe /c """e:\work\work\project-ui\node_modules\.bin\tslint"" --project ""e:\work\work\project-ui\tsconfig.json"" ""e:\work\work\project-ui\src\actions\index.ts""".
18:01:11 [D      ] [3.3:2:1] cwd: e:\work\work\project-ui (changed).
18:01:11 [D +0.04] [3.3:2:1] stderr: tslint: ['The filename, directory name, or volume label syntax is incorrect.
', ''].
18:01:11 [D      ] [3.3:2:1] exit: tslint: 1.
18:01:11 [D      ] [3.3:2:1] tslint: processing 1 lines of output.
18:01:11 [D      ] [3.3:2:1] Creating location list.
18:01:11 [D      ] [3.3:2:1] Processing 1 entries.
18:01:11 [D      ] [3.-:2:1] File-level errors cleaned.
18:01:11 [D      ] [3.3:2:1] Skipped 1 entries without bufnr: [{'lnum': 0, 'bufnr': 0, 'col': 0, 'valid': 0, 'vcol': 0, 'nr': -1, 'type': 'W', 'maker_name': 'tslint', 'pattern': '', 'text': 'The filename, directory name, or volume label syntax is incorrect.'}].
18:01:11 [D      ] [3.3:2:1] tslint: completed with exit code 1.
18:01:11 [D      ] [3.3:2:1] Cleaning jobinfo.
18:01:11 [D      ] [3.-:2:1] Cleaning make info.

lt should not have quotes around it, but wrap it with parenthesis - so something is not working as expected then still.
Does cmd.exe /c ("e:\work\work\project-ui\node_modules\.bin\tslint" --project "e:\work\work\project-ui\tsconfig.json" "e:\work\work\project-ui\src\actions\index.ts") work?
If not, does cmd.exe /c (e:\work\work\project-ui\node_modules\.bin\tslint --project "e:\work\work\project-ui\tsconfig.json" "e:\work\work\project-ui\src\actions\index.ts") work?

Ok, 80e6707 was obviously not correct/complete, and the "Starting async job" message is not really correct, since it shellescapes every arg, which results in something not really used by Neovim then (I assume).  I've changed this recently to make it easier to copy'n'paste.
Will update the PR later.

@blueyed
Both of them works


Open Declaration in Package View

When I'm in SQL Script view and press F4, DBeaver jumps to the definition of whatever is under my cursor. But when I'm in the package body view, F4 does nothing. Is this intended behavior, am I missing something?
(DB: Oracle, DBeaver version 4.1.10)

This is actually a bug, thanks for report. It'll be fixed in the next version.
As a workaround you can use CTRL+mouse click to open object declaration.

Hi,
The feature discribed by Jayshua still does not work in the most recent version.
I'm in SQL Script view and press F4, DBeaver jumps to the definition of whatever is under my cursor. But when I'm in the package body view, F4 does nothing. Is this intended behavior, am I missing something?
If you can fixed it, thanks a lot !
(DB: Oracle, DBeaver version 23.3.3)


chore: decrease severity of valid schema/content example validation

This PR changes the severity of the spectral rules oas2-valid-definition-example and oas3-valid-schema-example from error to warn. The spectral rule oas3-valid-oas-content-example was also enabled with a warn severity, since the linked issue reported our validator was not reporting warnings in request or response body examples.
The src/.defaultsForValidator.js file was also updated to match the values in our default spectral ruleset.
fixes: https://github.ibm.com/arf/planning-sdk-squad/issues/2303

:tada: This PR is included in version 0.33.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 0.33.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Class to disable column behaviour

Sometimes is better not to use a defined col, but rather a static width for example for a logo and the website title, that can behave as columns with certain screen sizes (like mobile ones). The actual way of doing it is replacing min and max width css values with an @media.

I don’t think we’d add additional classes to disable core behaviors. If you need grid behaviors, but without the .col classes, I’d suggest checking out the flexbox utility classes.


Fix to failing Vagrant script

Added some more Ansible related documentation. Docs also say how to run a playbook, skip tasks by using the --skip-tags flag and specify the tag names of the tasks to skip
Added tags to tasks that are necessary for the moment
Added multiple tags on certain tasks to gain granularity of the steps in the task to point to them

Follow instructions, create and setup vagrant box, Inside the vagrant box, run:
ansible-playbook -i /path/to/hosts -s ubuntu.yml --skip-tags "install_zulu,jenkins_authorized_key,nagios_add_key,add_zeus_user_key"

Added "install_zulu", as I was still running into the Azul issue with the latest changes, in case one of you already has a fix, I'll be happy to rebase my repo.
Fixes #64

If there is another way to do it, I'll be happy to know.

Each of the changes you have proposed in the ubuntu.yml playbook would also have to be applied to all other playbooks to allow the same fuction to work everywhere.
Another why to do this would be to use --extra-vars as we do in AWX (Ansible-Tower).
Although the name says 'extra' it will also override variables defined inside the playbook.
The command line would looks like this:
ansible-playbook -i /path/to/hosts -s ubuntu.yml --extra-vars "Jenkins_Username=MY_USER Jenkins_User_SSHKey=/PATH/id_rsa.pub Nagios_Plugins=Disabled Slack_Notification=Disabled Superuser_Account=Disabled"

Editting the /etc/ansible/hosts file isn't required. This is a global setting.
Another option is to create a folder to work out of it, adding the hosts file and any other config, along with your playbook. ex:
/my_ansible_temp/
-- hosts
-- ansible.cfg
-- My_Play.yml

The hosts file could look like this
My_Host ansible_ssh_host=HOSTNAME_or_IPADDRESS ansible_ssh_pass=MYPASSWORD

Warning: ^this stores the password in plain text on the local machine
or you could use --ask-pass or have perviously setup ssh keys to that machine

Thanks @bblondin  I will give it a whirl, and change the PR accordingly.
@karianna or someone else, just for a sanity check, could you pls try to run the script and see if it works for you once I have a new update in.

@neomatrix369 I still get:
TASK [Install zulu-9-amd64] *************************************************************************************************************************************************************************************************************
failed: [localhost] (item=[u'zulu-9-amd64']) => {"changed": false, "failed": true, "item": ["zulu-9-amd64"], "msg": "No package matching 'zulu-9-amd64' is available"}

@karianna try running with the below:
ansible-playbook -i hosts -s ubuntu.yml --extra-vars "Jenkins_Username=MY_USER Jenkins_User_SSHKey=/PATH/id_rsa.pub Nagios_Plugins=Disabled Slack_Notification=Disabled Superuser_Account=Disabled" --skip-tags="install_zulu"

Although we will then need to supply it with a /PATH/ida_rsa.pub key or you will get the below:
TASK [Set authorized key for Jenkins user] *******************************************************************************************************
 [WARNING]: Unable to find '/PATH/id_rsa.pub' in expected paths.

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'file'. Error was a <class 'ansible.errors.AnsibleError'>, original message: could not locate file in lookup: /PATH/id_rsa.pub"}

@bblondin @karianna
Just created an ssh public-private key-pair in the vagrant box using:
ssh-keygen -t rsa -b 4096 -C<EMAIL_ADDRESS>
And re-ran the above script like this:
ansible-playbook -i hosts -s ubuntu.yml --extra-vars "Jenkins_Username=jenkins Jenkins_User_SSHKey=/home/vagrant/.ssh/id_rsa.pub Nagios_Plugins=Disabled Slack_Notification=Disabled Superuser_Account=Disabled" --skip-tags="install_zulu"

and got

PLAY RECAP ***************************************************************************************************************************************
localhost                  : ok=38   changed=23   unreachable=0    failed=0

I hope this works for you, if it does, I can remove some of the added tags and amend the docs.

The ubuntu.yml playbook was built in the below time:
real	11m9.965s
user	6m26.744s
sys	2m15.952s

@neomatrix369 I got
[DEPRECATION WARNING]: The sudo command line option has been deprecated in favor of the "become" command line arguments. This feature will be removed in version 2.6. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
 [WARNING]: Unable to parse /vagrant/playbooks/hosts as an inventory source

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

@karianna you ran the below:
ansible-playbook -i hosts -s ubuntu.yml --extra-vars "Jenkins_Username=jenkins Jenkins_User_SSHKey=/home/vagrant/.ssh/id_rsa.pub Nagios_Plugins=Disabled Slack_Notification=Disabled Superuser_Account=Disabled" --skip-tags="install_zulu"

and actually got:
[DEPRECATION WARNING]: The sudo command line option has been deprecated in
favor of the "become" command line arguments. This feature will be removed in
version 2.6. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
 [WARNING]: Unable to parse /vagrant/playbooks/hosts as an inventory source

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: Could not match supplied host pattern, ignoring: all

 [WARNING]: provided hosts list is empty, only localhost is available

PLAY [all] *********************************************************************
skipping: no hosts matched

PLAY RECAP *********************************************************************

because you didn't point to the correct hosts file, there was no hosts in the directory where your ran this command.
I would suggest create a hosts file in the /vagrant/playbooks folder (inside the VM), it should contain the same contents as mentioned in the README docs.
I should have mentioned that step about the hosts file again.

@neomatrix369 That works for me now thanks. I think we should update that doc as well to install the hosts file where you suggested.

WIll apply these changes to the other playbooks where applicable i.e.

aix.yml
centos_7.yml
rhel.yml
sles.yml
ubuntu.yml
windows.yml

via another PR.

@karianna Both those can be done. let me amend the scripts and give it a whirl

@karianna Done! Can you pls run them and see if works for you too


Tech debt: add from __future__ import annotations in the Data Class feature

Why is this needed?
Reference: https://github.com/aws-powertools/powertools-lambda-python/issues/4607
Add from future import annotations to all files, including examples
Which area does this relate to?
Static typing
Solution
No response
Acknowledgment

[X] This request meets Powertools for AWS Lambda (Python) Tenets
[ ] Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

Closing as completed!


Probability of false positives?

Hi, many thanks to you for providing such a tool!
I've been using this for quite a long time, and my impression is that it's got high precision (when it caught something it's usually correct), but low recall (still many cheaters got away with it). So when the scanner caught a case, I tend to be confident about it.
Recently, one demo was scanned and showed more than 20 lines of aim types with different weapons, from the start to the end of the demo. From my own experience, I'm confident that this is cheating but the guy insisted his innocence. Since I didn't really read the source code, I want to ask about whether it's indeed possible for this case to be a false positive?
Although I seriously doubt this, would some specific user config setting or change of weapon/skin/player lead to such a result? Or any specific pattern of moving the mouse that could trigger the scanner?
Would like to hear if you have similar experiences, thanks in advance.

@randomwalker42 You can share demo file or scan results screenshot ?


feature Forbid alter table's storage vault related properties

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


Changing the name of a project prevent it to run.

Godot version:

godot 3.2.3 stable
OS/device including version:

windows 1
Issue description:

i ran a project in editor after a name change, expected it to run but it didn't show the window
to the game
Steps to reproduce:
change a name of a demo/working project, then run it in editor

I can't reproduce this. Could you be more specific what the "name change" is, please? (project name, scene name, node name, etc...)
Also please upload a minimal reproduction project or, if using a demo, specify which one you're using, and also exact steps to reproduce your issue.

Closing due to lack of response, if you provide the requested information this can be reopened


Validate protostar.toml

Pretty print, an error about missing required field in protostar.toml.

Is it still relevant?

Maybe in the future.


wrong link in css property wiki ?

in the CSS property wiki, there is the link
https://drafts.csswg.org/css-display-3/#the-display-properties
but for me, it's a link to an unknown page of https://drafts.csswg.org

same here : http://lexbor.com/docs/lexbor/#css

@vtorri
All links are working correctly.

when i click on it, either the site saying the doc is not found, and this morning, see img attached:

either at home, or at work

and now working... maybe some problems at w3c


about test results

Thank you for your work. It seems like in your work, the model trained on CNNSpot does not play well on diffusion images in Imagenet and CelebA. I have three questions about it.

Why does this model trained on CNN play well on diffusion images in LSUN-Bedroom? Since this test dataset is also based on diffusion model, why the acc is so high against other two dataset?
I trained your model on CNNSpot and tested it on CelebA, but I got a much higher result than the result in your work.
dalle2    acc:0.998
if           acc:0.966
mid       acc:0.998
sdv2      acc:0.816
Did I do something wrong?
Could you please give me the diffusion dataset based on Imagenet, since in the DiffusionForensics, the writer did not give so many classes.

Hello,

The CelebA dataset primarily consists of faces, so the model trained on CNNSpot may struggle to achieve high accuracy. ImageNet contains a more complex variety of objects. The difference in content between the training and testing images leads to the performance gap.

No idea.

Please refer to https://genimage-dataset.github.io/


Update Composer dependencies (2021-02-09-00-07)

Visual regression test passed!
View the visual regression test report


MAINT: integrate: Clean the C code behind odeint.

Argh... I just noticed a mistake in the last commit message.  It should be _odepackmodule.c, not _quadpackmodeul.c.  I'll fix that before this gets merged.

You mean you just want to fix the commit message?

You mean you just want to fix the commit message?

Yes, just the commit message.

In fact, I'll do that right now...

The commit message is fixed.

You tested with numpy 1.10 master I assume, to check the no-deprecations is OK? that's not added in Travis config yet.

Yes, and I just rebuilt and tested it again with numpy 1.10.0.dev0+a866fdd, and all is well.  I didn't get any numpy API warnings for the odepack module, and the integrate tests all pass.

LGTM, in it goes. Thanks Warren.


ci: use vcpkg for openssl on windows

Hoping to find something to help with #261:

switch to using lukka/run-vcpkg
get rid of the llvm dependency on windows (it was not needed)
update openssl dep to 0.10.30

Caching seems to work, so this is good to go unless something unexpected happens.


I am not able to install Carbon 10 in Sketch

 I am not able to install carbon 10 in Sketch
Detailed description

Describe in detail the issue you're having.
I've followed all the steps to install Carbon 10 in Sketch and I am receiving these error message in sketch: "Failed to download the update" (for the carbon design system) and "Library version incompatible" (for the IBM Design Language).

Is this issue related to a specific component?
No.

What version of the Carbon Design System are you using?
Trying to install version 10

What offering/product do you work on? Any pressing ship or release dates we should be aware of?
MyPartnerWorld. Yes, because we are transitioning while it is still in the beginning

Please create a reduced test case in code sandbox
https://codesandbox.io/s/r4zxq9pvq

Additional information

Screenshots or code

Notes

What version of Sketch are you using? Carbon and Libraries require the current version (54.1 as of 4/30/2019)

I just checked and I'm using 52.6. This seems to be the problem.

Thanks,
Isabella Pedreira
Intern, Global Business Partners<EMAIL_ADDRESS>+55 (21) 98847 4838

----- Original message -----From: Chris Connors<EMAIL_ADDRESS>carbon-design-system/carbon-website<EMAIL_ADDRESS>Bellapedreira<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Re: [carbon-design-system/carbon-website] I am not able to install Carbon 10 in Sketch (#1574)Date: Tue, Apr 30, 2019 11:52 AM
What version of Sketch are you using? Carbon and Libraries require the current version (54.1 as of 4/30/2019)
—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or mute the thread.


Zeroconf does not work

[x] I confirm that this is an issue rather than a question.

Bug report
Zeroconf does not work as advertised (pun intended).
Steps to reproduce
Run avahi-browse on the local network, or on the machine hosting Hyperion:
(~) $ avahi-browse -r _hyperiond-http._tcp

... or ...
(~) $ avahi-browse -a | grep -i hyperion

What is expected?
Some form of output (as per diagram shown on zeroconf instructions).
What is actually happening?
No output.
System

Hyperion Server:
- Build:       master (GitHub-f719f0b/838092e-1597257777)
- Build time:  Aug 19 2020 04:01:56
- Git Remote:  https://github.com/hyperion-project/hyperion.ng.git
- Version:     2.0.0-alpha.7
- UI Lang:     auto (BrowserLang: en-US)
- UI Access:   expert
- Avail Capt:  dispmanx,v4l2,framebuffer,qt

Hyperion Server OS:
- Distribution: Raspbian GNU/Linux 10 (buster)
- Arch:         arm
- Kernel:       linux (5.4.51+ (WS: 32))
- Browser:      Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36

Thanks !

@m-seker Thanks!
I took the liberty of doing a build from head with this fix applied, but it does not appear to have fixed the problem, were you able to verify the fix on your end?

@m-seker Thanks!
I took the liberty of doing a build from head with this fix applied, but it does not appear to have fixed the problem, were you able to verify the fix on your end?

Interesting.
┌─[murse@Arch64] - [~/projects/hyperion.ng/build] - [Mon Sep 07, 23:37]
└─[$]> avahi-browse -a
+ docker0 IPv4 Arch64:8092                                   _hyperiond-http._tcp local
+ docker0 IPv4 Arch64:8090                                   _hyperiond-http._tcp local
+ enp0s3 IPv6 Arch64:8092                                   _hyperiond-http._tcp local
+ enp0s3 IPv6 Arch64:8090                                   _hyperiond-http._tcp local
+ enp0s3 IPv4 Arch64:8092                                   _hyperiond-http._tcp local
+ enp0s3 IPv4 Arch64:8090                                   _hyperiond-http._tcp local
+     lo IPv4 Arch64:8092                                   _hyperiond-http._tcp local
+     lo IPv4 Arch64:8090                                   _hyperiond-http._tcp local

Can you make sure -DENABLE_AVAHI=ON while configuring with cmake ?

-- ENABLE_AVAHI = ON

... in the build output. However, might be an issue with how I'm building ... investigating.

Yes, it was a build problem. Got it to work!
However, I note that only _hyperiond-http._tcp works, but _hyperiond-flatbuf._tcp and _hyperiond-json._tcp still do not (it's the latter I'm looking for in my usecase).

Yes, it was a build problem. Got it to work!
However, I note that only _hyperiond-http._tcp works, but _hyperiond-flatbuf._tcp and _hyperiond-json._tcp still do not (it's the latter I'm looking for in my usecase).

I'll have a look after work.

@dermotduffy Can you give another try ?

Looks good! Thank you!
+ eno1 IPv6 pi:8092                               _hyperiond-http._tcp local
+ eno1 IPv6 pi:8090                               _hyperiond-http._tcp local
+ eno1 IPv4 pi:8092                               _hyperiond-http._tcp local
+ eno1 IPv4 pi:8090                               _hyperiond-http._tcp local
+ eno1 IPv6 pi:19400                              _hyperiond-flatbuf._tcp local
+ eno1 IPv4 pi:19400                              _hyperiond-flatbuf._tcp local
+ eno1 IPv6 pi:19444                              _hyperiond-json._tcp local
+ eno1 IPv4 pi:19444                              _hyperiond-json._tcp local


Fix TestFoundation build order issue in Xcode

Currently when building the TestFoundation scheme in Xcode with a clean build directory, the build fails because it attempts to build SwiftXCTest before SwiftFoundation. This fixes the build order in the scheme.

AFAIK CI never touches anything related to this, so there's no point in requesting a build from it.

Ok


Fix the logging in spoof

Currently we're getting:
Retrying [http://activator-with-cc.default.svc.cluster.local?sleep=100 retrying for DNS error: Get http://activator-with-cc.default.svc.cluster.local?sleep=100: dial tcp: lookup activator-with-cc.default.svc.cluster.local on <IP_ADDRESS>:53: no such host]: %!v(MISSING)

Which is not cool. So fix it to report proper string.
URL is known to be non-nil here.
/assign @chizhg @chaodaiG

/lgtm

(ノಠ益ಠ)ノ彡┻━┻
/retest

One more time, before I think that spoof and metrics are related
(ノಠ益ಠ)ノ彡┻━┻
/retest


Feature/lookup external (dev)

Enable AL to lookup its tag values against other external systems to direct the analyst to additional links and information.
Provides an api endpoint to query. This end point will then look up the tags in a micro service that is dedicated to translating from AL to the external system and then returns results in a common manner.
A template for these micro services is provided, and some concrete implementations for common public services.
Requires changes from: https://github.com/CybercentreCanada/assemblyline-base/pull/1169

pipeline tests seem to be failing due to requiring the changes made in the assemblyline-base PR.


SameDiff/ND4J: Intermittent threading issue (CPU backend)

Encountered on CI here:
https://jenkins.konduit.ai/blue/organizations/jenkins/konduitai%2Fdeeplearning4j/detail/PR-330/5/pipeline/672
Also reproduced locally at least once; the vast majority of runs are OK
Edit: increasing the number of threads and runs for SameDiffMultiThreadTests.testMobilenet reliably reproduces this locally.
[2020-03-20T20:42:04.487Z] Running org.nd4j.autodiff.samediff.SameDiffMultiThreadTests
[2020-03-20T20:42:04.487Z] o.n.BaseND4JTest - SameDiffMultiThreadTests.testMobilenet
[2020-03-20T20:42:06.385Z] o.n.a.s.SameDiff - Creating new InferenceSession for thread 2171
[2020-03-20T20:42:06.643Z] o.n.a.s.SameDiff - Creating new InferenceSession for thread 2172
[2020-03-20T20:42:06.643Z] o.n.a.s.SameDiff - Creating new InferenceSession for thread 2173
[2020-03-20T20:42:06.643Z] o.n.a.s.SameDiff - Creating new InferenceSession for thread 2174
[2020-03-20T20:42:06.643Z] o.n.a.s.SameDiff - Creating new InferenceSession for thread 2175
[2020-03-20T20:42:07.576Z] o.n.a.s.SameDiffMultiThreadTests - Error in thread: 2175
[2020-03-20T20:42:07.576Z] java.lang.RuntimeException: Bad datatype B
[2020-03-20T20:42:07.576Z] 	at org.nd4j.nativeblas.Nd4jCpu.setGraphContextOutputBuffer(Native Method)
[2020-03-20T20:42:07.576Z] 	at org.nd4j.linalg.cpu.nativecpu.ops.CpuOpContext.setOutputArray(CpuOpContext.java:117)
[2020-03-20T20:42:07.576Z] 	at org.nd4j.autodiff.samediff.internal.InferenceSession.getAndParameterizeOp(InferenceSession.java:895)
[2020-03-20T20:42:07.576Z] 	at org.nd4j.autodiff.samediff.internal.InferenceSession.getAndParameterizeOp(InferenceSession.java:69)
[2020-03-20T20:42:07.576Z] 	at org.nd4j.autodiff.samediff.internal.AbstractSession.output(AbstractSession.java:384)

OK, so it looks like the same error is happening at different locations:

Nd4jCpu.setGraphContextOutputBuffer
Nd4jCpu.setGraphContextInputBuffer
NativeOpExecutioner.exec

Note that in the exec case here, there's no boolean datatype used - it's just a simple reshape :/
o.n.l.c.n.o.NativeOpExecutioner - Failed to execute op reshape. Attempted to execute with 2 inputs, 1 outputs, 0 targs,0 bargs and 0 iargs. Inputs: [(FLOAT,[1,1001],c), (INT,[2],c)]. Outputs: [(FLOAT,[1,1001],c)]. tArgs: -. iArgs: -. bArgs: -. Op own name: "MobilenetV2/Predictions/Reshape_1". Input var names: [MobilenetV2/Predictions/Softmax, MobilenetV2/Predictions/Shape]. Output var names: [MobilenetV2/Predictions/Reshape_1] - Please see above message (printed out from c++) for a possible cause of error.
o.n.a.s.SameDiffMultiThreadTests - Error in thread: 42 at iteration 7
java.lang.RuntimeException: Bad datatype B
	at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:2059)
	at org.nd4j.linalg.factory.Nd4j.exec(Nd4j.java:6578)
	at org.nd4j.autodiff.samediff.internal.InferenceSession.doExec(InferenceSession.java:496)
	at org.nd4j.autodiff.samediff.internal.InferenceSession.getOutputs(InferenceSession.java:223)
	at org.nd4j.autodiff.samediff.internal.InferenceSession.getOutputs(InferenceSession.java:69)

o.n.a.s.SameDiffMultiThreadTests - Error in thread: 34 at iteration 95
java.lang.RuntimeException: Bad datatype B
	at org.nd4j.nativeblas.Nd4jCpu.setGraphContextInputBuffer(Native Method)
	at org.nd4j.linalg.cpu.nativecpu.ops.CpuOpContext.setInputArray(CpuOpContext.java:109)
	at org.nd4j.linalg.api.ops.BaseOpContext.setInputArrays(BaseOpContext.java:187)
	at org.nd4j.autodiff.samediff.internal.InferenceSession.getAndParameterizeOp(InferenceSession.java:858)
	at org.nd4j.autodiff.samediff.internal.InferenceSession.getAndParameterizeOp(InferenceSession.java:69)

I have also noted that:

it appears to always be the same reshape operation (never seen any other nodes for this).
it is not related to SameDiff memory management (switching to ArrayCloseMemMgr makes no difference)

Here's the calculateOutputShape error in debugger. Note that:

Same "Bad datatype B" error as before
Shape infos look fine (no booleans here; dtype extra matches array datatype - 8192=float, 131072=int)

Looks like it's almost certainly a threading issue... can't reproduce this single threaded after around 85,000 iterations of forward pass through this net.


Resolve bug in lib/post_install.js for node 0.10.x

execSync is replaced with spawn, and the dependency sync-exec is dropped.
Also, npm prune --production is run after the transpilation, to remove
unecessary devDependencies.

Thanks! Included in 0.14.2.


SimpleVertxHttpServerWrapper should not create a HashMap on every request.

SimpleVertxHttpServerWrapper should not create a HashMap on every request.

    private void handleHttpRequest(final HttpServerRequest request) {
        handleHttpRequest(request, new HashMap<>());
    }

Part of the issue is HttpRequest holds a final map.
public class HttpRequest implements Request<Object> {

    private final Map<String, Object> data;

We should probably create a final atomicReference to a Map so that the map can be set by the users. Then add an addDataObject("foo", foo) method that will lazy init the map if needed.

@bbyk


initial parsedObj type definition

The type definition is a bit messy. Could you give us some suggestions?

Citation will be done as Wikipedia style footnote, since we just have a single web page.

Yeah sure, I'll have a look at it now


Style fixes (better rendering, smaller code, less redundancy)

This pull request fixes 4 problems (all of them relatively minor):

It fixes headers rendered in the markdown so they're not squished together. It does this by...
a. Fixing the *.less stylesheet to correctly compile to css that's evaluated only for the first header of the markdown document (rather than for every header -- as it is now).
b. Gave the headers a bit more breathing room so not everything is crammed together (margin-top of 1.4em instead of 1em).

Remove redundant font-family from _markdown.less (it's already specified in the body element in _base.less).

Fix the font-family to use the "system" sans-serif fonts on each particular device to feel more like a native app (it also, consequently, matches the behavior of Github).

Remove the semantic-ui Font-family references so that the font-family from _base.less is respected.

The 3 pictures below show (1) how it was, (2) fixed header spacing, and (3) fixed font family.

I don't like that vendored semantic ui css is changed, it can easily be forgotten to do these changes when upgrading to newer version. Can these changes be made otherwise?

No, I don’t like that either. But unfortunately the semantic UI Library overrides any existing font faces. So, as I see it, we have a few (practical) options:

When the semantic Library is updated either force the person that updates it to remove their font-face css properties (which is all I did to ensure the Gitea font-faces were properly inherited)

Make a script to automate that css change.

Build the Semantic UI css from the .less source code.

Really, no matter what option is ultimately decided upon it requires changing the Semantic UI Library.

I think best option would be building it from less as we are already building our own css from less files

Is there a reason we can't just use !important to override Semantic UI?

We could use semantic-ui's build tool with less files. Would allow changing fonts, colors etc.
So on update we just have to update semantic-ui's less files and build again with our fixed less variables.

I think think for this to merge we need to change semantic-ui building from less files, so currently I'm moving this to 1.4 milestone


Problem startMonitoring not have callback in setMonitoringListener ?

I run example code all not found my becons but Estimote app in store found why?

I had same issue? is there any solution And I'm using android os 6.0 and 7.1.1.

Yes i uesed version 6.0.1

But If your have 7.0 then?

Hello guys,
In order for me to deduce the exact problem, I need any information from your side - which SDK version you use, type of phone, Android version, any code snippets or any data from debug logs (can be enabled via EstimoteSDK.enableDebugLogging(true))
I would also like to know which example code you are using, @T-Jedsada  - there is a chance that this example is outdated and we forgot to update it.
I'm waiting for more details, guys!

OK man to day i will send you the information.