An existing connection was forcibly closed by the remote host

Expected behavior
Netty should not have thrown this error: "An existing connection was forcibly closed by the remote host"
Steps to reproduce
I'm using Netty in our charting library where Netty is used for socket communication between our java and javascript code. It is used to send continuous real-time data and candle data to javascript. Initially, it works fine but after about 2 hours it throws an error. Here's the stacktrace:
java.io.IOException: An existing connection was forcibly closed by the remote host 	at sun.nio.ch.SocketDispatcher.read0(Native Method) 	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43) 	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) 	at sun.nio.ch.IOUtil.read(IOUtil.java:192) 	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) 	at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:288) 	at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1055) 	at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:245) 	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112) 	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:510) 	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:467) 	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:381) 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:353) 	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742) 	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137) 	at java.lang.Thread.run(Thread.java:745)
Minimal yet complete reproducer code (or URL to code)
As per company policy, I can't disclose the code and as it not reproduced by any specific steps so I can't provide a POC. It happens after 2 hours of continuous flooding of data.
Netty version
4.1.0.CR3
JVM version (e.g. java -version)
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
OS version (e.g. uname -a)
Windows 8.1 64 bit

Please update to the latest Netty version (4.1.8) and report back if you still think there is an issue.
based upon the limited information it sounds like your peer is sending a TCP RST. Please verify this with a packet analyzer (e.g. tcpdump, Wireshark, etc..). If this is the case then you need to figure out why your peer is closing the socket.

I think is related with the WebSocketClientHandshakerX
headers.add(Names.HOST, wsURL.getHost() + ':' + wsPort);
I have pulled locally the source code, and removed : wsPort and my websocket connection works.
The Host Header must have only the domain name of the server, and not to include the port.

@radut hmm is this by any chance ipv6 ?

Sure, NO, ipV4 in my test.
It took us a lot of days to find this.
We are connecting to a wamp server using Netty wamp client.

@radut can you tell me what exactly the host header contains when it not works ?

Host: api-ws.xxxxx.com:80
was fix by using
Host: api-ws.xxxxx.com
which works,
PS : xxxxx.com does not exists, it was sanitized

You could check this : http://www.websocket.org/echo.html
click connect,
network tab  and check for WS, request headers :
Host:us40.zopim.com

@radut hmm... while the port is "optional" for the default port (80 / 443) it is definitely allowed to be added: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.23 . Sounds more like a bug in the server that you are trying to talk to. That said I think we could also detect if we need to add it or not and just not add it for port 80 and 443.

I understand that it was put there for an purpose, but I debugged why an wamp 2 api works in browser and from the Jawampa+Netty, does not.
Found out this difference, that the browser does not send the port name.
In my opinion if the browser connects, sets headers like this, netty should expose an api to be able to do the same, by either letting Api overwrite host header with any value that user wants. (He will put the domain without port).
And the netty implementation remains almoust the same, to respect rfc.

@radut gimme a few minutes...

@radut PTAL https://github.com/netty/netty/pull/6463

However, I wonder if the owner of the issue could reproduce this. In our case this error was at handshake/connecting. I noticed now that he says something about after 2 hours.

Thanks for your points.
In my case, it doesn't look like a handshake issue because the connection is setup successfully and the problem occurs after, roughly around, 2 hours.
I tried Scottmitch's suggestion and updated the Netty version. Since then I'm waiting for the issue to reproduce. Looks like it is fixed now. I'll monitor it for further more days and update here with some more information if any.

@normanmaurer I also checked the source code for WebSocketClientHandshaker08,07 and 00 and they have only the host added, no port in "HOST" header.

@radut so I merged the websockets thingy. @kuldeep992 please open a new issue with more infos if you still think this is a netty bug.


[BUG] website title

Describe the bug
The current title is showing Vite + React + TS.
Expected behavior
It should be AssistMe.

Hi,
I would like to work on this issue. Can you please assign me ?

Hi, I would like to work on this issue. Can you please assign me ?

Thank you for your interest, but this issue has already been resolved and waiting for review in the pull request #28 .


Drifter requires ansible 2.1.1, host has <IP_ADDRESS>

Recently I have following error with new drifter installations:
Vagrant has automatically selected the compatibility mode '2.0'
according to the Ansible version installed (<IP_ADDRESS>).

Alternatively, the compatibility mode can be specified in your Vagrantfile:
https://www.vagrantup.com/docs/provisioning/ansible_common.html#compatibility_mode

The requested Ansible version (2.1.1) was not found on the guest.
Please check the Ansible installation on your Vagrant guest system (currently: <IP_ADDRESS>),
or adapt the provisioner `version` option in your Vagrantfile.
See https://docs.vagrantup.com/v2/provisioning/ansible_common.html#version
for more information.

I can avoid that by changing the variable ansible_version to  "<IP_ADDRESS>" in virtualization/drifter/Vagrantfile. (I wonder it has to be that specific, aren't "2.1.1" and "<IP_ADDRESS>" kind of the same?)
Anyway, is anybody else having this problem? Could we change it to "<IP_ADDRESS>"? Or are there other solutions to this?

This will be resolved in the next release. See also #198

Fixed in 1.5.0.


Fix reading of test file on Python 2.7.

Add tests for test file usage.
open builtin on Python 2.7 doesn't take a encoding parameter. io.open works on both 2.7 and 3.7. I've removed the < 2.6 conditional import since we haven't supported 2.5 for unittests in a while.
Fix #5534

When running tests locally:


[Bug] Trys to setup fswatchers in my home dir on start

VSCode Version: 1.38.1
Commit: b37e54c98e1a74ba89e03073e5a3761284e3ffb0
Date: 2019-09-11T13:30:08.229Z
Electron: 4.2.10
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Linux x64 5.0.0-29-generic

So let me explain the error more well. When i open vscode on my desktop with no additional folder it starts to watch for my whole home dir as default folder maybe? it should start watching only opened directorys.
its also verifyied that this happens when i open a folder .
  ERR [File Watcher (node.js)] Failed to watch /home/frank/.config/Code/User for changes using fs.watch() (Error: ENOSPC: no space left on device, watch '/home/frank/.config/Code/User')
/usr/share/code/reso…esktop.main.js:3579 [Extension Host] debugger listening on port 57389
/usr/share/code/reso…esktop.main.js:1504   ERR Error: [File Watcher (node.js)] Failed to watch /home/frank/.config/Code/logs/20190926T115203/output_4_20190928T094647 for changes using fs.watch() (Error: ENOSPC: no space left on device, watch '/home/frank/.config/Code/logs/20190926T115203/output_4_20190928T094647')
/usr/share/code/reso…esktop.main.js:1504   ERR [File Watcher (node.js)] Failed to watch /home/frank/.config/Code/logs/20190926T115203/output_4_20190928T094647 for changes using fs.watch() (Error: ENOSPC: no space left on device, watch '/home/frank/.config/Code/logs/20190926T115203/output_4_20190928T094647')
/usr/share/code/reso…esktop.main.js:1504   ERR Error: [File Watcher (chokidar)] Inotify limit reached (ENOSPC)
/usr/share/code/reso…esktop.main.js:1504   ERR [File Watcher (chokidar)] Inotify limit reached (ENOSPC)
```

Steps to Reproduce:

1. start vscode
2. open new window
3. open folder
4. watch console output

<!-- Launch with `code --disable-extensions` to check. -->
Does this issue occur when all extensions are disabled?: Yes

@frank-dspeed does this help https://code.visualstudio.com/docs/setup/linux#_visual-studio-code-is-unable-to-watch-for-file-changes-in-this-large-workspace-error-enospc ?

@bpasero you don't understand right look at the path it trys to setup watchers for my home dir with when i have nothing opened in my workspace no folder is open ?

and we have the same bug in the code-server builds from cdr there we also use 1.38.1 https://github.com/cdr/code-server/issues/1036

So let me explain the error more well. When i open vscode on my desktop with no additional folder it starts to watch for my whole home dir as default folder maybe? it should start watching only opened directorys.

Moving to @sandy081 to explain why we watch files/folders in home dir. This seems to be related to output or config watching.

Let me know if it makes sense

@sandy081 i understand that but i am not happy with that design. VSCode killed already a ssd from me by accident.
we can close this i opend a new one with a solution for this.
https://github.com/microsoft/vscode/issues/81702


Custom Sound Notifications Plugin

This plugin is meant to provide the user with the ability to get sound notifications on a game message. The user can add a list of keywords to a text box in a comma separated list. The plugin will check game messages and play a sound if it matches on of the keywords.

This is already something that can be done with Watchdog.

Thank you, I can stop development on this now.


Instaloader not downloading (KeyErrors)

Describe the bug
Downloads interrupted since today
To Reproduce
python3 /instaloader/instaloader.py USER --login=LOGIN --stories --tagged --no-videos --no-captions --no-metadata-json --highlights --fast-update --dirname-pattern /downloads/{target}/ --sessionfile /settings/.instaloader-session

Error messages and tracebacks
Loaded session from /settings/.instaloader-session.
Logged in as LOGINUSER.
/downloads/X//X_profile_pic.jpg already exists
Retrieving tagged posts for profile DOWNLOADUSER.
Retrieving posts from profile DOWNLOADUSER.
[  1/  X] Traceback (most recent call last):
  File "/instaloader/structures.py", line 157, in _field
    d = d[key]
KeyError: 'edge_sidecar_to_children'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/instaloader/instaloader.py", line 6, in <module>
    main()
  File "/instaloader/instaloader/__main__.py", line 414, in main
    storyitem_filter_str=args.storyitem_filter)
  File "/instaloader/instaloader/__main__.py", line 189, in _main
    download_stories, fast_update, post_filter, storyitem_filter)
  File "/instaloader/instaloader/instaloader.py", line 1039, in download_profiles
    downloaded = self.download_post(post, target=profile_name)
  File "/instaloader/instaloader/instaloader.py", line 466, in download_post
    for sidecar_node in post.get_sidecar_nodes():
  File "/instaloader/instaloader/structures.py", line 227, in get_sidecar_nodes
    for edge in self._field('edge_sidecar_to_children', 'edges'):
  File "/instaloader/instaloader/structures.py", line 160, in _field
    d = self._full_metadata
  File "/instaloader/instaloader/structures.py", line 143, in _full_metadata
    self._obtain_metadata()
  File "/instaloader/instaloader/structures.py", line 135, in _obtain_metadata
    self._full_metadata_dict = pic_json['entry_data']['PostPage'][0]['graphql']['shortcode_media']
KeyError: 'graphql'

Instaloader version
v4.2.9
This Command I am using worked fine for the last months with no problem, so I guess Instagram changed something in their backend?!

That error has already been corrected. You must update this file:
https://github.com/instaloader/instaloader/blob/master/instaloader/instaloadercontext.py

That error has already been corrected. You must update this file:
https://github.com/instaloader/instaloader/blob/master/instaloader/instaloadercontext.py

Thanks! Haven't seen that there was an updated file. Fixed Error!

@ungenio Hi, how can I solve this problem with windows 10? I install instaloader with "pip install" and I don't know where I should change instaloadercontext.py. Thank you.
I have the same problem:
  File "instaloader\structures.py", line 157, in _field
KeyError: 'edge_sidecar_to_children'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "__main__.py", line 446, in <module>
  File "__main__.py", line 442, in __main
  File "__main__.py", line 417, in main
  File "__main__.py", line 192, in _main
  File "instaloader\instaloader.py", line 1039, in download_profiles
  File "instaloader\instaloader.py", line 466, in download_post
  File "instaloader\structures.py", line 227, in get_sidecar_nodes
  File "instaloader\structures.py", line 160, in _field
  File "instaloader\structures.py", line 143, in _full_metadata
  File "instaloader\structures.py", line 135, in _obtain_metadata
KeyError: 'graphql'
[2632] Failed to execute script __main__

Instaloader version v4.2.9
Windows 10 64 Bit

It seems it is probably on the folder you just run, inside of the "instaloader" folder should have a instaloadercontext.py file. I replaced mine with the one above and it worked just fine.
If the file above doesn't exist, mine was in:
/c/Users/<username>/AppData/Local/Programs/Python/<pythonversion>/lib/site-packages/instaloader/instaloadercontext.py

this problem still happens to me

Thank you for this update!

@shadow443 the update must be done manually. Download this file (right click on RAW and download):
https://github.com/instaloader/instaloader/blob/master/instaloader/instaloadercontext.py
and paste it here:
C:\Users\<username>\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\instaloader\instaloadercontext.py

@ungenio  Thank you. It is working for me.

@ungenio I am on linux. I performed a deep search for instaloadercontext.py (sudo find / -name instaloadercontext.py). Where I found this file in 2 places. To make sure, I replaced both files. Still, I am experiencing the problem with both video posts and multi-photo posts. I am downloading favorited posts by my user (instaloader :saved).


Support query strings on static content

Hi there,
A really simple test I did was to just stuff query string parameters on to html pages as I make requests and I'm realizing that this leads to 404 errors.
Can you confirm if no-query-string support is a feature or a bug?

Ignore query string when serving static content. (Dev server and API calls should proxy the http request as-is, including query string)
Also need to print a disclaimer that the CLI is an emulator and might not match the cloud environment 100%. Will create a separate issue to track.

@anthonychu  the code responsible for parsing query strings is located here: https://github.com/Azure/static-web-apps-cli/blob/efc9cad2906309398cf7393b93444a4af1202379/src/msha/middlewares/request.middleware.ts#L258-L270

@anthonychu It looks like we already ignore query string when serving static content (and our e2e tests are passing). We will probably need more details on how to reproduce this issue.
@rwinwilliams are you able to share your app so I can try it locally? Or, can you run the SWA CLI using the --verbose=silly and copy/paste the logs of the request that leads to the 404 page? Thank you.
FYI, the debugging logs might look like the following:
debug logs using --verbose=silly

Hey @manekinekko, here's the output when --verbose=silly is on:
Detected runtime: [object Object]
[swa] - config:
[swa]   - appBuildCommand: npm run build --if-present
[swa]   - apiBuildCommand: npm run build --if-present
[swa]   - appLocation: C:\Development\del\TestSWA
[swa]   - apiLocation: C:\Development\del\TestSWA\api
[swa]   - outputLocation: C:\Development\del\TestSWA
[swa]  - files:
[swa]    - 0: C:\Development\del\TestSWA.github\workflows\azure-static-web-apps-white-glacier-094809c10.yml
[swa] - ssl:
[swa]   - 0: false
[swa]   - 1:
[swa]   - 2:
[swa] - env:
[swa]   - SWA_CLI_DEBUG: silly
[swa]   - SWA_CLI_API_PORT: 7071
[swa]   - SWA_CLI_APP_LOCATION: C:\Development\del\TestSWA
[swa]   - SWA_CLI_OUTPUT_LOCATION: C:\Development\del\TestSWA
[swa]   - SWA_CLI_API_LOCATION: C:\Development\del\TestSWA\api
[swa]   - SWA_CLI_ROUTES_LOCATION: .
[swa]   - SWA_CLI_HOST: <IP_ADDRESS>
[swa]   - SWA_CLI_PORT: 4280
[swa]   - SWA_WORKFLOW_FILES: C:\Development\del\TestSWA.github\workflows\azure-static-web-apps-white-glacier-094809c10.yml
[swa]   - SWA_CLI_APP_SSL: false
[swa]   - SWA_CLI_APP_SSL_CERT:
[swa]   - SWA_CLI_APP_SSL_KEY:
[swa]   - SWA_CLI_STARTUP_COMMAND:
[swa] - commands:
[swa]   - swa: node<EMAIL_ADDRESS>[swa]   - api: cd "C:\Development\del\TestSWA\api" && func start --cors "*" --port 7071
[swa]   - run:
[api] Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET
[api] Copyright (C) Microsoft Corporation.    Looking at the log messages, it looks like this might be an older version of the CLI. Can you check the version of the CLI? Ensure you're running 0.6.0.
swa --version

My swa --version output was 0.4.0.
So I ran, "npm install -g @azure/static-web-apps-cli"
Got it up to 0.6.0. Same issue. Restarted vscode. No change.
Going off to restart.

Please share the app you’re testing with and repro steps.
It’s possible this is an issue on Windows. We’ll test this on a Windows machine.

It's a private repo, @anthonychu, I can add you/your team.

@irwinwilliams are you able to print the version of the SWA installed on your system, using the command swa -v.
Thank you.

@manekinekko, sure, it's now 0.6.0.

Oh! Are you still having the same issue with 0.6.0?

Hey, @manekinekko, yep.

Thank you @irwinwilliams !
@anthonychu I confirm I am not seeing this issue on Macos. So this might be Windows-related.

Hello @irwinwilliams
Could please confirm if there is the following navigationFallback setting in your staticwebapp.config.json?
If there isn't, please add this and try again.
  "navigationFallback": {
    "rewrite": "/index.html",
    "exclude": ["*.{txt}"]
  },

As far as I investigated, it seems that the query string is not handled appropriately if there is no this setting.

By comparing between manekinekko's log and irwinwilliams' log, some different points were found.

Request path

/folder/index.html in manekinekko's log
/index.html or / in irwinwilliams' log

Route matching result
In manekinekko's log, there are following lines.

[swa] turning glob expression into valid RegExp
[swa]  - glob: /folder/*.{html,xml}
[swa]  - regexp: /^\/folder\/.*(html|xml)$/
[swa]  - isMatch: true
[swa]  - doesRequestPathMatchWildcardRoute: true

This is caused by the following route setting.
https://github.com/Azure/static-web-apps-cli/blob/34f96cd9475b38168cd4edd1ec9ade3d7b6728ea/cypress/fixtures/static/staticwebapp.config.json#L33-L36
@manekinekko Could you please confirm if this issue was reproduced after removing the route setting from your staticwebapp.config.json?
And @irwinwilliams Could you please confirm if this issue was mitigated by adding the following route setting into your staticwebapp.config.json?
  {
    "route": "/*.{html,xml}",
    "rewrite": "/"
  },

Thanks

Thank you so much @horihiro for taking the time to investigate this. Please note that the logs I shared are those of our E2E application and do not reflect @irwinwilliams configuration. I simply shared my logs as an example.
Also, I don't think the following rule is necessary:
{
    "route": "/*.{html,xml}",
    "rewrite": "/"
},

Because this rule will rewrite ALL *.html and *.xml (eg: /foo/bar/baz.html) requests to /index.html, which can be problematic.
@irwinwilliams are you able to share your staticwebapp.config.json if you are using one, so @horihiro and I can give investigate the issue?

@manekinekko Do we have any way to handle query strings in any pages without the navigationFallback setting?

@horihiro for static content, query params are ignored by the CLI. See: https://github.com/Azure/static-web-apps-cli/blob/0abec9309688e991400b38d21d973f68145033ce/src/msha/middlewares/request.middleware.ts#L266-L269

Thank you!
I also confirmed the access to /folder/index.html?query=value returns 404 if the following route setting is removed from staticwebapp.configjson in this.
{
    "route": "/folder/*.{html,xml}",
    "rewrite": "/folder/"
}

/folder/index.html?query=value -> 404 Not Found
/folder/index.html -> 200 OK

Is this the expected behavior of the CLI?

@horihiro I am confused because I can't reproduce this issue. As you can see from the screenshot and logs, the file /folder/index.html?query=value is served as expected!

```
[swa] --------------------------------------------------------
[swa] ------------------- processing route -------------------
[swa] --------------------------------------------------------
[swa] processing /folder/index.html?key=value
[swa] checking for matching route
[swa] check if request match route
[swa]  - route: /redirect/301
[swa]  - wildcard: false
[swa]  - alternateRequestPath: /folder/index.html
[swa] check if request match route
[swa]  - route: /redirect/302
[swa]  - wildcard: false
[swa]  - alternateRequestPath: /folder/index.html
[swa] check if request match route
[swa]  - route: /redirect/*
[swa]  - wildcard: true
[swa] checking wildcard route
[swa]  - glob: /redirect/*
[swa]  - pathBeforeWildcard: /redirect/
[swa]  - substring don't match. Exit
[swa]  - alternateRequestPath: /folder/index.html
[swa] checking wildcard route
[swa]  - glob: /redirect/*
[swa]  - pathBeforeWildcard: /redirect/
[swa]  - substring don't match. Exit
[swa] check if request match route
[swa]  - route: /*.google
[swa]  - wildcard: true
[swa] checking wildcard route
[swa]  - glob: /*.google
[swa]  - pathBeforeWildcard: /
[swa] turning glob expression into valid RegExp
[swa]  - glob: /*.google
[swa]  - regexp: /^\/.*google$/
[swa]  - isMatch: false
[swa]  - alternateRequestPath: /folder/index.html
[swa] checking wildcard route
[swa]  - glob: /*.google
[swa]  - pathBeforeWildcard: /
[swa] turning glob expression into valid RegExp
[swa]  - glob: /*.google
[swa]  - regexp: /^\/.*google$/
[swa]  - isMatch: false
[swa] check if request match route
[swa]  - route: /*.{jpg}
[swa]  - wildcard: true
[swa] checking wildcard route
[swa]  - glob: /*.{jpg}
[swa]  - pathBeforeWildcard: /
[swa] turning glob expression into valid RegExp
[swa]  - glob: /*.{jpg}
[swa]  - regexp: /^\/.*(jpg)$/
[swa]  - isMatch: false
[swa]  - alternateRequestPath: /folder/index.html
[swa] checking wildcard route
[swa]  - glob: /*.{jpg}
[swa]  - pathBeforeWildcard: /
[swa] turning glob expression into valid RegExp
[swa]  - glob: /*.{jpg}
[swa]  - regexp: /^\/.*(jpg)$/
[swa]  - isMatch: false
[swa] check if request match route
[swa]  - route: *.foo
[swa]  - wildcard: true
[swa] checking wildcard route
[swa]  - glob: *.foo
[swa]  - pathBeforeWildcard:
[swa] turning glob expression into valid RegExp
[swa]  - glob: *.foo
[swa]  - regexp: /^.*foo$/
[swa]  - isMatch: false
[swa]  - alternateRequestPath: /folder/index.html
[swa] checking wildcard route
[swa]  - glob: *.foo
[swa]  - pathBeforeWildcard:
[swa] turning glob expression into valid RegExp
[swa]  - glob: *.foo
[swa]  - regexp: /^.*foo$/
[swa]  - isMatch: false
[swa] check if request match route
[swa]  - route: /*.{png,gif}
[swa]  - wildcard: true
[swa] checking wildcard route
[swa]  - glob: /*.{png,gif}
[swa]  - pathBeforeWildcard: /
[swa] turning glob expression into valid RegExp
[swa]  - glob: /*.{png,gif}
[swa]  - regexp: /^\/.*(png|gif)$/
[swa]  - isMatch: false
[swa]  - alternateRequestPath: /folder/index.html
[swa] checking wildcard route
[swa]  - glob: /*.{png,gif}
[swa]  - pathBeforeWildcard: /
[swa] turning glob expression into valid RegExp
[swa]  - glob: /*.{png,gif}
[swa]  - regexp: /^\/.*(png|gif)$/
[swa]  - isMatch: false
[swa] check if request match route
[swa]  - route: /folder/*.{html,xml}
[swa]  - wildcard: true
[swa] checking wildcard route
[swa]  - glob: /folder/*.{html,xml}
[swa]  - pathBeforeWildcard: /folder/
[swa] turning glob expression into valid RegExp
[swa]  - glob: /folder/*.{html,xml}
[swa]  - regexp: /^\/folder\/.*(html|xml)$/
[swa]  - isMatch: true
[swa]  - doesRequestPathMatchWildcardRoute: true
[swa] - matchingRouteRule:
[swa]   - route: /folder/*.{html,xml}
[swa]   - rewrite: /folder/
[swa] checking auth request
[swa]  - not an auth request
[swa] checking function request
[swa]  - not a function request
[swa] checking HTTP method: GET
[swa]  - method is valid (allow-list: GET,HEAD,OPTIONS)
[swa] checking for query params
[swa] checking rewrite auth login request
[swa] checking rewrite auth logout request
[swa] checking authorizations for route
[swa]  - shouldCheckRoles: false
[swa]  - shouldLookupAuthCookie: false
[swa]  - access authorized
[swa] using userConfig
[swa] - userConfig:
[swa]  - routes:
[swa]   - 0:
[swa]     - route: /redirect/301
[swa]     - redirect: /index2.html
[swa]     - statusCode: 301
[swa]   - 1:
[swa]     - route: /redirect/302
[swa]     - redirect: /index2.html
[swa]     - statusCode: 302
[swa]   - 2:
[swa]     - route: /redirect/*
[swa]     - redirect: /index2.html
[swa]   - 3:
[swa]     - route: /*.google
[swa]     - redirect: https://www.google.com/
[swa]   - 4:
[swa]     - route: /*.{jpg}
[swa]     - redirect: /jpg.html
[swa]   - 5:
[swa]     - route: *.foo
[swa]     - redirect: /foo.html
[swa]   - 6:
[swa]     - route: /*.{png,gif}
[swa]     - redirect: /png_gif.html
[swa]   - 7:
[swa]     - route: /folder/*.{html,xml}
[swa]     - rewrite: /folder/
[swa]   - 8:
[swa]     - route: /rewrite_index2
[swa]     - rewrite: /index2.html
[swa]    - headers:
[swa]      - a: c
[swa]      - foo: bar
[swa]   - 9:
[swa]     - route: /rewrite-to-function
[swa]     - rewrite: /api/headers
[swa]   - 10:
[swa]     - route: /login-github
[swa]     - rewrite: /.auth/login/github
[swa]   - 11:
[swa]     - route: /logout
[swa]     - rewrite: /.auth/logout
[swa]   - 12:
[swa]     - route: /.auth/login/aad
[swa]     - statusCode: 404
[swa]   - 13:
[swa]     - route: /status-code-401.txt
[swa]     - statusCode: 401
[swa]   - 14:
[swa]     - route: /status-code-403.txt
[swa]     - statusCode: 403
[swa]   - 15:
[swa]     - route: /status-code-404.txt
[swa]     - statusCode: 404
[swa]   - 16:
[swa]     - route: /only-authenticated
[swa]    - allowedRoles:
[swa]      - 0: authenticated
[swa]   - 17:
[swa]     - route: /api/info
[swa]    - allowedRoles:
[swa]      - 0: authenticated
[swa]   - 18:
[swa]     - route: /api/error
[swa]     - statusCode: 403
[swa]  - navigationFallback:
[swa]    - rewrite: /index.html
[swa]   - exclude:
[swa]     - 0: *.{txt}
[swa]  - responseOverrides:
[swa]   - 401:
[swa]     - rewrite: /custom-401.html
[swa]   - 403:
[swa]     - rewrite: /custom-403.html
[swa]   - 404:
[swa]     - rewrite: custom-404.html
[swa]  - mimeTypes:
[swa]    - .swaconfig: application/json
[swa]  - globalHeaders:
[swa]    - a: b
[swa]   - isLegacyConfigFile: false
[swa] checking storage content
[swa] finding file for request
[swa]  - requestPath: /folder/index.html
[swa]  - filePath<EMAIL_ADDRESS>[swa]  - exists: true
[swa]  - filePathFromRequest: /folder/index.html
[swa] checking mime types
[swa]  - extension: .html
[swa]  - found: text/html
[swa] constructing headers
[swa]  - matchingRouteHeaders:
[swa] - globalHeaders:
[swa]   - a: b
[swa] checking global headers
[swa]  - Strict-Transport-Security: max-age=10886400; includeSubDomains; preload
[swa]  - Referrer-Policy: same-origin
[swa]  - X-Content-Type-Options: nosniff
[swa]  - X-XSS-Protection: 1; mode=block
[swa]  - X-DNS-Prefetch-Control: off
[swa]  - Cache-Control: must-revalidate, max-age=30
[swa]  - ETag: "SWA-CLI-ETAG"
[swa]  - url: /folder/index.html
[swa]  - target: /Users/wassimchegham/oss/@azure/static-web-apps-cli/cypress/fixtures/static
[swa] customUrl: false
[swa] custom page or index.html detected
[swa]  - url: /folder/index.html
[swa]  - statusCode: 200
[swa]  - target: /Users/wassimchegham/oss/@azure/static-web-apps-cli/cypress/fixtures/static
[swa] is4xx: false
[swa] checking if file exists in user's output location
[swa]  - file<EMAIL_ADDRESS>[swa]  - exists: true
[swa] serving static content
[swa]  - file<EMAIL_ADDRESS>[swa]  - url: /folder/index.html
[swa]  - code: 200
```

Thanks @manekinekko
It seems that your config file still contains the following route setting according to the following log.
{
    "route": "/folder/*.{html,xml}",
    "rewrite": "/folder/"
}

[swa] turning glob expression into valid RegExp
[swa]  - glob: /folder/*.{html,xml}
[swa]  - regexp: /^\/folder\/.*(html|xml)$/
[swa]  - isMatch: true

Please remove this route setting at first from staticwebapp.config.json and try again if you can.
In my env this issue can be reproduced when removing the setting and accessing /folder/index.html?key=value.

@horihiro your analysis was correct. When the matching route is absent, then isMatchingRewriteRoute becomes undefined and the rest of the logic fails. We will fix that soon.
https://github.com/Azure/static-web-apps-cli/blob/09eb7223330bf42b94ee87cb2ff639087919c8f0/src/msha/middlewares/request.middleware.ts#L271
Thank you again for your feedback 🙌🏼

@irwinwilliams this should be resolved. Please reopen this thread if you still have the issue.

Hey, very cool @manekinekko, I see it was merged to main, how do I get access to it via npm?

👋 @irwinwilliams Did you manage to get it working on 0.6.1?
@manekinekko I've been struggling to get vite's dev server to run nicely on swa, and it turns out it's because it's ignoring query params. I'm on swa-cli v0.6.1.
Relevant part of the log:
The HelloWorld.vue file is being requested with query params to fetch the styles for the component, but it looks like it's stripped out when swa proxies it?
--------------------------------------------------------
[swa] ------------------- processing route -------------------
[swa] --------------------------------------------------------
[swa] processing /src/components/HelloWorld.vue
[swa] checking for matching route
[swa] checking auth request
[swa]  - not an auth request
[swa] checking function request
[swa]  - not a function request
[swa] checking HTTP method: GET
[swa]  - method is valid (allow-list: GET,HEAD,OPTIONS)
[swa] checking for query params
[swa] checking rewrite auth login request
[swa] checking rewrite auth logout request
[swa] checking authorizations for route
[swa]  - no matching rule
[swa]  - access authorized
[swa] using userConfig
[swa]  - userConfig: <undefined>
[swa] checking storage content
[swa] finding file for request
[swa]  - filePathFromRequest: /src/components/HelloWorld.vue
[swa] checking mime types
[swa]  - extension: .vue
[swa]  - found: application/octet-stream
[swa] constructing headers
[swa]  - matchingRouteHeaders: <undefined>
[swa]  - globalHeaders: <undefined>
[swa]  - url: /src/components/HelloWorld.vue
[swa]  - target: http://localhost:3000
[swa] customUrl: false
[swa] is4xx: false
[swa] remote dev server detected. Proxying request
[swa]  - url: /src/components/HelloWorld.vue
[swa]  - code: 200
[swa] GET http://localhost:3000/src/components/HelloWorld.vue (proxy)
[swa]
[swa] --------------------------------------------------------
[swa] ------------------- processing route -------------------
[swa] --------------------------------------------------------
[swa] processing /src/App.vue?vue&type=style&index=0&lang.css
[swa] checking for matching route
[swa] checking auth request
[swa]  - not an auth request
[swa] checking function request
[swa]  - not a function request
[swa] checking HTTP method: GET
[swa]  - method is valid (allow-list: GET,HEAD,OPTIONS)
[swa] checking for query params
[swa]  - query: vue=&type=style&index=0&lang.css=
[swa] checking rewrite auth login request
[swa] checking rewrite auth logout request
[swa] checking authorizations for route
[swa]  - no matching rule
[swa]  - access authorized
[swa] using userConfig
[swa]  - userConfig: <undefined>
[swa] checking storage content
[swa]  - query: vue=&type=style&index=0&lang.css=
[swa] finding file for request
[swa]  - filePathFromRequest: /src/App.vue
[swa] checking mime types
[swa]  - extension: .vue
[swa]  - found: application/octet-stream
[swa] constructing headers
[swa]  - matchingRouteHeaders: <undefined>
[swa]  - globalHeaders: <undefined>
[swa]  - url: /src/App.vue
[swa]  - target: http://localhost:3000
[swa] customUrl: false
[swa] is4xx: false
[swa] remote dev server detected. Proxying request
[swa]  - url: /src/App.vue
[swa]  - code: 200
[swa] GET http://localhost:3000/src/App.vue (proxy)
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher viteTimeMiddleware  : /src/components/HelloWorld.vue
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher corsMiddleware  : /src/components/HelloWorld.vue
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher viteDecoreURIMiddleware  : /src/components/HelloWorld.vue
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher viteServePublicMiddleware  : /src/components/HelloWorld.vue
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher viteTransformMiddleware  : /src/components/HelloWorld.vue
[run] 2021-08-11T11:50:17.132Z vite:cache [memory] /src/components/HelloWorld.vue
[run] 2021-08-11T11:50:17.132Z vite:time 0ms   /src/components/HelloWorld.vue
[swa] getting response from dev server
[swa] GET http://localhost:4280/src/components/HelloWorld.vue - 200
[swa] getting response from dev server
[swa] GET http://localhost:4280/src/App.vue - 200
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher viteTimeMiddleware  : /src/App.vue
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher corsMiddleware  : /src/App.vue
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher viteDecoreURIMiddleware  : /src/App.vue
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher viteServePublicMiddleware  : /src/App.vue
[run] Wed, 11 Aug 2021 11:50:17 GMT connect:dispatcher viteTransformMiddleware  : /src/App.vue
[run] 2021-08-11T11:50:17.133Z vite:cache [memory] /src/App.vue
[run] 2021-08-11T11:50:17.133Z vite:time 0ms   /src/App.vue


Build Performance Test

Build Performance (No bin/obj)

Before Cold Build Times Changes:

Cold Build Before = Time Elapsed 00:00:57.796

After Cold Build Times Changes:

Cold Build After = Time Elapsed 00:00:51.735

Build Performance (Incremental change)

Before Incremental Build Times Changes:

Incremental Build Before = Time Elapsed 00:00:23.325

After Incremental Build Times Changes:

Incremental Build After = Time Elapsed 00:00:22.457

Bonus

MSBuild output:

MSBuild Log - Cold Build Before Changes.txt
MSBuild Log - Cold Build After Changes.txt
Any comments / feedback for how we can improve this experience?
These tests were run on my home development machine which is sorely in need of an upgrade. AMD FX-8350 CPU, 8 GB RAM, Windows on SSD, VS2019 on WD Black HDD. My incremental build times are usually a lot faster with MSBuild set to Minimal output (approx 6 - 8s).
I was unable to get apkanalyser working on Win 10, even using WSL, so I wasn't able to measure app size or I would have done all the challenges. A similar tool from Microsoft, or a way to run the Android SDK tool on Win 10, would be useful.
Version Information
Microsoft Visual Studio Community 2019
Version 16.2.3
VisualStudio.16.Release/16.2.3+29215.179
Microsoft .NET Framework
Version 4.7.03190
Installed Version: Community
Visual C++ 2019   00435-60000-00000-AA798
Microsoft Visual C++ 2019
ADL Tools Service Provider   1.0
This package contains services used by Data Lake tools
Application Insights Tools for Visual Studio Package   9.1.00611.1
Application Insights Tools for Visual Studio
ASP.NET and Web Tools 2019   16.2.290.48383
ASP.NET and Web Tools 2019
ASP.NET Web Frameworks and Tools 2019   16.2.290.48383
For additional information, visit https://www.asp.net/
Azure App Service Tools v3.0.0   16.2.290.48383
Azure App Service Tools v3.0.0
Azure Data Lake Node   1.0
This package contains the Data Lake integration nodes for Server Explorer.
Azure Data Lake Tools for Visual Studio   2.3.9000.0
Microsoft Azure Data Lake Tools for Visual Studio
Azure Functions and Web Jobs Tools   16.2.290.48383
Azure Functions and Web Jobs Tools
Azure Stream Analytics Tools for Visual Studio   2.3.9000.0
Microsoft Azure Stream Analytics Tools for Visual Studio
C# Tools   3.2.1-beta4-19408-03+2fc6a04980f800c59e8ede97e6ae294ff47d666e
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.
Common Azure Tools   1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.
Dotfuscator Community Edition   <IP_ADDRESS>14-e0e25f754
PreEmptive Protection - Dotfuscator CE
Extensibility Message Bus   1.2.0 (d16-2@8b56e20)
Provides common messaging-based MEF services for loosely coupled Visual Studio extension components communication and integration.
Fabric.DiagnosticEvents   1.0
Fabric Diagnostic Events
GitHub.VisualStudio   <IP_ADDRESS>81
A Visual Studio Extension that brings the GitHub Flow into Visual Studio.
IntelliCode Extension   1.0
IntelliCode Visual Studio Extension Detailed Info
JustDecompilePackage2017 Extension   1.0
JustDecompilePackage2017 Visual Studio Extension Detailed Info
Microsoft Azure HDInsight Azure Node   2.3.9000.0
HDInsight Node under Azure Node
Microsoft Azure Hive Query Language Service   2.3.9000.0
Language service for Hive query
Microsoft Azure Service Fabric Tools for Visual Studio   2.5
Microsoft Azure Service Fabric Tools for Visual Studio
Microsoft Azure Stream Analytics Language Service   2.3.9000.0
Language service for Azure Stream Analytics
Microsoft Azure Stream Analytics Node   1.0
Azure Stream Analytics Node under Azure Node
Microsoft Azure Tools   2.9
Microsoft Azure Tools for Microsoft Visual Studio 0x10 - v2.9.20626.2
Microsoft Continuous Delivery Tools for Visual Studio   0.4
Simplifying the configuration of Azure DevOps pipelines from within the Visual Studio IDE.
Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines
Microsoft Library Manager   1.0
Install client-side libraries easily to any web project
Microsoft MI-Based Debugger   1.0
Provides support for connecting Visual Studio to MI compatible debuggers
Microsoft Visual C++ Wizards   1.0
Microsoft Visual C++ Wizards
Microsoft Visual Studio Tools for Containers   1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.
Microsoft Visual Studio VC Package   1.0
Microsoft Visual Studio VC Package
Mono Debugging for Visual Studio   16.2.6 (4cfc7c3)
Support for debugging Mono processes with Visual Studio.
NuGet Package Manager   5.2.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/
PowerShell Pro Tools for Visual Studio   1.0
A set of tools for developing and debugging PowerShell scripts and modules in Visual Studio.
Productivity Power Tools 2017/2019   16.0
Installs the individual extensions of Productivity Power Tools 2017/2019
ProjectServicesPackage Extension   1.0
ProjectServicesPackage Visual Studio Extension Detailed Info
ResourcePackage Extension   1.0
ResourcePackage Visual Studio Extension Detailed Info
ResourcePackage Extension   1.0
ResourcePackage Visual Studio Extension Detailed Info
SQL Server Data Tools   16.0.61906.28070
Microsoft SQL Server Data Tools
ToolWindowHostedEditor   1.0
Hosting json editor into a tool window
TypeScript Tools   16.0.10627.2001
TypeScript Tools for Microsoft Visual Studio
Visual Basic Tools   3.2.1-beta4-19408-03+2fc6a04980f800c59e8ede97e6ae294ff47d666e
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.
Visual F# Tools 10.4 for F# 4.6   16.2.0-beta.19321.1+a24d94ecf97d0d69d4fbe6b8b10cd1f97737fff4
Microsoft Visual F# Tools 10.4 for F# 4.6
Visual Studio Code Debug Adapter Host Package   1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio
Visual Studio Spell Check Everywhere   VSSpellCheckEverywhere
An extension that enables spell checking within any Visual Studio file editor or tool window that uses WPF text boxes.
https://GitHub.com/EWSoftware/VSSpellChecker
Visual Studio Spell Checker   VSSpellChecker
An editor extension that checks the spelling of comments, strings, and plain text as you type or interactively with tool windows.
https://GitHub.com/EWSoftware/VSSpellChecker
Visual Studio Tools for Containers   1.0
Visual Studio Tools for Containers
Visual Studio Tools for Kubernetes   1.0
Visual Studio Tools for Kubernetes
VisualStudio.Mac   1.0
Mac Extension for Visual Studio
VSColorOutput   2.6.5
Color output for build and debug windows - http://mike-ward.net/vscoloroutput
Xamarin   <IP_ADDRESS> (d16-2@37df81894)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.
Xamarin Designer   <IP_ADDRESS>5 (remotes/origin/d16-2@357d38ef4)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.
Xamarin Templates   16.3.117 (59a59e8)
Templates for building iOS, Android, and Windows apps with Xamarin and Xamarin.Forms.
Xamarin.Android SDK   <IP_ADDRESS> (d16-2/ebd65c9)
Xamarin.Android Reference Assemblies and MSBuild support.
Mono: mono/mono/2019-02@e6f5369c2d2
Java.Interop: xamarin/java.interop/d16-2@d64ada5
LibZipSharp: grendello/LibZipSharp/d16-2@caa0c74
LibZip: nih-at/libzip/rel-1-5-1@b95cf3f
ProGuard: xamarin/proguard/master@905836d
SQLite: xamarin/sqlite/3.27.1@8212a2d
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-2@6f6c969
Xamarin.iOS and Xamarin.Mac SDK   <IP_ADDRESS> (c669116)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

@irongut Thank you so much for your submission! To claim your swag, please fill out this quick survey!
http://aka.ms/androidappchallenge


Navbar covering the entire screen on Mobile Devices

Fixes Issue🛠
Closes #434
Description👨💻
The Navbar on the  mobile screen did not having the appropriate height to cover the entire screen which was creating a design flaw
Type of change📄

[x] Bug fix (non-breaking change which fixes an issue)

How this has been tested✅
Chrome developer tools
Checklist✅

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have added demonstration in the form of GIF/video file
[x] I am an Open Source Contributor

Screenshots/GIF📷

Hi, @Rakesh9100 Please check and verify the commit

Merged 🥳 @pb0126


TypeError: input.size is not a function at dateTimePicker (bootstrap-datetimepicker.js:2305)

I new in allcountjs the first my application has javascript error
TypeError: input.size is not a function
at dateTimePicker (bootstrap-datetimepicker.js:2305)
at HTMLDivElement. (bootstrap-datetimepicker.js:2364)
at Function.each (jquery.js:368)
at jQuery.each (jquery.js:157)
at jQuery.$.fn.datetimepicker (bootstrap-datetimepicker.js:2359)
at Object.date (allcount.js:153)
at Object.service.fieldEditor (allcount-base.js:538)
at allcount-base.js:616
at publicLinkFn (angular.js:6940)
at boundTranscludeFn (angular.js:7080)
Any one please help me.

Hey Hi,
Fix this issue by replacing input.size() with input.length in the following files:
.\node_modules\allcountjs\public\assets\lib\js\eonasdan-bootstrap-datetimepicker\bootstrap-datetimepicker.js
.\node_modules\allcountjs\bower_components\eonasdan-bootstrap-datetimepicker\src\js\bootstrap-datetimepicker.js
Hope that solves it for others as well.
Cheers! :+1:


$translateProvider.use() always returns empty string

I use localstorage and it work perfectly but whatever i do i need to know the current selected language
the call to $translateProvider.use() always returns undefined
any idea?

Are you using it in the bootstrap/config of your app? Or why calling $translateProvider?
A Plnkr usually helps getting things easier for us.

I wish to know wich language useLocalStorage() and detect language have to find out in .config
Of cource $translate.use() its working in my home controller
but, if possibile, i wish to know the language that's going to be used a little earlier.

a) could you please post a complete setup?
b) Well, the thing is, that the variable given back by asking use() is only set in one of these cases:

$translateProvider.use() or $translate.use() is explicitly called
refresh is called

Maybe I missed some.
But it's also depending on some asyncloaders responding or resolving.
So it also means in total, that you'll probably not get a reliable result in the config section. Maybe you'll get a result if you call it at the end of the config section, but preferrable I'd check if your logic can't be in the run section?

i've a few other settings that are based on the locale choosed, i will try to read them from localStorage directly or by moving to the run/ready section
thanks :)


✨ Update everything ✨

Summary

[x] - RN 0.76.1
[x] - Re.Pack 5.0.0-rc.0
[x] - removed shell app
[x] - updated runtime dependencies
[x] - regenerated splashscreen (this likely needs fixing)
[ ] - use Rspack instead of webpack

Test plan

[x] - works in dev for iOS
[x] - works in dev for Android

LFG!

Known issues at this moment:

Re.Pack has a bug that causes infinite reloading when it looks for HMR update file where there isn't any
RN Screens has a bug that causes error on Android when going back to previous screen.

I'm good with merging this as is and following up with fixes to these known issues


[feature] Manual address

There should be an option for a user to simply put in an ethereum address (or ens) such that they can use the dapp without signing anything.
Thoughts:

Require an rpc url by default as the fallback.
Use a mock provider for missing methods

With stable version 2.0.0 of Web3Modal now released, we are officially dropping support for version 1.x
Due to this this issue/pr was marked for closing. It is highly recommended to upgrade as 2.x will be receiving further updates that will enable functionality for some of our newer sdks like auth and push as well as support for WalletConnect v2 (See this post about WalletConnect v1 being deprecated https://medium.com/walletconnect/walletconnect-v1-0-sunset-notice-and-migration-schedule-8af9d3720d2e)
If you need to continue using Web3Modal 1.x and require this feature/fix implemented, we suggest adding it via forking V1 branch.


CentOS 7 is EOL, find a replacement for its repositories

Container part done in #3070
We don't have similar nice way for VM yet

Container part done in #3070 We don't have similar nice way for VM yet

For testcloud, I'll add it there, and once I get to address the feedback at  #3036 , these changes would be just Workarounds.add_os('platform:el7', -'those sed commands-', key='el7_eol_repos')

Container part done in #3070 We don't have similar nice way for VM yet

VMs got addressed by #3036 as testcloud now implicitly applies this on any os having "CentOS Linux 7" in /etc/os-release.


Fix: unable to add tags to Volumes during creation.

Description
This fixes a bug in develop where creating a new volume with a tag results in a field error: "Unable to tag Volume."
This appears to have been introduced in the recent Yup upgrade. I don't exactly know why, though. We've always been validating tags in the CreateVolumeSchema as array().of(string()), but we've been sending an array of _Tag, which are objects. Maybe yup wasn't validating properly beforehand. In any case, we should dig more into this issue because it may also be present elsewhere.
To fix this, I extended the schema in CreateVolumeForm.
How to test
What are the steps to reproduce the issue or verify the changes?
Create a Volume from the Create Volume page with a tag. Check that other validation still works as expected.

Domains, Firewalls, and maybe other entities don't allow you to add a tag until after the entity is created (should we allow this at some point?)

That's a good question. Some of these (like Domains) used to support tags at creation, but we took them out to streamline the creation flow.


Remove OVB from validation options

I notice that OVB still exists in the validation Chapman Code options. It needs to be removed.
Geoff

Anne
I Agree.
This story should be actioned as soon as the last OVB change to OVF has been run.
Geoff

Anne
I have a problem on test3.
I have tried three SOM files and got the same result:
The first 2 were pieces that I had started validation on. I selected a Warning from the list and clicked Display. I got the dreaded red warning!
We're sorry, but something went wrong with processing your request. A report has been produced and sent to the webmaster. Please return to your prior activity but try to avoid doing the exact same action again.
So I went back and tried a third piece that I had to commence validation on in case there was something strange from a previous test. – Same result.
Geoff
From: Anne Vandervord @.>
Sent: Thursday, July 13, 2023 3:12 AM
To: FreeUKGen/FreeCENMigration @.>
Cc: Geoff J @.>; Mention @.>
Subject: Re: [FreeUKGen/FreeCENMigration] Remove OVB from validation options (Issue #1538)
@geoffj-FUG https://github.com/geoffj-FUG  This is ready for testing on Test3 now.
—
Reply to this email directly, view it on GitHub https://github.com/FreeUKGen/FreeCENMigration/issues/1538#issuecomment-1632910000 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AKCPIFIRI4EZS4AJTDZA4GLXP3LGNANCNFSM6AAAAAAZC6GYHQ .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/AKCPIFM3PU2UVC4575B46N3XP3LGNA5CNFSM6AAAAAAZC6GYHSWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTBKQ5LA.gif Message ID: @.*** @.***> >

@geoffj-FUG - started to look at it and the same thing happened to me when trying to display a record in a CSVPro file.
I’ll have to investigate further.

@geoffj-FUG The issue on Test3 has now been resolved I believe (it was unrelated to my recent modifications).

Looks good. This can me deployed.
Geoff

@Vino-S to check if it's ready to be deployed

Deployed but Vino will close once rake task is set up to run on 1st of each month


fix: Django REST Framework integration

Hi,
this PR fixes the integration with Django REST Framework (especially the filtering and the exception handling). So it should probably fix #50. I used #6 as a starting point to implement the functionality but needed to extend it a little bit.
The filtering is fixed by returning a valid model (abstract or not doesn't really matter). DRF then allows filtering against all defined properties.
The exception handling is a little bit more complicated as Django (and DRF) decided to only handle 404 errors for the model.DoesNotExist exception (https://github.com/django/django/blob/f9ec777a826816e20e68021c0e73b5a76be650af/django/shortcuts.py#L86). So if there is a normal model, we can raise the provided DoesNotExist exception. But for abstract models, there is no DoesNotExist exception. In this case it seems to be the easiest way to monkey patch the model and provide the normal ObjectDoesNotExist.

Looks like #50 has some info about when this might have broken (in #38).
Looks like #6 didn't add tests that explicitly tested against DjangoFilterBackend. 😢
I'm not a huge fan of supporting the model field though since it could vary between the different QuerySets. (I believe this is why I removed it in #38). If you're using the same model everywhere I think most things can now be done without QuerySetSequence?

Yes, I understand why QuerySetSequence is not supporting the model property at the moment. The problem is, that without it, I don't really see a way to make the DRF 404 and especially the filtering work correctly.
We currently use the library where we have multiple different models that share a property like name that needs to be filtered. The model provided to QuerySetSequence is an abstract one with just the properties that DRF needs to filter against. I hope that makes sense.

Hard to say, as long as the provided (abstract) model only contains fields that are defined in the models as well, everything is fine. I will try this with some incorrect models and report back what DRF does.

Thanks! I skimmed the changes and overall they look OK. Will try to take a closer look soon!
Thanks for including tests! 👍

You're welcome.
I've locally modified the test to check what happens. If you add an invalid property to the search_fields like invalid_field, like so:
    class AbstractModelTestListView(generics.ListAPIView):
        queryset = QuerySetSequence(Author.objects.all(), model=AbtractModel)
        serializer_class = TestSerializer
        permission_classes = []
        filter_backends = [filters.SearchFilter]
        search_fields = ["^name", "invalid_field"]

Django raises:
django.core.exceptions.FieldError: Cannot resolve keyword 'invalid_field' into field. Choices are: article, blogpost, book, id, name

Which looks good in my opinion. Furthermore, if you modify the AbtractModel to include a property that the actual models do not have, like:
class AbtractModel(models.Model):
    name = models.CharField(max_length=50)
    invalid_field = models.CharField(max_length=50)

    class Meta:
        abstract = True

It still raises
django.core.exceptions.FieldError: Cannot resolve keyword 'invalid_field' into field. Choices are: article, blogpost, book, id, name

So I think this is fine. Should I include some test cases for this behavior as well?

Thanks for working through this and adding the tests! 🎉

@j0nm1 I've released version 0.16 with this fix (and a couple of maintenance tasks). Sorry for the delay! Thanks again!


AMQP ack failed when an event task is executed and not consumed, while the workflow is terminated

Describe the bug
File: Event Task (core\src\main\java\com\netflix\conductor\core\execution\tasks\Event.java).
Issue: The start method creates a Message object using task.getTaskId() as the receipt property.
The value of task.getTaskId() is a UUID represented as String.
When the task's cancel method is called, it tries to ack the message, probably in order to avoid the consumption of the event when a workflow is terminated.
In the AMQPObservableQueue.ack method, there is a call to  chn.basicAck(Long.parseLong(message.getReceipt()), false); which expects the message.getReceipt() to be parse as a Long, while the current underlying value is a UUID.
This causes an exception and the message will stay in the amqp queue.
Details
Conductor version: 3.6.1
Persistence implementation:Dynomite
Queue implementation: Dynoqueues
Lock: Redis
To Reproduce
Steps to reproduce the behavior:

Create a workflow with an event task
Publish the event without consuming it
Cancel the workflow

Expected behavior
The message created should have a receipt with a Long value to avoid errors.

After looking into the issue, I think that the problem is with the ack call on the task's cancel method.
My guess is that the ack call was made to avoid consumption of the event when the task is terminated.
There are two problems in terms of AMQP (specifically, RabbitMQ):

The message's delivery tag is managed by the channel and cannot be set externally.
Additionally, the delivery tag is not saved anywhere (i.e. on the task itself or something) after the message is published.
Therefore, future ack's can be made only by consumers which will read the delivery tag from the consumed message's envelope.
Trying to ack a specific message by a generated delivery tag (task.getTaskId()) like in the task's cancel method isn't supported by AMQP.
Reference - section 1.1

The cancel method can be called while the message was already consumed and acked by some kind of external process.
Re-acking it again will cause a "delivery-tag doesn't exist error".

An attempt to ack a non-existing delivery tag will result in the channel being shutdown with the following error:
message:2022-04-19T11:41:47,704 ERROR [AMQP Connection x.x.x.x:5672] (com.netflix.conductor.contribs.queue.amqp.AMQPConnection) - SUBSCRIBER Channel has been shutdown: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - unknown delivery tag XXXXXXXXXXXXX, class-id=60, method-id=80)


How to create a custom function?

Hi there!
We're happy users of Graphite :)
I created a custom function hacking functions.py and besides being a bit hacky, it's working for our purpose. It's basically a summarizeToMonth function which aggregates all the datapoints of one month into the first day of the month so that we can create graphs with stats per-month. It's hacky, otherwise I would have created a PR ;)
Anyway, I'd like to clean the code a bit and add the function without touching the code functions.py file. Is that possible in any way? How?
Thanks!
Stefano

Hi @stefanosala! Unfortunately we don't currently support "pluggable functions". Coincidentally, we recently merged support for integralByInterval() which sounds like it does the same thing. 😸

Oh, interesting, thanks a lot!


Interactive mode cannot trigger the copy to server location like in the headless option

added in 0.3.5


Absorb arraypool from CoreFX

Why absorb arraypool

Too many different platforms to mistarget.  Easier to do this.


Feature: Native gradle with GraalVM's native-image tool

Expected Behavior
Fast and quick compilation, no matter what
Current Behavior

Slow deamon start
After some idle time
Waiting for JVM to warm is not acceptable, fast compilation is needed no matter what

Context
This is mostly due to the way JVM works
I think with the help of the native-image tool from GraalVM, it could help a lot!
Steps to Reproduce (for bugs)
Compile any project, for kotlin one, recompile simple project takes 8+ seconds for gradle to be ready
Your Environment
Gradle 4.10
Kotlin 1.2.71

oh i think it is duplicate with : https://github.com/gradle/gradle/issues/6905


Has anyone tried Implementing Breakout rooms ?

Not yet but it is on quite popular request from our users so we look at it and had some discussions about how to implement. User interface is quite important here.

I add some proposed engineering ideas on the breakout room topic. Many users think at breakout rooms the way they are implented in Zoom architecture: the meeting moderator creates the rooms, and then moves / ask people to move into the rooms, the rooms are totally separated one from the other. I think this is a STRONG limitation to the idea, and the limitation just comes from the internal "central point of A/V mixing" which is in Zoom architecture.
We use webRTC, and can implement a much better, flexible, useful architecture on the concept of "breakout rooms".
We shall of course allow the same very basic functions described above, with the same principles. However we shall add to this:

when breakout rooms exists, there is a central "atrium room" where ALL existing breakout rooms are visible. they look like a composite single video stream which is what you would see if you are in that room. You can see (by clicking some option on the room icon/video) who is in the room, and in case enable audio from the room you are clicking on.
the user can then decide autonomously to enter / ask to be admitted into a room he selects and he is in the room
when the user leaves a room he goes back to the "atrium" room and can start again the process.
The moderator of the meeting of course has the power to configure the breakout sessions choosing the mode he prefers (usual Zoom style stuff, the new way I propose above). This is just a rough scenario. to be discussed.

I have implemented one to one Breakout rooms ( not group ) and are very simple to use ( compared to zoom ).

I have implemented one to one Breakout rooms ( not group ) and are very simple to use ( compared to zoom ).


Upload API stress/load testing

Stress test the following scenarios:

[ ] Multiple clients with multiple simultaneous uploads (one upload per client)
[ ] One client with multiple simultaneous uploads under same client session
[ ] Single large file upload
[ ] Simultaneous large file uploads

[ ] Test DEX -> EDAV copy by azure function for large payloads

Re: large file uploads

Single large file upload
Simultaneous large file uploads

Flagged by Frank Wang

Azure function is limited to 10 min execution, right? I am thinking about larger datasets especially all CDC data coming in using the Upload API and Functions may overload.

We're using a higher-tier, so we don't have a 10 min execution limit. But it's possible than an Azure Function timeout could kill the copy. It depends on whether the function is waiting for a synchronous copy complete response, or just triggers the copy (async). I'm not sure how it's implemented.

File Name
File Size
Upload Duration [ms]
Upload Duration [s]
Note

large_file_95MB.txt
95MB
25205
25.205

large_file_95MB.txt
95MB
46841
46.841

large_file_95MB.txt
95MB
24303
24.303

large_file_95MB.txt
95MB
8878
8.878

large_file_95MB.txt
95MB
28440
28.44
concurrent upload

large_file_190MB.txt
190MB
45984
45.984

large_file_190MB.txt
190MB
70151
70.151

large_file_190MB.txt
190MB
35144
35.144

large_file_190MB.txt
190MB
39293
39.293

large_file_190MB.txt
190MB
52680
52.68
concurrent upload

large_file_476MB.txt
476MB
931273
931.273
not on z-scaler, received response patch 500 once

large_file_476MB.txt
476MB
32093
32.093
on z-scaler

large_file_476MB.txt
476MB
30174
30.174
on z-scaler

large_file_476MB.txt
476MB
91478
91.478
not on on z-scaler

large_file_953MB.txt
963MB
261317
261.317
(~4.35 min) on on z-scaler

large_file_4GB.txt
4GB
2212094
2212.094
(~36 min) on z-scaler, received responses patch 500, head 423

Thx @cosmycx. This is fine for individual file, but doesn't really cover the "one client, many concurrent files" use case.
It looks like you did concurrency of 2x.
A concurrent file test would cover 2-100x files. Sparse is fine [2, 3, 5, 8, 13, 21, 34, 55, 89...].
My current understanding is concurrency fails at ~4.

changed function plan:

looks to be working differently and crashing less, still getting 500s


I run the train.py and got this,I think there is something wrong,and I do not know why.

I use 18 images to train the model, and I got loss like this~I do not know how to solve this?Is the training sets too small?
Epoch 2/20
19/19 [==============================] - 3s - loss: -0.0191 - dice_coef: 0.0191
Epoch 3/20
19/19 [==============================] - 3s - loss: -0.0191 - dice_coef: 0.0191
Epoch 4/20
19/19 [==============================] - 4s - loss: -0.0191 - dice_coef: 0.0191
Epoch 5/20
19/19 [==============================] - 4s - loss: -0.0191 - dice_coef: 0.0191
Epoch 6/20
19/19 [==============================] - 4s - loss: -0.0191 - dice_coef: 0.0191
Epoch 7/20
19/19 [==============================] - 3s - loss: -0.0191 - dice_coef: 0.0191
Epoch 8/20
19/19 [==============================] - 4s - loss: -0.0191 - dice_coef: 0.0191
Epoch 9/20
19/19 [==============================] - 4s - loss: -0.0191 - dice_coef: 0.0191
Epoch 10/20
19/19 [==============================] - 3s - loss: -0.0192 - dice_coef: 0.0192
Epoch 11/20
19/19 [==============================] - 3s - loss: -0.0192 - dice_coef: 0.0192
Epoch 12/20
19/19 [==============================] - 3s - loss: -0.0192 - dice_coef: 0.0192
Epoch 13/20
19/19 [==============================] - 3s - loss: -0.0192 - dice_coef: 0.0192
Epoch 14/20
19/19 [==============================] - 3s - loss: -0.0192 - dice_coef: 0.0192
Epoch 15/20
19/19 [==============================] - 3s - loss: -0.0192 - dice_coef: 0.0192
Epoch 16/20
19/19 [==============================] - 3s - loss: -0.0192 - dice_coef: 0.0192
Epoch 17/20
19/19 [==============================] - 4s - loss: -0.0193 - dice_coef: 0.0193
Epoch 18/20
19/19 [==============================] - 4s - loss: -0.0193 - dice_coef: 0.0193
Epoch 19/20
19/19 [==============================] - 3s - loss: -0.0193 - dice_coef: 0.0193
Epoch 20/20
19/19 [==============================] - 4s - loss: -0.0193 - dice_coef: 0.0193

18 images is WAY too low to train a deep neural network. You won't be able to train it and/or you will overfit almost certainly. You could try:

getting more data - depends on the problem, but I'd say at least 1k images
using pretrained models - but even with pretrained models 18 images is too few


Add humidifiers to fixed mode

I'm trying to add my humidifier (which previously belonged to the fan domain), but it's not whitelisted as a correct device.
This should not be a problem though. I would expect this to work:
- entity_id: humidifier.air_humidifier
  fixed:
    states_power:
      on: 25
  standby_power: 0.5

I suppose it would even work with linear mode because humidifiers have min_humidity, max_humidity, and target_humidity attributes.
Thanks for the project! Even been looking for such integration for a long time! Thanks for making it!

Added support with #427. This has been merged into master. Are you able to test with the master branch?
I was only able to do some testing with setting up a MQTT humidifier, but I don't have a real humidifier to test with.

I suppose it would even work with linear mode because humidifiers have min_humidity, max_humidity, and target_humidity attributes.

Could you explain how min, max or target humidity can affect power draw?
I understand that if it expose also current humidity - so we can determine if it works or not, but the attributes you have mentioned are not enough I think.

@nepozs You got me thinking some more. But it makes no sense to have a humidifier working with linear mode. Think I will revert my additions for that.
I implemented by looking at min, max and current humidity value. But I suspect a humidifier will just switch on until target humidity is reached. Same as a freezer or heater would work.
Let's assume the target humidity is 40. Regardless of the current humidity being 10, 15, 20 I suspect will take the same amount power to get the humidity up to 40, and the humidifier will turn off at 40. Until at 39 it will turn on again.
I don't have a real smart humidifier to see what the real pattern is and how it behaves. Maybe @denysdovhan can elaborate about this some more. Or check the state history in HA.

My humidifier has 4 different modes: Low, Medium, High, and Humidity. The first three modes will change the intensity of the emission. I suppose this also affects power consumption.
Though you're right I doubt target humidity actually tells anything about power consumption. Additionally, the logic might differ from model to model.

I'll try to measure the power consumption of my humidifier in different modes next week. Then I'll be able to specify precise power via states_power and attributes.
My humidifier attributes look like this:
min_humidity: 30
max_humidity: 80
available_modes:
  - Low
  - Medium
  - High
  - Humidity
humidity: 60
mode: Humidity
device_class: humidifier
friendly_name: Air Humidifier
supported_features: 1

The sensor would look like this:
sensor:
  - platform: powercalc
    entity_id: humidifier.air_humidifier
    fixed:
      power: 12
      states_power:
        mode|Low: 5
        mode|Medium: 10
        mode|High: 15
        mode|Humidity: 12

@denysdovhan Thanks for the input! Let me know if you have the measurements ready.
states_power for the humidifier should already be supported now with the master branch version.

So? whats with power depend of mode of humidifier?
i have 3 modes and have no idea how to use it.

The creator of this issue suggested his humidifier used a relatively fixed amount of power. So he measured the amount of power for his humidifier on each mode (using a power meter) and powercalc allows to set the power for each mode:
Like this:
sensor:
  - platform: powercalc
    entity_id: humidifier.air_humidifier
    fixed:
      states_power:
        mode|Low: 5
        mode|Medium: 10
        mode|High: 15

The power values and attributes are different for each humidifier model, so you should change them to fit your use case.

I've finally measured my humidifier's power. Here's what I got:
sensor:
- entity_id: humidifier.air_humidifier
        fixed:
          states_power:
            mode|Low: 18
            mode|Medium: 21
            mode|High: 23
            mode|Humidity: 21
        standby_power: 4

However, in Humidity (auto) mode. The power can change dramatically depending on the current humidity level (from 4W to 24W). I've specified the middle value for my Humidity mode, bad I guess it's gonna be far from realistic.


Small read buffer with some sites

Using liche to check links in a set of markdown file documentation.  A link to cuda is causing trouble.   Not sure of a workaround, however this issue over in muffet seems useful but unsure: https://github.com/raviqqe/muffet/issues/37
docker run -it --rm -v ${PWD}:/src raviqqe/liche:latest --concurrency 1 -v /src/test-links.md
/src/test-links.md
	ERROR	https://developer.nvidia.com/cuda-toolkit
		Error when reading response headers: small read buffer. Increase ReadBufferSize. Buffer size=4096, contents: "HTTP/1.1 200 OK
        Age: 553
        Cache-Control: no-cache
Cache-Control: no-store
Content-Language: en
Content-Security-Policy-Report-Only: default-src 'self' 'unsafe-eval' 'unsafe-inline' assets.adobedtm"..."developer.nvidia.com forums.developer.nvidia.com www.youtube.com 20762408p.rfihub.com a.rfihub.com platform.twitter.com s.dca0.com bid.g.doubleclick.net nvidia.demdex.net syndication.twitter.com stati"

The response header is about 5.8kb:
curl -D header_size.txt https://developer.nvidia.com/cuda-toolkit

ls -l header_size.txt
5844 Jun  8 14:10 header_size.txt

At the moment raviqqe/liche:latest was pushed on Oct 1, 2019 at 11:49 pm. #24 was merged here https://github.com/raviqqe/liche/commit/9ab22f4c2728f75dc7bbc8112b09caad3b65be96 on Dec 6, 2019.
@raviqqe could you please rebuild the docker image to include changes that fixes the issue?


Change the output of the Hello World sample to say "Hello there  or stranger"

IBM Cloud toolchain: Delivery Pipeline deployed ibm-garage-method to dev, including commit ae6ab042af811740f8adedd95ca37e1be85a5703


Cleanup and improve tox.ini

Replace '{envbindir}/python' with '{envpython}'
See: https://tox.readthedocs.io/en/latest/config.html#substitutions-for-virtualenv-related-sections

Drop "discover" argument. It is now the default for unittest.
https://docs.python.org/3/library/unittest.html#command-line-interface

When executed without arguments Test Discovery is started:

Add the '{posargs}' placeholder to allow passing arguments to unittest
from the tox command line.
https://tox.readthedocs.io/en/latest/config.html#substitutions-for-positional-arguments-in-commands

These look like improvements, thanks!
I'm inclined to remove Tox, but I said I wouldn't in 2016 and was going to quietly do it in 2021 or something. Would anyone care?

tox allows contributors to test all supported Python versions with a single, familiar command, tox. The alternative would be running the same unittest command across different Python versions (or waiting for a Travis round trip).
tox would also allow the project to change its test runner, should it ever choose to. For example, one day you may wish to move the project to pytest. If that were to happen, contributors could still use the same generic entry point, tox to run the tests.


Update to new API

After an API change in libfreenect [1] to support multiple resolutions, ffi-libfreenect is no longer compatible with the latest version of libfreenect.  It'd be awesome if the Ruby bindings were updated to support this new functionality, if you have a chance.
Thanks,
Drew Fisher
[1] - https://github.com/OpenKinect/libfreenect/commit/752e2f2476a99d640f6d56ae8d4aad60ce273cc2

Hey guys,
I've updated the code from the libfreenect/wrappers repo, added a gemspec and updated the Rakefile a bit.
Any chance of getting this merged into master and getting the gem updated on RubyGems?