Unable to launch spyder after upgrading to 4.0.1

Issue Report Checklist

[x] Searched the issues page for similar reports
[x] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
[x] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
[x] Tried basic troubleshooting (if a bug/error)

[x] Restarted Spyder
[x] Reset preferences with spyder --reset
[ ] Reinstalled the latest version of Anaconda
[ ] Tried the other applicable steps from the Troubleshooting Guide

[x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description
After updating spyder, I was unable to launch it. I uninstalled it, installed spyder 3.3.6, 4.0.0, and 4.0.1. They all cause the same error message:
Traceback (most recent call last):
  File "/Users/colinww/anaconda3/bin/spyder", line 11, in <module>
    sys.exit(main())
  File "/Users/colinww/anaconda3/lib/python3.7/site-packages/spyder/app/start.py", line 201, in main
    from spyder.app import mainwindow
  File "/Users/colinww/anaconda3/lib/python3.7/site-packages/spyder/app/mainwindow.py", line 167, in <module>
    from spyder.utils.environ import WinUserEnvDialog
  File "/Users/colinww/anaconda3/lib/python3.7/site-packages/spyder/utils/environ.py", line 19, in <module>
    from spyder.plugins.variableexplorer.widgets.collectionseditor import (
  File "/Users/colinww/anaconda3/lib/python3.7/site-packages/spyder/plugins/variableexplorer/widgets/collectionseditor.py", line 29, in <module>
    from pympler.asizeof import asizeof
  File "/Users/colinww/anaconda3/lib/python3.7/site-packages/pympler/asizeof.py", line 1476, in <module>
    import numpy  # NumPy array, matrix, etc.
  File "/Users/colinww/anaconda3/lib/python3.7/site-packages/numpy/__init__.py", line 158, in <module>
    from . import core
  File "/Users/colinww/anaconda3/lib/python3.7/site-packages/numpy/core/__init__.py", line 100, in <module>
    from . import _internal
  File "/Users/colinww/anaconda3/lib/python3.7/site-packages/numpy/core/_internal.py", line 20, in <module>
    IS_PYPY = platform.python_implementation() == 'PyPy'
  File "/Users/colinww/anaconda3/lib/python3.7/platform.py", line 1267, in python_implementation
    return _sys_version()[0]
  File "/Users/colinww/anaconda3/lib/python3.7/platform.py", line 1228, in _sys_version
    repr(sys_version))
ValueError: failed to parse CPython sys.version: '3.7.0 (default, Jun 28 2018, 07:39:16) \n[Clang 4.0.1 (tags/RELEASE_401/final)]'

So I modified platform.py around line 1220 to report some more info:
        # CPython
        match = _sys_version_parser.match(sys_version)
        if match is None:
            print("This is the executable: {}".format(sys.executable))
            print("And this is the version: {}".format(sys.version))
            raise ValueError(
                'failed to parse CPython sys.version: %s' %
                repr(sys_version))

And now this is the output I get:
...
This is the executable: /Users/colinww/anaconda3/bin/python
And this is the version: 3.7.0 (default, Jun 28 2018, 07:39:16)
[Clang 4.0.1 (tags/RELEASE_401/final)]
Traceback (most recent call last):
  File "/Users/colinww/anaconda3/bin/spyder", line 11, in <module>
    sys.exit(main())
  File "/Users/colinww/anaconda3/lib/python3.7/site-packages/spyder/app/start.py", line 201, in main
    from spyder.app import mainwindow
...

Which is super weird, because if I launch that executable, I get:
colinww@ca-l-colin system-model % /Users/colinww/anaconda3/bin/python
Python 3.7.6 | packaged by conda-forge | (default, Jan  7 2020, 14:13:53)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.version
'3.7.6 | packaged by conda-forge | (default, Jan  7 2020, 14:13:53) \n[Clang 4.0.1 (tags/RELEASE_401/final)]'

This would be parsed correctly...
Anyway, long story short. I traced it back to the launcher in /Users/colinww/anaconda3/bin/spyder. The script is launched with: #!/bin/bash /Users/colinww/anaconda3/bin/pythonw.
When I run this, I get the same output which is causing crashing.
colinww@ca-l-colin system-model % /Users/colinww/anaconda3/bin/pythonw
Python 3.7.0 (default, Jun 28 2018, 07:39:16)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin

So I changed the header line to read #!/Users/colinww/anaconda3/bin/python, and so far so good...
So is there something wrong with my system? This is a fairly clean install of 2019.10, all I added were a few packages always using conda.
Versions

Spyder version: 4.0.1
Python version: 3.7.6
Qt version: 5.9.7
PyQt version: 5.9.2
Operating System name/version: Darwin 15.2

By the way, this is the result of python.app:
colinww@ca-l-colin system-model % conda list python.app
# packages in environment at /Users/colinww/anaconda3:
#
# Name                    Version                   Build  Channel
python.app                2                        py37_9
colinww@ca-l-colin system-model % conda install python.app
Collecting package metadata (current_repodata.json): done
Solving environment: done

# All requested packages already installed.

I met the same problem as you did

I also just had this probelm with my Mac after conda updating few hours ago, but not when I updated conda in my Windows PC

I just ran into the a very similar problem on my Mac after a conda update.
This is from a clean environment only installing python=3.7 and spyder.
  File ".../anaconda3/envs/test_spyder/lib/python3.7/site-packages/spyder/app/mainwindow.py", line 3718, in main
    mainwindow = run_spyder(app, options, args)
  File ".../anaconda3/envs/test_spyder/lib/python3.7/site-packages/spyder/app/mainwindow.py", line 3559, in run_spyder
    main.setup()
  File ".../anaconda3/envs/test_spyder/lib/python3.7/site-packages/spyder/app/mainwindow.py", line 922, in setup
    from spyder.plugins.completion.plugin import CompletionManager
  File ".../anaconda3/envs/test_spyder/lib/python3.7/site-packages/spyder/plugins/completion/plugin.py", line 27, in <module>
    from spyder.plugins.completion.languageserver.plugin import (
  File ".../anaconda3/envs/test_spyder/lib/python3.7/site-packages/spyder/plugins/completion/languageserver/plugin.py", line 30, in <module>
    from spyder.plugins.completion.languageserver.client import LSPClient
  File ".../anaconda3/envs/test_spyder/lib/python3.7/site-packages/spyder/plugins/completion/languageserver/client.py", line 24, in <module>
    import zmq
  File ".../anaconda3/envs/test_spyder/lib/python3.7/site-packages/zmq/__init__.py", line 42, in <module>
    _load_libzmq()
  File ".../anaconda3/envs/test_spyder/lib/python3.7/site-packages/zmq/__init__.py", line 13, in _load_libzmq
    PYPY = platform.python_implementation().lower() == 'pypy'
  File ".../anaconda3/envs/test_spyder/lib/python3.7/platform.py", line 1265, in python_implementation
    return _sys_version()[0]
  File ".../anaconda3/envs/test_spyder/lib/python3.7/platform.py", line 1226, in _sys_version
    repr(sys_version))
ValueError: failed to parse CPython sys.version: '3.7.0 (default, Jun 28 2018, 07:39:16) \n[Clang 4.0.1 (tags/RELEASE_401/final)]'

@jjhelmus, @mingwandroid, @msarahan, do you know what's happening here?

After more poking, this seems to also be the root cause of seeing this error: #10984 Just somehow in the past, spyder was allowed to startup successfully.

The python 3.7.6 build 1 packages in defaults accidentally have the conda-forge branding applied which I believe can cause this issue.  Updating to the build 2 packages which are already available should fix this issue. Newly created environments should install this build of Python automatically.
I have no idea where the 3.7.0 (defaults, Jun 28 2018... string is coming from.

For what is is worth, the 3.7.0 version is coming from the python binary in the python.app package.

Thanks @jjhelmus for the clarification! That means that running
conda update python

should fix the problem.
People that posted on this issue, please confirm.

Yes, for me running
conda update python
has fixed the problem.

Yes that fixes it for me too, thank you!

Why is this not handled when I run conda update --all?

for me everything is running again as well

Ok, this is great news! Thanks a lot @jjhelmus for your help!

Why is this not handled when I run conda update --all?

Any comments about this @jjhelmus?

Thank you so much!

In my case, I uninstalled Spyder 2 and run:

conda activate
spyder


AWS: Agent does not update instance type changes

Given the following scenario:
EC2 node deployed via Docker Cloud using t2.medium instance type. After changing the instance type to t2.large via AWS console (stop, change instance type, start) the WebUI continues to list node details for the old instance instance type.
The WebUI does receive a Node Update (agent) success event, which triggers the corresponding Container update events successfully. Details about the new instance type are available via the instance metadata (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-retrieval)
Note: I have not tested this scenario with other providers (i.e. resizing a DigitalOcean droplet, though I suspect the flow should be fairly similar).

Hi Sebastian,
The agent is not telling DC that the size of an instance has changed. That task belongs to a system container deployed in the node called "metrics". DC should have been notified about this change and should have modified the info, but according to your issue, looks like there was an error here. Can you provide me more info about the node? need to troubleshoot it to know what's going on there.
Thanks!

@amegianeg are you saying that your guys removed the metrics system container from the codebase? or that it is removed when the AWS instance type changes? (stop EC2 instance, change type, start EC@ instance).
The issue remains the same, I just tried the following:

Create a new node cluser using AWS as provider, us-east-1 and using an m4-large instance (or any type)
Once the cluster is deployed everything works as expected
Now go into the AWS Console and stop the EC2 instance launched by DC.
At this point the node is show as "Unreachable" in DC
Change the instance type to something else (c4.8xlarge in my test) via AWS Console
Start the EC2 instance
New node type (and CPU & RAM) are not updated in DC

Note: this time in my test, the node remains as "Unreachable" in DC indefinitely.
I think the expected behaviour would be that if a Node is resized, the specs would be updated by DC.3

@sebastianvilla sorry for not clarifying that: yes, we no longer maintain the metrics container. Sorry for the inconvenience

Same problem here - when changing EC2 type directly at AWS, Docker Cloud still showing old instance type.
Is there some way how to change instance type on-the-fly?
Thanks :-)


create @Media Types

create @Media Types
thank you

@Hatef-Rostamkhani Can you be more specific? Is this just different media query types or something else?

I believe he means things such as ttl, tv, speech, projection. This should maybe be another sub-feature of the CSS3 Media Queries entry?
I'm also interested in media features like aspect-ratio, but I'm not sure where that would fall because they may only apply to certain media types.

yes  , exactly like as tv , speech ,  ttl
for more information see https://www.w3.org/TR/CSS21/media.html#media-intro


Enable whitelisting of tasks and services in sdk_diag via pytest markers

As discussed on slack with @mpereira.
As background, in the Kubernetes package we install the package only once for the whole test suite (with the exception of testing installs and upgrades obviously). In this case, that means that sdk_diag does not collect logs, plans, bundles, etc for tasks created by Kubernetes package because it is running before tests run. Adding whitelist in this PR allows us to collect diagnostic data that we need when tests fail.

Re-triggered failed checks as all of them failed for the same reason: timeout downloading some JAR artifacts.

This is a regular problem with bintray - can we switch to Maven Central?

This is a regular problem with bintray - can we switch to Maven Central?

@jimmidyson is this something we can do in this repo, or is it a TeamCity setting. Could you create a ticket for tools infra if it's the latter?


Values are not updated and too many tiles

I use the plugin in my homebridge with v16.13.1. The plugin is up to date.
I am currently using two govee sensors.

Bug: The values are not currently synchronized. This means that the sensor delivers a different value than the home app.
Bug: e are now shown ten tiles instead of four in the home app, how can I delete that?

I look forward to an answer.

LG

@ARPIpappen Your system might be picking up readings from other Govee devices, perhaps your neighbors.
You can use Remove Single Cached Accessory in Homebridge to delete unwanted accessories but they will reappear. If you want to permanently ignore, you can try v1.1.4 which has a new feature: https://github.com/asednev/homebridge-plugin-govee/issues/42


vendor.js load so slow!

my project is builded with vue-cli, and the vendor.js is loaded very very slow!

somebody please help me !

Hi @vopjie,

This is not a Vue problem (and problably not a vue-cli problem either), it's a webpack problem
Issues on Github are reserved for bug reports.

Please use forum.vuejs.org or ask webpack experts on Stakc Overflow.

ok, thank you so much


Is there any roadmap for Blazor?

I'm thinking about to use blazor for a new requiriment in the job, but since it's an experimental technology I would like to know if is there any roadmap or planning for the product; for example, It's already an offical Microsoft Product? and It's a fact that Blazor reach any stable versión (I guess that I and thousands of developers really hope so)?

Do you know who's Daniel Roth ? If you don't then you should know...
The following was said by him:
Razor Components to ship with ASP.NET Core in .NET Core 3.0
We announced last month at .NET Conf that we've decided to move forward with shipping the Blazor server-side model as part of ASP.NET Core in .NET Core 3.0. About half of Blazor users have indicated they would use the Blazor server-side model, and shipping it in .NET Core 3.0 will make it available for production use. As part of integrating the Blazor component model into the ASP.NET Core we've decided to give it a new name to differentiate it from the ability to run .NET in the browser: Razor Components. We are now working towards shipping Razor Components and the editing in .NET Core 3.0. This includes integrating Razor Components into ASP.NET Core so that it can be used from MVC. We expect to have a preview of this support early next year after the ASP.NET Core 2.2 release has wrapped up.
Our primary goal remains to ship support for running Blazor client-side in the browser. Work on running Blazor client-side on WebAssembly will continue in parallel with the Razor Components work, although it will remain experimental for a while longer while we work through the issues of running .NET on WebAssembly. We will however keep the component model the same regardless of whether you are running on the server or the client. You can switch your Blazor app to run on the client or the server by changing a single line of code. See the Blazor .NET Conf talk to see this in action and to learn more about our plans for Razor Components:
Source: https://blogs.msdn.microsoft.com/webdev/2018/10/02/blazor-0-6-0-experimental-release-now-available/


VTOL: disable WV when moving

This PR disables weather vane (WV) when moving. Background is that when moving lateral, the roll offset required for this motion also introduces a yawing rate from the WV. This is normally not desired for VTOL when flying in windy conditions, as then the nose of the vehicle gets turned out of the wind, leading to crosswind or tailwind situations.
After this PR, the weather vane would still be active when:

no stick inputs while flying in manual position controlled mode
when the velocity setpoint is below half of the mission hover velocity (MPC_XY_CRUISE ), which is typically the case in loiter, takeoff and land situations in auto (mission) mode

In low-wind conditions it it could maybe be desired to have the WV always active in auto mode (like it is now), such that the vehicle automatically yaws in direction of flight. We could thus add a parameter to enable WV in auto mode also when moving. Or instead of making the threshold for WV on/off depending on MC_XY_CRUISE like it is now, making a threshold parameter (which, when set to a high velocity, would always enable WV).
I'm currently rather for the option without another parameter if it can be avoided.

Found an issue with the previous implementation, fixed it and flight tested it again now. No conceptual changes, so I assume you're still okay with that @RomanBapst ?


typing interval and deleting interval gets slow on Android

We are using this library for our project in React and we found an issue on mobiles that use Android when we scroll down the page and scroll up again to show the words that use react-rotating-text the typing interval and the deleting interval get slower than before we do the scrolling.
class Home extends Component {
  render() {
    const sportTypes = ['swim', 'tennis', 'soccer', 'basketball'];
  };
  return (

Let's play
    <ReactRotatingText items={sportTypes}/>

  );
}

Is there any suggestion on how to solve this? Thanks in advance!

Hey @paolitaclo thanks for the issue. Unfortunately, I'm not sure if I can reproduce the error.
If nothing else works for you, you might want to consider switching to typed.js, which others have found to be pretty good.

Closed due to inactivity.


Allow using tabs for indentation

From<EMAIL_ADDRESS>on 2010-10-13T10:06:19Z
While python standards say you should use spaces for indentation, some people prefer tabs.
Add an option to allow switching between using tabs and spaces.
Other related features that would be nice: ability to set tab size, auto-detect indentation type from file, indent using whitespace from previous line.
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=413

From pierre.raybaut on 2010-10-16T04:04:42Z
Labels: -Type-Defect -Priority-Medium Type-Enhancement Priority-Low

From<EMAIL_ADDRESS>on 2010-12-10T06:39:13Z
currently the large fixed tab size of 10 chars makes some codes unreadable (the ones indented with tabs)...

From ahaarnos on 2011-04-02T15:46:06Z
At least allowing the custom tab length (number of spaces to insert on tab characters) setting would be good if it's not already offered.  IDLE has had that option for quite some time (some people prefer 2 spaces to the default 4).

From pierre.raybaut on 2011-04-03T06:34:21Z
This issue was closed by revision c91e943375fe .
Status: Fixed

From pierre.raybaut on 2011-04-24T07:38:46Z
This issue was closed by revision c91e943375fe .

From<EMAIL_ADDRESS>on 2011-05-17T13:24:01Z
A couple of notes after testing the change made:
First sforne.. mentioned that tabs are fixed at a size of (about) 10 chars.  Please change this so that it is either a configurable number of spaces (usually 3 or 4), or fixed at 4 spaces.
Second, the smart indentation still uses 4 spaces regardless of the indentation setting (e.g. start a new line after a ':').

From pierre.raybaut on 2011-05-21T06:07:20Z
Everything should now be working with: https://code.google.com/p/spyderlib/source/detail?r=d6ae750db50c46c467cddab35e641fd1c2ba809d and https://code.google.com/p/spyderlib/source/detail?r=5a6e2a1de59270dcb3b72ad82a988f6016dedd8b Moreover, I've added an option (just below the "indentation chars" option in Preferences) to set the tab stop width -- in pixels, not in characters (that's the way Qt works). Default was 80 pixels which is indeed a bit large: now default is 40 pixels: https://code.google.com/p/spyderlib/source/detail?r=ed96fe41324676b0eec57ac1dc7485d15681ce7a

From<EMAIL_ADDRESS>on 2011-05-21T18:31:38Z
Thank you.  I really appreciate the efforts you've put in.
Spyder is turning into a really nice IDE.


new_resource.path not getting set on windows platform

I am new to chef and trying to use ark to install JMeter on a windows VM node. I created a cookbook called my_jmeter_install. I added below windows specific attribute to my_jmeter_install.rb –
normal['ark']['win_install_dir'] = 'C:\Users\Administrators\JMeter'
In metadata.rb of my_jmeter_install cookbook, I added - depends "ark"
In default.rb of my_jmeter_install cookbook, I added -
include_recipe "ark"
ark 'my_jmeter' do
url 'http://mirror.cogentco.com/pub/apache//jmeter/binaries/apache-jmeter-2.13.zip'
version '2.13'
action :install
end
When I converge the node, I get the following error –
Error executing action install on resource 'ark[my_jmeter]'
ArgumentError
You must supply a name when declaring a directory resource
After turning on debug option for chef-client, I can see the 'new_resource.path' is not set in ark/libraries/default.rb –
if node['platform_family'] === 'windows'
new_resource.path = new_resource.win_install_dir
Any ideas what could be causing ‘new_resource.path’ not to be set? It is blank in the debug output.

I wanted to close this thread in case anyone else runs into this. Adding win_install_dir attribute to the recipe code solved the issue.
ark 'apache-jmeter' do
url 'http://mirror.cogentco.com/pub/apache//jmeter/binaries/apache-jmeter-2.13.zip'
version '2.13'
owner 'administrator'
win_install_dir 'c:\users\administrator\jmeter'
strip_components 0
action :install
end
Closing issue...it was not really a issue, but I am not sure how to post a question here!


[BUG] Can't import kashgari after successful installation

After successful installation

I'm getting this error and in Jupyter, and in python 3.5 in console.

I use Linux Ubuntu 16.04.6
I tried to install version, that I have used some time ago, but it throws the same error
Of course, I didn't change anything in original code

Kashgari need to use python 3.6+.
Karina<EMAIL_ADDRESS>周四23:23写道：

Assigned #131 https://github.com/BrikerMan/Kashgari/issues/131 to
@BrikerMan https://github.com/BrikerMan.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/BrikerMan/Kashgari/issues/131?email_source=notifications&email_token=ACHPKS46D5U2W7KELTARSP3P5YIY3A5CNFSM4H54MEL2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOSKXJDLY#event-2460914095,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACHPKS3GZ5OGYCPGC2PGPJTP5YIY3ANCNFSM4H54MELQ
.

Ok, thank you,
sorry for bothering


Fix default layout dockwidget row heights. Clean code.

Fix #2555

@ccordoba12 can you check if it works now as expected?

Sorry, still now working. Now the Object Inspector is 1/4 of the vertical screen size and the consoles take the other 3/4. But the window still overflows the screen.

Well... not sure what is the problem then, cause I cannot reproduce it.

Then please create a Kubuntu 14.04 virtual machine, so you can debug and solve this problem by yourself.
Another alternative would be to use the old layout system just the first time (because that one didn't have this problem).

... ok

Sorry, but it's an ugly bug ;-)

I installed Kubuntu 14.04 in a spare machine I had and checked and I see no overflow in the window when doing a fresh start (I deleted .spyder2 prior to launching spyder).
I can get it to look right if I set the timer to a much higher value (tested with 5 seconds)...

Ok, can you check again with this version now?
In my setup (kubuntu 14.04) it oes not overflow, and the sizes look ok.

I am erasing everything (.spyder) and running this version and it looks like this after a clean start.

Yes, that's exactly how it's looking for me :-) So it's fixed, great!!
Then there's one thing left: the IPython console is too small compared to the Object Inspector. Please make it appear a bit bigger :-)

I don't remember if code comments trigger a notification, but the timer value is still 5000 in master.


SP4: Improve launcher API

Hello, as asked in #2627 I removed the extension methods to move it to the launcher class. I removed the launcher extensions I don't know if this what you excepted from me since we modify the API of the project.
Regards

Thanks @Guillaume-Deville, this looks great!


online help fix & no-inherit pydoc option

fix online help w/class items not in pydoc; add option to not show inherited class info

Description of Changes

[ ] Wrote at least one-line docstrings (for any new functions)
[ ] Added unit test(s) covering the changes (if testable)

[ ] Included a screenshot or animation (if affecting the UI, see Licecap)

Online help gives a display with only the following three lines:

Error
.html
AssertionError

for modules where there are classes with items that are not displayed in the pydoc (these include readonly items).
Also, subclassed classes display with all the parent info, which can be huge and unwanted. An option has been added for this.
Issue(s) Resolved
Fixes #15396
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct:
brian<EMAIL_ADDRESS>

Hello @briantoby! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file spyder/plugins/onlinehelp/pydoc_patch.py:

Line 28:1: E302 expected 2 blank lines, found 0
Line 494:17: E265 block comment should start with '# '

In the file spyder/plugins/onlinehelp/widgets.py:

Line 45:1: W293 blank line contains whitespace
Line 54:1: E302 expected 2 blank lines, found 1
Line 56:1: W293 blank line contains whitespace
Line 257:1: W293 blank line contains whitespace
Line 263:1: W293 blank line contains whitespace


load config file issue with ESP8266 (LittleFS and SPIFFS)

Hello, I'm using this board, which I can't perfectly identify. I believe the flash size is 1-2MB, based on the Sketch size %, and since using 4MB flash partitions results in failure to begin either format.
esptool.py says Auto-detected Flash size: 2MB.
When I run the example code from Async_ConfigOnSwitch, I see a few things. I've manually censored SSID and password in this serial output:
Stored: SSID = ssid , Pass = password
[WM] * Add SSID =  ssid , PW =  password
Got stored Credentials. Timeout 120s for Config Portal
[WM] LoadWiFiCfgFile
[WM] failed

So it appears that the last known working configuration is recovered from the esp8266 flash. (As detailed here)
The file system begins correctly, but FileFS.open(CONFIG_FILENAME, "r") returns false.
Then, I run the config portal and input the wifi info! It saves correctly!
[WM] SaveWiFiCfgFile
[WM] OK

However, if I reset the device, I get the same input for LoadWifiCfgFile as before. file is false, and the load fails. I can't figure out why this is.
I wrote this separate small sketch which simply reads the config file. If I flash this sketch, instead of resetting the Async_ConfigOnSwitch sketch, it properly loads the file and prints it with no issue.
So it appears to only run into this problem only occurs somewhere during Async_ConfigOnSwitch when starting, strangely. It reproduces on both SPIFFS and LittleFS. I'm not entirely convinced it's not my hardware, to be honest.

@rorosaurus
Thanks for using the library.
That's happening either because

you haven't entered ConfigPortal to input a valid sets of WiFi Credentials which would then be saved into SaveWiFiCfgFile  or
the file has been erased / corrupted between resets (forced format, bad flash,bad power, etc.). I'm afraid only you can deal with this issue.

Pls see the sequence below about how the process is working (actually just tested on a Wemos D1)

No Config Portal (CP) access and/or without valid input WiFi Credentials. This is what you get at the beginning

Starting Async_ConfigOnSwitch using LittleFS on ESP8266_NODEMCU
[WM] RFC925 Hostname = ConfigOnSwitch
[WM] setSTAStaticIPConfig for USE_CONFIGURABLE_DNS
[WM] Set CORS Header to :  Your Access-Control-Allow-Origin
Stored: SSID = HueNet1, Pass = 12345678
[WM] * Add SSID =  HueNet1 , PW =  12345678
Got stored Credentials. Timeout 120s for Config Portal
[WM] LoadWiFiCfgFile
[WM] failed                                           <=============== always failed
ConnectMultiWiFi in setup
[WM] ConnectMultiWiFi with :
[WM] * Flash-stored Router_SSID =  HueNet1 , Router_Pass =  12345678
[WM] Connecting MultiWifi...
[WM] WiFi connected after time:  1
[WM] SSID: HueNet1 ,RSSI= -43
[WM] Channel: 2 ,IP address: <IP_ADDRESS>
After waiting 3.44 secs more in setup(), connection result is connected. Local IP: <IP_ADDRESS>
[WM] freeing allocated params!
HHHHHHHHHH HHHHHHHHHH HHHHHHHHHH HHHHHHHHHH HHHHHHHHHH HHHHHHHHHH

Enter CP and input valid WiFi Credentials  ==>

Starting Async_ConfigOnSwitch using LittleFS on ESP8266_NODEMCU
[WM] RFC925 Hostname = ConfigOnSwitch
[WM] setSTAStaticIPConfig for USE_CONFIGURABLE_DNS
[WM] Set CORS Header to :  Your Access-Control-Allow-Origin
Stored: SSID = HueNet1, Pass = 12345678
[WM] * Add SSID =  HueNet1 , PW =  12345678
Got stored Credentials. Timeout 120s for Config Portal
[WM] LoadWiFiCfgFile
[WM] failed                                   <============= still failed at the beginning
ConnectMultiWiFi in setup
[WM] ConnectMultiWiFi with :
[WM] * Flash-stored Router_SSID =  HueNet1 , Router_Pass =  12345678
[WM] Connecting MultiWifi...
[WM] WiFi connected after time:  1
[WM] SSID: HueNet1 ,RSSI= -37
[WM] Channel: 2 ,IP address: <IP_ADDRESS>
After waiting 3.45 secs more in setup(), connection result is connected. Local IP: <IP_ADDRESS>
[WM] freeing allocated params!
HHHHHHHHHH HHHHHHHHHH HHHHHHHHHH HHHHHH
Configuration portal requested.
[WM] RFC925 Hostname = ConfigOnSwitch
[WM] setSTAStaticIPConfig for USE_CONFIGURABLE_DNS
[WM] Set CORS Header to :  Your Access-Control-Allow-Origin
Opening configuration portal. Got stored Credentials. Timeout 120s
[WM] WiFi.waitForConnectResult Done
[WM] SET AP_STA
[WM]
Configuring AP SSID = ESP_705385
[WM] AP PWD = your_password
[WM] AP Channel = 11
[WM] AP IP address = <IP_ADDRESS>
[WM] HTTP server started
[WM] ESPAsync_WiFiManager::startConfigPortal : Enter loop
[WM] Connecting to new AP
[WM] Previous settings invalidated
[WM] Custom STA IP/GW/Subnet
[WM] DNS1 and DNS2 set
[WM] setWifiStaticIP IP = <IP_ADDRESS>
[WM] Connect to new WiFi using new IP parameters
[WM] Connected after waiting (s) : 3.21
[WM] Local ip = <IP_ADDRESS>
[WM] Connection result:  WL_CONNECTED
connected...yeey :)
Local IP: <IP_ADDRESS>
[WM] * Add SSID =  HueNet1 , PW =  12345678
[WM] * Add SSID =  HueNet2 , PW =  12345678
[WM] SaveWiFiCfgFile
[WM] OK                                        <============= after this, the file is present
[WM] freeing allocated params!

Reset many times => LoadWiFiCfgFile still OK

Starting Async_ConfigOnSwitch using LittleFS on ESP8266_NODEMCU
[WM] RFC925 Hostname = ConfigOnSwitch
[WM] setSTAStaticIPConfig for USE_CONFIGURABLE_DNS
[WM] Set CORS Header to :  Your Access-Control-Allow-Origin
Stored: SSID = HueNet1, Pass = 12345678
[WM] * Add SSID =  HueNet1 , PW =  12345678
Got stored Credentials. Timeout 120s for Config Portal
[WM] LoadWiFiCfgFile
[WM] OK                                                        <========= file is still OK here
[WM] * Add SSID =  HueNet1 , PW =  12345678
[WM] * Add SSID =  HueNet2 , PW =  12345678
ConnectMultiWiFi in setup
[WM] ConnectMultiWiFi with :
[WM] * Flash-stored Router_SSID =  HueNet1 , Router_Pass =  12345678
[WM] * Additional SSID =  HueNet1 , PW =  12345678
[WM] * Additional SSID =  HueNet2 , PW =  12345678
[WM] Connecting MultiWifi...
[WM] WiFi connected after time:  1
[WM] SSID: HueNet1 ,RSSI= -49
[WM] Channel: 2 ,IP address: <IP_ADDRESS>
After waiting 3.46 secs more in setup(), connection result is connected. Local IP: <IP_ADDRESS>
[WM] freeing allocated params!
HHHHHHH

Check if you accidentally turned on auto format in every startup

#define FORMAT_FILESYSTEM       true

in the code
// From v1.1.1
// You only need to format the filesystem once
//#define FORMAT_FILESYSTEM       true
#define FORMAT_FILESYSTEM         false

...

  if (FORMAT_FILESYSTEM)
    FileFS.format();

  // Format FileFS if not yet
#ifdef ESP32
  if (!FileFS.begin(true))
#else
  if (!FileFS.begin())
#endif
  {
    Serial.print(FS_Name);
    Serial.println(F(" failed! AutoFormatting."));

#ifdef ESP8266
    FileFS.format();
#endif
  }

If not this case,

try on other boards or types of boards
add more debugging messages to see where it's breaking.

Try on these boards and still OK

ESP8266_WEMOS_D1R1

Starting Async_ConfigOnSwitch using LittleFS on ESP8266_WEMOS_D1R1
[WM] RFC925 Hostname = ConfigOnSwitch
[WM] setSTAStaticIPConfig for USE_CONFIGURABLE_DNS
[WM] Set CORS Header to :  Your Access-Control-Allow-Origin
Stored: SSID = HueNet2, Pass = 12345678
[WM] * Add SSID =  HueNet2 , PW =  12345678
Got stored Credentials. Timeout 120s for Config Portal
[WM] LoadWiFiCfgFile
[WM] OK
[WM] * Add SSID =  HueNet1 , PW =  12345678
[WM] * Add SSID =  HueNet2 , PW =  12345678
ConnectMultiWiFi in setup
[WM] ConnectMultiWiFi with :
[WM] * Flash-stored Router_SSID =  HueNet2 , Router_Pass =  12345678
[WM] * Additional SSID =  HueNet1 , PW =  12345678
[WM] * Additional SSID =  HueNet2 , PW =  12345678
[WM] Connecting MultiWifi...
[WM] WiFi connected after time:  1
[WM] SSID: HueNet1 ,RSSI= -49
[WM] Channel: 2 ,IP address: <IP_ADDRESS>
After waiting 3.47 secs more in setup(), connection result is connected. Local IP: <IP_ADDRESS>
[WM] freeing allocated params!

ESP8266_WEMOS_D1MINI

Starting Async_ConfigOnSwitch using LittleFS on ESP8266_WEMOS_D1MINI
[WM] RFC925 Hostname = ConfigOnSwitch
[WM] setSTAStaticIPConfig for USE_CONFIGURABLE_DNS
[WM] Set CORS Header to :  Your Access-Control-Allow-Origin
Stored: SSID = HueNet1, Pass = 12345678
[WM] * Add SSID =  HueNet1 , PW =  12345678
Got stored Credentials. Timeout 120s for Config Portal
[WM] LoadWiFiCfgFile
[WM] OK
[WM] * Add SSID =  HueNet1 , PW =  12345678
[WM] * Add SSID =  HueNet2 , PW =  12345678
ConnectMultiWiFi in setup
[WM] ConnectMultiWiFi with :
[WM] * Flash-stored Router_SSID =  HueNet1 , Router_Pass =  12345678
[WM] * Additional SSID =  HueNet1 , PW =  12345678
[WM] * Additional SSID =  HueNet2 , PW =  12345678
[WM] Connecting MultiWifi...
[WM] WiFi connected after time:  1
[WM] SSID: HueNet2 ,RSSI= -57
[WM] Channel: 4 ,IP address: <IP_ADDRESS>
After waiting 3.56 secs more in setup(), connection result is connected. Local IP: <IP_ADDRESS>
[WM] freeing allocated params!
HHHHH

Thanks for the quick reply! And thanks for the very helpful library!
I spent a lot of time yesterday trying to figure out this issue, and I think I added a print statement inbetween some #ifdef's that didn't behave like I expected! I think the end result was FileFS.format() was getting called every time, even if I set the flag to false! I really should have just taken a break before looking at it again.
Regardless, I can confirm there's no issue with the library. I'm sorry to have consumed your cycles on this! Thank you again for your time and patience!

Good to know it's not the library's bug.

I really should have just taken a break before looking at it again.

We all need break some time, and lot of coffee too ;-)


added calendar view and might have fixed the cli issue.

WHY are these changes introduced?
Fixes #0000

WHAT is this pull request doing?

Checklist
Note: once this PR is merged, it becomes a new release for this template.

[ ] I have added/updated tests for this change
[ ] I have made changes to the README.md file and other related documentation, if applicable

Hi @ShyGuy76,
Thank you for your contribution. This PR added a react-calendar dependency to the template which will apply to everyone using this template. While react-calendar is a good library, I don't think that everyone needs it. It's best to use the Shopify CLI to create a new Shopify app (documentation). It'll use this template and you can add react-calendar there as a dependency for your app.
Therefore, I'll close this PR as it's not applicable.


on incoming emitter

very usefull for things like pre-render

once is also a nice addition for this


[Go Live][July 11]

Docs Build status updates of commit 5de7de4:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/toolkit/provision.md
:white_check_mark:Succeeded
View

msteams-platform/toolkit/TeamsFx-multi-env.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


adapt code to work in Python 3.8

Following a request from PR #27, I tried to run the code in Python 3.8 but we were using some features introduced in later versions. This PR adapts the code to work for python versions >= 3.8.

Cheers for immediately processing the needed changes!


[FEATURE REQUEST] Always have an ammonite shell in the background to avoid the startup overhead

Have you seen https://github.com/ninjudd/drip ? Using it directly or implementing something similar would virtually destroy the startup overhead of the shell. We can always have one Ammonite server process in the background that a new Ammonite shell can connect to.

Doesn't look good on the drip front: https://github.com/ninjudd/drip/issues/99

Probably not going to happen any time soon


chore: upgrade all dependencies and add react-email preview

[x] upgrade deps
[x] Resend v2
[x] Preview for React Email

fixed this build error Build error - Cannot get final name for export 'encodeXML' by upgrading to resend canary


[Feature Request] Group chat support

Creating a ticket to track this important issue.

@rodneyrod Thank you for your interest but this project is only a PoC and I have no plan to work on it in the future. I will mark the repository as archived to make that clear.


feat: add file handler option for logging

Add option to log to a file.
Overall I'm pretty unhappy adding this behaviour but a specific customer needs it.

I feel this caters for a too specific problem. I'd spend a bit more time and try to incorporate logging.dictConfig or logging.fileConfig into edge-proxy's logging setup routine to allow users more flexibility. The hard, but not impossible, part would be allowing users to specify they want structlog when providing their own stdlib Python logging config.

Superseded by #115


Feature: Support Bulk operations

Should support bulk operations for updates/reads/deletes.
Either through a Bulk object, or through collection.bulkWrite interface.

Hi, ehlertjd
Not all API is exposed directly, but you can call it through mc_worker_api:command/2/3.


Compilation error

 == Compilation error on file lib/excellent/parser.ex ==
** (RuntimeError) error parsing file /usr/lib/erlang/lib/xmerl-1.3.8/include/xmerl.hrl, got: {:error, :enoent}
    (elixir) lib/record/extractor.ex:80: Record.Extractor.read_file/1
    (elixir) lib/record/extractor.ex:50: Record.Extractor.extract_record/2
    lib/excellent/parser.ex:4: (module)
    (stdlib) erl_eval.erl:669: :erl_eval.do_apply/6

could not compile dependency :excellent, "mix compile" failed. You can recompile this dependency with "mix deps.compile
excellent", update it with "mix deps.update excellent" or clean it with "mix deps.clean excellent"

Solved  by installing erlang separately using apt instead of using the default one bundled with elixir.


How have default tenant for main site?

I have a customer that wanna resell his app. For 'main.com' I wish to use his own tenant and only use 'tenant1.main.com' for his customers. This only need a way to say 'main.com' = 'master.main.com'

@mamcx don't quite understand the question as this is what this project allows you to do

Yeah, but I need to make a subdomain for the main customer.

schema name is separate from domain name (which can be without subdomain or use multiple domains). Check 2nd page of tutorial.


Regression in 4.1.0 to create repositories under github organization for GHE

Hi there,
Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.
Terraform Version
Run terraform -v to show the version. If you are not running the latest version of Terraform, please upgrade because your issue may have already been fixed.
terraform -v
Terraform v0.14.3
+ provider registry.terraform.io/fluxcd/flux v0.0.8
+ provider registry.terraform.io/gavinbunney/kubectl v1.9.4
+ provider registry.terraform.io/hashicorp/github v2.9.2
+ provider registry.terraform.io/hashicorp/kubernetes v1.13.3

Affected Resource(s)
Please list the resources as a list, for example:

github_repository

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.
Terraform Configuration Files
terraform {
  required_version = ">= 0.13"

  required_providers {
    github = {
      source  = "hashicorp/github"
      //version = ">= 4.1.0"
      version = "~> 2.9.2"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = ">= 1.13.3"
    }
    kubectl = {
      source  = "gavinbunney/kubectl"
      version = ">= 1.9.1"
    }
    flux = {
      source  = "fluxcd/flux"
      version = ">= 0.0.7"
    }
  }
}

provider "flux" {}

provider "kubectl" {
  config_path      = var.shoot_kubeconfig_path
  load_config_file = true
}

provider "kubernetes" {
  config_path      = var.shoot_kubeconfig_path
  load_config_file = true
}

provider "github" {
  token = var.github_token
  organization = var.github_org
  base_url = var.github_api_url
}

# Flux
data "flux_install" "master" {
  target_path = var.target_path
}

data "flux_sync" "master" {
  target_path = var.target_path
  //url         = "https://${var.github_repository}/${var.github_org}/${var.github_owner}/${var.repository_name}.git"
  url         = "https://${var.github_repository}/${var.github_org}/${var.repository_name}.git"
  branch      = var.branch
}

# Kubernetes
resource "kubernetes_namespace" "flux_system" {
  metadata {
    name = "flux-system"
  }

  lifecycle {
    ignore_changes = [
      metadata[0].labels,
    ]
  }

}

data "kubectl_file_documents" "install" {
  content = data.flux_install.master.content
}

resource "kubectl_manifest" "install" {
  for_each  = { for v in data.kubectl_file_documents.install.documents : sha1(v) => v }
  depends_on = [kubernetes_namespace.flux_system]

  yaml_body = each.value
}

data "kubectl_file_documents" "sync" {
  content = data.flux_sync.master.content
}

resource "kubectl_manifest" "sync" {
  for_each  = { for v in data.kubectl_file_documents.sync.documents : sha1(v) => v }
  depends_on = [kubectl_manifest.install, kubernetes_namespace.flux_system]

  yaml_body = each.value
}

resource "kubernetes_secret" "master" {
  depends_on = [kubectl_manifest.install]

  metadata {
    name      = data.flux_sync.master.namespace
    namespace = data.flux_sync.master.namespace
  }

  data = {
    password   = var.github_token
    username   = var.github_owner
  }
}

// Old github provider v2.9.2
# GitHub
resource "github_repository" "master" {
  name           = var.repository_name
  default_branch = var.branch
  private = true
  auto_init =  true
}

/*
// New github provider  v4.1.0
resource "github_repository" "master" {
  name           = var.repository_name
  visibility     = var.repository_visibility
  auto_init =  true
}

resource "github_branch_default" "master" {
  repository = github_repository.master.name
  branch     = var.branch
}
*/

resource "github_repository_file" "install" {
  repository = github_repository.master.name
  file       = data.flux_install.master.path
  content    = data.flux_install.master.content
  branch     = var.branch
}

resource "github_repository_file" "sync" {
  repository = github_repository.master.name
  file       = data.flux_sync.master.path
  content    = data.flux_sync.master.content
  branch     = var.branch
}

resource "github_repository_file" "kustomize" {
  repository = github_repository.master.name
  file       = data.flux_sync.master.kustomize_path
  content    = data.flux_sync.master.kustomize_content
  branch     = var.branch
}

Debug Output
Please provide a link to a GitHub Gist containing the complete debug output: https://www.terraform.io/docs/internals/debugging.html. Please do NOT paste the debug output in the issue; just paste a link to the Gist.
https://gist.github.com/linuxbsdfreak/0a583947965b533f0e4a8ee9e4a3ba67
https://gist.github.com/linuxbsdfreak/97752d2f2c67548b01632bf13cfe88dc
Panic Output
If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the crash.log.
Expected Behavior
With v4.1.0 of the provider it should have created the repo under the github org
Actual Behavior
Error: GET https://github.tools.domain.com/login?return_to=https%3A%2F%2Fgithub.tools.domain.com%2Fapi%2Fapi%2Fv3%2Fuser%2Frepos: 406  []

  on main.tf line 118, in resource "github_repository" "master":
 118: resource "github_repository" "master" {

With v2.9.2 it created the repos under the organization
------
github_repository.master: Creating...
github_repository.master: Creation complete after 5s [id=flux2-kustomize-helm-example]
github_repository_file.kustomize: Creating...
github_repository_file.sync: Creating...
github_repository_file.install: Creating...
github_repository_file.sync: Creation complete after 9s [id=flux2-kustomize-helm-example/clusters/staging/flux-system/gotk-sync.yaml]
github_repository_file.kustomize: Creation complete after 9s [id=flux2-kustomize-helm-example/clusters/staging/flux-system/kustomization.yaml]
github_repository_file.install: Creation complete after 9s [id=flux2-kustomize-helm-example/clusters/staging/flux-system/gotk-components.yaml]

TF sample based on https://github.com/fluxcd/terraform-provider-flux/tree/main/examples/github
Steps to Reproduce
Please list the steps required to reproduce the issue, for example:

terraform apply

Important Factoids
Is there anything atypical about your accounts that we should know? For example: Running in EC2 Classic? Custom version of OpenStack? Tight ACLs?
References
Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

GH-1234

Hi,
Any progress regarding this issue?  Would be nice to use the latest version of the provider.

Hi,
Any progress regarding this issue?  Would be nice to use the latest version of the provider.


enabling LDAP_GROUP_PROVIDER breaks Kanboard

When setting LDAP_GROUP_PROVIDER to true, Kanboard displays a blank page.
Error logs says :
[Wed Dec 16 12:11:28 2015] [error] [client <IP_ADDRESS>] PHP Parse error:  syntax error, unexpected ')', expecting T_PAAMAYIM_NEKUDOTAYIM in /srv/kanboard/app/Group/LdapBackendGroupProvider.php on line 48
LDAP server is ActiveDirectory.
User auth works.
LDAP_GROUP_ADMIN_DN and LDAP_GROUP_MANAGER_DN are left empty.
This is an upgrade from 1.0.17 to 1.0.22 with already registered users and existing projects.

Same here v 1.0.22 (not sure what the previous version was because I was just asked to get LDAP working, which I did, now I am stuck trying to get group permissions to work).
LDAP Server is ActiveDirectory.
User auth works.
LDAP_GROUP_ADMIN_DN and LDAP_GROUP_MANAGER_DN are filled out and working.

This issue is now fixed in the development branch. It was happening only to people using PHP < 5.5 (5.3 and 5.4).
Thanks to report this bug.

Yes, running PHP 5.4, should I update to 5.5 or wait for update?
On Tue, Dec 22, 2015 at 12:51 PM, Frédéric Guillot<EMAIL_ADDRESS>
wrote:

This issue is now fixed in the development branch. It was happening only
to people using PHP < 5.5 (5.3 and 5.4).
Thanks to report this bug.
—
Reply to this email directly or view it on GitHub
https://github.com/fguillot/kanboard/issues/1557#issuecomment-166701986.

PHP 5.3.3 here

If you cannot wait the next release you can apply the fix manually: https://github.com/fguillot/kanboard/commit/8ff2032ea3fa49972fe076166c831719131e829d (change the line 48)
It's better to have the most recent version of PHP but it's not always possible.

Awesome thank you!!
On Tue, Dec 22, 2015, 2:21 PM Frédéric Guillot<EMAIL_ADDRESS>wrote:

If you cannot wait the next release you can apply the fix manually:
8ff2032
https://github.com/fguillot/kanboard/commit/8ff2032ea3fa49972fe076166c831719131e829d
(change the line 48)
It's better to have the most recent version of PHP but it's not always
possible.
—
Reply to this email directly or view it on GitHub
https://github.com/fguillot/kanboard/issues/1557#issuecomment-166720288.


Location not with id 1 not found

Describe the bug
If you installed the app and try to create a post it will produce some errors.
To Reproduce
Steps to reproduce the behavior:

Go to 'New'
Click on 'Post''
See error

Expected behavior
If there is no location set then set id to null
Desktop (please complete the following information):

OS: Windows 10
Browser Google Chrome

fixed


Slate now exports defaultPlugins

Fixes #482

Coverage remained the same at 49.621% when pulling 01937da39bd62404b90d9f04ef1912ba52b36b72 on DeedMob:master into 80092a11071113308acab4c9a7906a04743c076c on ory:master.

Thanks!


Fix file types which might be uncompressed.

I found this bug when I was doing another modification. This PR is just to fix and improve the part responsible to extract the compressed files

Thanks @marcelotrevisani! Sorry it took me so long to review

Thanks @marcelotrevisani! Sorry it took me so long to review

No problem! :)


Flash Attention V2

https://github.com/Dao-AILab/flash-attention
Flash attention v2 was released claiming 2x speedups. Making an issue to remind myself to have a look at it. And also if anyone else wants to try implement it.

I use benchmarks/benchmark_throughput.py to test flash attention V2, but it doesn't seem to have any effect. my test step is like this,

update xformers to latest version
modify the line  to self.attn_op = xops.fmha.flash.FwOp()
python3 benchmark_throughput.py  --dataset=./ShareGPT_V3_unfiltered_cleaned_split.json --model=/huggingface_data/llama-7b-hf/ --tokenizer=hf-internal-testing/llama-tokenizer --num-prompts=500

test time is like this,
Further analysis of performance, i found that the replaced part (flash attention V2) cost is too small, only at the beginning of the execution, i am confused , for flash attention V2, what can we do for vllm？

update xformers to latest version

hi, which version specifically? i don't think flash v2 support has been released yet, so you would have to install from git. also there are still some open PRs to bump xformers to flash-attn v2.0.4 bugfix release (https://github.com/facebookresearch/xformers/pull/816).

modify the line  to self.attn_op = xops.fmha.flash.FwOp()
python3 benchmark_throughput.py  --dataset=./ShareGPT_V3_unfiltered_cleaned_split.json --model=/huggingface_data/llama-7b-hf/ --tokenizer=hf-internal-testing/llama-tokenizer --num-prompts=500

I tried this as well, and there was no improvement in the benchmarks after switching to flash-attn v2.
I will try to profile the benchmark script.

modify the line to self.attn_op = xops.fmha.flash.FwOp()
i dont think this one really works. because flash-attn's another important feature is to decrease the highly gpu-memory usage in super long-context like more than 5k.
when i set that line and run inference . i dont see any changes on memory usage.

Hi @nivibilla, thanks for submitting the issue. The latest version of xformers now uses the FlashAttention-V2 algorithm, so vLLM also now takes advantage of it. Please upgrade vLLM to v0.1.4.
@tmm1 @Zhuqln To my understanding, the overall speedup should depend on your workload. At the inference time, FlashAttention is only used for the prompt inputs, and never used for the decoding inputs. For many workloads, the decoding stage takes a majority of the total execution time, so changing to FlashAttention V2 may not give a notable speedup. However, for other workload like text summarization where the prompts are very long, I believe computing attention for the prompt inputs will take a significant portion of the execution time, and thus FlashAttention V2 will have a huge impact on the overall performance.

@WoosukKwon thanks for the explanation!

The latest version of xformers now uses the FlashAttention-V2 algorithm, so vLLM also now takes advantage of it. Please upgrade vLLM to v0.1.4.

Hi, this is inaccurate since the code is still forcing xops.fmha.cutlass.FwOp to be used. If you want to take advantage of FA2, you would need to switch to xops.fmha.flash.FwOp
See benchmark results in https://github.com/facebookresearch/xformers/issues/832

Hi @nivibilla, thanks for submitting the issue. The latest version of xformers now uses the FlashAttention-V2 algorithm, so vLLM also now takes advantage of it. Please upgrade vLLM to v0.1.4.
@tmm1 @Zhuqln To my understanding, the overall speedup should depend on your workload. At the inference time, FlashAttention is only used for the prompt inputs, and never used for the decoding inputs. For many workloads, the decoding stage takes a majority of the total execution time, so changing to FlashAttention V2 may not give a notable speedup. However, for other workload like text summarization where the prompts are very long, I believe computing attention for the prompt inputs will take a significant portion of the execution time, and thus FlashAttention V2 will have a huge impact on the overall performance.

Thanks for the details. I just have a question. Why FlashAttention could not be used for decoding phase?

Why FlashAttention could not be used for decoding phase?

Its tiling strategy is not optimized for Q with seqlen=1 https://github.com/Dao-AILab/flash-attention/issues/427#issuecomment-1668257877

你好@nivibilla，感谢您提交问题。最新版本的 xformers 现在使用 FlashAttention-V2 算法，因此 vLLM 现在也利用了它。请将vLLM升级到v0.1.4。
@tmm1 @Zhuqln据我了解，整体加速应该取决于您的工作负载。在推理时，FlashAttention 仅用于提示输入，从不用于解码输入。对于许多工作负载，解码阶段占用了总执行时间的大部分，因此更改为 FlashAttention V2 可能不会带来显着的加速。然而，对于其他工作负载，例如提示很长的文本摘要，我相信提示输入的计算注意力将占用执行时间的很大一部分，因此 FlashAttention V2 将对整体性能产生巨大影响。

感谢您提供详细信息@WoosukKwon。我只是有一个问题。为什么FlashAttention不能用于解码阶段？

I'm delighted to engage in this discussion. Your report has been immensely helpful, but I do have some questions. For instance, I'm curious to know if there's a performance comparison available between trtLLM and vLLM. Such information would be greatly beneficial in guiding my decision on which framework to choose.

WoosukKwon

you assume that in summarization task most of the workload is by decoding the input. in my experimentation I saw that the scale of generation is much bigger. so, if you generate only 1-5 token then most of the workload is decoding input, there will be dependency on input length and flash attention 2 will be advantageous (as it linear in input length while naive implementation is exponential in input length). but if you generate a considerable amount of tokens, then that factor is prominent, the input decoding is negligible, and flash attention 2 has no power here.
(usually when you have long text you want a longer summarization. it doesn't make sense to summarize 1000 words article by 5 tokens)
attached link to the simulation.
please LMK if you have any comments.
https://github.com/matanhol/summarization_with_flash_attn_2_simulation

Hi @nivibilla, thanks for submitting the issue. The latest version of xformers now uses the FlashAttention-V2 algorithm, so vLLM also now takes advantage of it. Please upgrade vLLM to v0.1.4.
@tmm1 @Zhuqln To my understanding, the overall speedup should depend on your workload. At the inference time, FlashAttention is only used for the prompt inputs, and never used for the decoding inputs. For many workloads, the decoding stage takes a majority of the total execution time, so changing to FlashAttention V2 may not give a notable speedup. However, for other workload like text summarization where the prompts are very long, I believe computing attention for the prompt inputs will take a significant portion of the execution time, and thus FlashAttention V2 will have a huge impact on the overall performance.

Thanks for the details @WoosukKwon . I just have a question. Why FlashAttention could not be used for decoding phase?

you assume that in summarization task most of the workload is by decoding the input. in my experimentation I saw that the scale of generation is much bigger. so, if you generate only 1-5 token then most of the workload is decoding input, there will be dependency on input length and flash attention 2 will be advantageous (as it linear in input length while naive implementation is exponential in input length). but if you generate a considerable amount of tokens, then that factor is prominent, the input decoding is negligible, and flash attention 2 has no power here.
(usually when you have long text you want a longer summarization. it doesn't make sense to summarize 1000 words article by 5 tokens)
attached link to the simulation.
please LMK if you have any comments.
https://github.com/matanhol/summarization_with_flash_attn_2_simulation

I tried installing vllm with flash attn but it didn't work, my attempts:
Install flash attention:
```bash
# my current vllm setup without flash
# pip install --upgrade pip
# pip install torch==2.2.1
# pip install vllm==0.4.1

# flash attn https://amzn-aws.slack.com/archives/C06Q26TNN8G/p1724182667464149
# flash-attn>=2.5.8
# pip install flash-attn
# Collabs's setup with flash
# vllm                              0.5.4
# vllm-flash-attn                   2.6.1
# flash-attn                        2.6.3
# torch                             2.4.0
# Python 3.10.8

# try to install flash attn in a new py env
python3.11 -m venv ~/.virtualenvs/flash_attn_test_py10
source ~/.virtualenvs/flash_attn_test/bin/activate
pip install --upgrade pip
pip install -e ~/snap-cluster-setup

pip list | grep vllm
pip list | grep torch
pip list | grep flash-attn
pip list | grep vllm-flash-attn

# # didn't work
# pip install torch==2.2.1
# pip install vllm==0.4.1
# MAX_JOBS=4 pip install flash-attn --no-build-isolation --force

# this installed flash but vllm didn't say in it's output it was using it
pip install torch==2.4.0
pip install vllm==0.5.4
pip install flash-attn==2.6.3
pip install vllm-flash-attn==2.6.1

python ~/snap-cluster-setup/py_src/evals/boxed_acc_eval.py --model internlm/internlm2_5-1_8b --hf_gen_type vllm --path_2_eval_dataset ~/snap-cluster-setup/data/MATH/test --max_tokens 2048 --batch_size 100 --end 100 -n 1 --shuffle True --mode dryrun 2>&1 | tee $LOG_FILE && echo "Log file created at: $LOG_FILE"

# later try with py 3.10
# python3xxx -m venv ~/.virtualenvs/flash_attn_test_py10
# source ~/.virtualenvs/flash_attn_test_py10/bin/activate
# pip install --upgrade pip
# pip install -e ~/snap-cluster-setup
# pip install torch==2.4.0
# pip install vllm==0.5.4
# pip install flash-attn==2.6.3
# pip install vllm-flash-attn==2.6.1

my setting is python 3.11, that is what I really want/need.

related vllm general issues for vllm ver: https://github.com/vllm-project/vllm/issues/2747


Fix CI(Release)

Only RC channel working and main channel not properly triggering.

Local DNS server misbehaving on PVE-01. Requests on timing out on runner node. No logs are on bind9 instance.

All it needed was DNS cache flush. Working now. Seems like it's working. Main thread log
[12-14-24 12:10:06] Current Version: v0.23.6
[12-14-24 12:10:07] Latest Version: v0.23.7
[12-14-24 12:10:07] update needed
[12-14-24 12:10:07] making changes
[12-14-24 12:10:07] committing
[main efba688] bump to 0.23.7
 3 files changed, 3 insertions(+), 3 deletions(-)
[12-14-24 12:10:07] create tag with v0.23.7
Enumerating objects: 14, done.
Counting objects: 100% (14/14), done.
Delta compression using up to 4 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (8/8), 1.28 KiB | 436.00 KiB/s, done.
Total 8 (delta 4), reused 0 (delta 0), pack-reused 0 (from 0)
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
To github.com:kdpuvvadi/pocketbase.git
   acc2027..efba688  main -> main
 * [new tag]         0.23.7 -> 0.23.7
[12-14-24 12:10:11] push changes
[12-14-24 12:10:11] Create release
https://github.com/kdpuvvadi/pocketbase/releases/tag/0.23.7
[12-14-24 12:10:12] exiting

If no update is needed,
[12-14-24 12:13:58] Current Version: v0.23.7
[12-14-24 12:13:59] Latest Version: v0.23.7
[12-14-24 12:13:59] Up to date.


Remove deprecations [4.0]

This PR kicks off our 4.0 development branch by removing all the deprecated methods, fields, and behavior from the 3.x branch. Specifically:

SquidDatabase.onDowngrade now returns false by default
Removes all the various generated methods and fields for the ID field in table models. All models in 4.0 should be using either rowid or an explicitly declared primary key field. Also removes restrictions on user properties named ID -- any user-declared property, even a non-primary key one, can now be named ID. Only ROWID is reserved.
Removes deprecated SQL compilation methods that used VersionCode instead of CompileContext
Other misc deprecated methods/fields that have either been removed, renamed, or refactored into better alternatives

Definitely yes, I was planning to write a bunch of test cases that cover the migration from 3.x to 4.0 (and I also think we should release a 4.0 beta for people to experiment with and report any issues). I'll fix that extra 'a' in a future PR.


docs: install

regionmask is now available on pypi, needs no longer be downloaded from git

The need to update:
http://regionmask.readthedocs.io/en/stable/installation.html#instructions

done


Image is not recognised after install.

Hi I'm WorldOverEnd and I tried to install Deep-Daze on my computer but after I installed it I tried an example that you guys left and it left me with this error: 'imagine' is not recognized as an internal or external command, operable program or batch file.
I'm using windows 10 on a gtx 1060 6GB vram with an ryzen 7 1800x and 8 Gb ram.
Can somebody help me thanks.

You did not install Deep-Daze completely.
You need to download a version of TensorFlow before V2 because Deep Daze doesn't support V2
You will also need CUDA 11.1, go to the site I linked beneath, select "Stable 1.8.0", Select your OS, Select "pip", select "python", then select "CUDA 11.1"
Microsoft Visual C++ > Download Here
Python 3.8 > Download Here
TensorFlow python package > Download Here
CUDA 11.1 > Download Here
Make a new folder where ever you choose, drag and drop the TensorFlow python package (The .whl) file
after you have done that type this in a new command line

python -m venv --system-site-packages .\venv

.\venv\Scripts\activate

pip install --upgrade pip

pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio===0.8.0 -f https://download.pytorch.org/whl/torch_stable.html

pip install --upgrade CLIP

pip install --upgrade wheel

(CLIP, wheel, and torch are just to verify that you have the most up to date installation)
this creates a virtual environment and upgrades the different things you need to run the command.


[Bug] [K8S] projectname is missing

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
Create a K8S task.  Some parameters of ds are transformed into environment variables passed to the pod of k8s.
But you'll notice that system.project.name is empty in the environment variable.
    spec:
      containers:
        - name: aaa-2
          image: busyboy:latest
          env:
            - name: taskInstanceId
              value: '2'
            - name: system.task.definition.name
              value: aaa
            - name: system.project.name
            - name: system.project.code
              value: '10377893770976'
            - name: system.workflow.instance.id
              value: '2'
            - name: system.biz.curdate
              value: '20230728'
            - name: system.biz.date
              value: '20230727'
            - name: system.task.instance.id
              value: '2'
            - name: system.workflow.definition.name
              value: '1111'
            - name: system.task.definition.code
              value: '10377975344097'
            - name: system.workflow.definition.code
              value: '10377979443552'
            - name: system.datetime
              value: '20230728094029'
          resources:
            limits:
              cpu: '2'
              memory: 40Mi
            requests:
              cpu: '1'
              memory: 20Mi

What you expected to happen
system.project.name should be passed correctly
How to reproduce
As described above
Anything else
No response
Version
dev
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

It seems not only for k8s task, we never set value of projectName for ProcessDefinition. cc @zhongjiajie


685 - Added changes to front end for introductions on sections

What is the context of this PR?
Added front end changes to be able to add an introduction to a section
How to review
Run tests
Make sure front end looks ok and you are able to add, update, delete and undelete an introduction

 Section introduction page - front end

yarn.lock has been removed. Please revert that change.


Move @types dependencies to devDependencies

Do the following deps need to be in the "dependencies" section of package.json? Can they be moved to "devDependencies" instead?
"@types/clone": "^0.1.29",
"@types/node": "^4.0.30",
"@types/parse5": "^2.2.32"

This will reduce the code being downloaded when someone depends on dom5.

(Apologies for the necro)

The issue we're running into are type conflicts when running our webpack based builds that include this lib. Since our project uses webpack, we import the type definitions for wepack-env, instead of node. However, since both wepack-env.d.ts and node.d.ts declare some of the same names (ex. require), we get duplicate/conflicting declaration errors when compiling.
I would think this would also be an issue when a different version of a common dependent library is needed by the app. Say, for example, our app needs to use type definitions for<EMAIL_ADDRESS>Since dom5 already includes a type-def for this library in it's distribution which gets installed in the "special" @types folder, this would cause TypeScript duplicate identifier errors.

A lot of the time this is a poor interaction between the flat node_modules structure used by default by npm (post v3) and yarn, and the default behavior of typescript to pick up all ambient types in node_modules/@types. You should be able to work around it by not having flattened packages (npm's global-style or using pnpm), or explicitly listing the ambient type packages to load.


TSDB: Don't rely on integer overflow in head compaction check

Don't rely on integer overflow for correct behavior in the Head.compactable() check.
Previously, the logic for determining if the head should be compacted relied on the default values for min and max time and integer overflow when they were checked in Head.compactable(). The check in Head.compactable() effectively did math.MinInt64 - math.MaxInt64 which overflowed and wrapped to 1. Since 1 is less than 1.5 times the chunk range, compaction did not happen. This was the correct behavior but relying on overflow wrapping is surprising.
This change adds a method for checking if the min time for the head is unset and uses it to short-circuit compaction in that case. It also replaces several explicit checks for the default value to determine if the head has not yet had any samples added.

The one thing that I would like to see is a testcase in TestNoEmptyBlocks in db_test.go that fails with the old code - due to it not checking that the data is valid.

I'm not actually sure I can do this. The old code worked correctly, it just relied on integer overflow which was confusing. In both the old code and the new code, Head.compactable() returns the correct result.

The one thing that I would like to see is a testcase in TestNoEmptyBlocks in db_test.go that fails with the old code - due to it not checking that the data is valid.

I'm not actually sure I can do this. The old code worked correctly, it just relied on integer overflow which was confusing. In both the old code and the new code, Head.compactable() returns the correct result.

This synthetic test fails on main I think:
func TestHeadCompactibleCornerCase(t *testing.T) {
	dir := t.TempDir()
	wal, err := wlog.NewSize(nil, nil, filepath.Join(dir, "wal"), 32768, wlog.CompressionSnappy)
	require.NoError(t, err)

	opts := DefaultHeadOptions()
	opts.ChunkRange = 1  // This is the extreme bit
	opts.ChunkDirRoot = dir

	h, err := NewHead(nil, nil, wal, nil, opts, nil)
	require.NoError(t, err)
	defer func() {
		require.NoError(t, h.Close())
	}()

	require.False(t, h.compactable())
}

This synthetic test fails on main I think:

Ah, I see. I'll add something like that. Thank you!


cSpell.spellCheckDelayMs seems to be ignored

"cSpell.spellCheckDelayMs": 3000, appears in my vsCode user settings, but this seems to have no effect on how quickly spell check flags a word as misspelled. I would prefer plenty of time to get to the end of a word without the feedback that I haven't yet spelled a valid dictionary word.
Frustratingly, when a word goes from misspelled -> correctly spelled, spell checker seems to take a significant amount of time (perhaps equal to the delay value?) to remove the squiggly line that mark it as misspelled.

Code Spell Checker 1.4.12
VS Code:

Version 1.18.0
Commit dcee2202709a4f223185514b9275aa4229841aa7
Shell 1.7.9
Renderer 58.0.3029.110
Node 7.9.0
Architecture x64

Ubuntu 16.04.3 LTS (xenial), kernel 4.4.0-98-generic

Thank you very much for an essential extension!

Thank you. I'll check it out.

Thank you for pointing this out. I did find an issue with the delay. I hope to have it fixed in the next release.

This was fixed in a previous release.


Support for history

Url.pathname("foo", true) will keep the old url in the history.

:ship: 2.1.0 :tada:


pop up hidden

When creating a new wallet, and clicking "connected hardware wallet" button, the pop up to scan the HWW stays behind the main screen. The user is not made aware there is a popup behind and is left confused and thinking it's not working.
Environment: Linux mint, 64bit, cinnamon. Sparrow version 1.78

Try the latest version?

Imported into Parmanode today, I'll let you know if it happens again.
On Thu, 7 Dec 2023, 4:58 pm craigraw, @.***> wrote:

Try the latest version?
—
Reply to this email directly, view it on GitHub
https://github.com/sparrowwallet/sparrow/issues/1194#issuecomment-1844634028,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ASQCCX4Q4WOT5H6I5NWT7JTYIFLIZAVCNFSM6AAAAABAKL7PX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBUGYZTIMBSHA
.
You are receiving this because you authored the thread.Message ID:
@.***>

I see Parmanode is at Sparrow v1.8.1 :)
Can we close this off?

Go ahead. I haven't formally tested if I can reproduce the erorr... It happened on a student's computer. If it happens again I'll let you know.


How to forward audit logs to its own event index?

There's a similar issue: https://github.com/splunk/splunk-connect-for-kubernetes/issues/94
But what I'm trying to do is forward audit logs, not just kube api logs. The audit logs don't belong to a container itself. Is there a way to filter that to its own bucket?

Hi! Is it something like this you're looking for?
  kube-audit:
    from:
      file:
        path: /var/log/kube-audit/audit-log.json
    timestampExtraction:
      format: "%Y-%m-%dT%H:%M:%SZ"
    sourcetype: paas:rancher-audit-logs

You would need to use your props.conf and transforms.conf to forward the data to another index if you aren't wanting to use the default one from the YAML file.
Hope that helps!
Stephen


Idea to use BigUint64Array

While exploring the project I noticed that the sort was taking a significant amount of time.
https://github.com/antimatter15/splat/blob/d85a1efa260f042ba76314b35af65a1896467242/main.js#L307
I was thinking about how to avoid the separate typed arrays for sorting and realized that it might be possible to pack the content into a single BigUint64Array or BigInt64Array such that the native sort would yield the correct resolve.
So something along the lines of:
const depth = new BigUint64Array(vertexCount);
for (let j = 0; j < vertexCount; j++) {
	let z = viewProj[2] * f_buffer[8 * j + 0] +
				viewProj[6] * f_buffer[8 * j + 1] +
				viewProj[10] * f_buffer[8 * j + 2];
	depth[j] = BigInt(z * scaleToU32 + offsetToU32) << 32n | BigInt(j);
}
depth.sort();

Sorting a BigUint64Array vs the two arrays was about 3x faster on my computer.
However, I'm not quite sure what the appropriate scaling constants would be, but I assume one exists. Or maybe some other transformation that would allow this to work.
Unfortunately I don't have time to experiment with this yourself, but this approach could be an easy perf win compared to pushing sorting to GPU or alternate approaches.

That's a really interesting idea! I wonder if we reduce the precision of depth to 7 bits we can do it with 32 bit ints and whether or not that would be usable

I had been thinking something along the lines of using something like timsort so that the sorting process can leverage existing partially sorted regions, or to pre sort along the x/y/Z axes and somehow having some kind of "merge" like operator that can assemble a sorted list more quickly

I do agree that using pre-sorted regions or even just preserving the previously sorted array should speed things up. I'm not sure whether JS has this property, but many usual sorting algorithms work faster on a mostly sorted array.

It depends on the implementation, I think chrome switched to timsort which is adaptive in that way, but in my experience it is still somewhat slow even when sorting a fully sorted list.

but many usual sorting algorithms work faster on a mostly sorted array.

Yes. I was also working on my own implementation and this was much faster.

or to pre sort along the x/y/Z axes

Yes, also see https://iquilezles.org/articles/volumesort/

depth[j] = BigInt(z * scaleToU32 + offsetToU32) << 32n | BigInt(j);

You shouldn't even have to do the scale and offset. IEEE floats are sorted even if interpreted as integer (unless negative, which isn't an issue here).
So you can access your floats as a Uint32Array and just shift that exact value in the upper 32 bits.

For WebGL2, you can also do the transformation (for depth calculation) in a transform-feedback shader. Alternatively you can also render into a float texture (for WebGL1 with extensions), but this probably only benefits you if you also do the sorting and indexing on the GPU (otherwise the GPU → CPU → GPU round-trip is probably horribly slow).

Wow that looks great- seems like we could just allocate a BigInt64Array and
then initialize a Float32Array with a shared buffer to write the depths
(though I don't think we can ignore negatives, because things behind the
camera need to still be represented in case you move backwards- though I
think it should be fine to just add a fixed number like 1000 and clip
numbers that are below zero)
Just calling .sort() and not passing in a lambda seems really important-
seems like the function call overhead is very substantial. Seems like
sorting an already mostly sorted list can go from hundreds of milliseconds
to under a millisecond
And for subsequent calls we can load the same buffer as a UInt32Array and
use the odd indices to fetch the appropriate positions and then set the
even ones to the float values
Also seems like it would be possible per the inigo technique to check which
hectohectant the current view is in and skip the sort altogether if that
hasn't changed enough to warrant a re-sort at all.
On Tue, Sep 12, 2023 at 3:50 PM Jannik Vogel @.***>
wrote:

but many usual sorting algorithms work faster on a mostly sorted array.
Yes. I was also working on my own implementation and this was much faster.
or to pre sort along the x/y/Z axes
Yes, also see https://iquilezles.org/articles/volumesort/
depth[j] = BigInt(z * scaleToU32 + offsetToU32) << 32n | BigInt(j);
You shouldn't even have to do the scale and offset. IEEE floats are sorted
even if interpreted as integer (unless negative, which isn't an issue here).
So you can access your floats as a Uint32Array and just shift that exact
value in the upper 32 bits.
For WebGL2, you can also do the transformation (for depth calculation) in
a transform-feedback shader. Alternatively you can also render into a float
texture (for WebGL1 with extensions), but this probably only benefits you
if you also do the sorting and indexing on the GPU (otherwise the GPU → CPU
→ GPU round-trip is probably horribly slow).
—
Reply to this email directly, view it on GitHub
https://github.com/antimatter15/splat/issues/5#issuecomment-1716622469,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAHKZVMVCUREF5NJT5L6NTX2DRJVANCNFSM6AAAAAA4U2K42M
.
You are receiving this because you commented.Message ID:
@.***>

(though I don't think we can ignore negatives, because things behind the
camera need to still be represented in case you move backwards- though I
think it should be fine to just add a fixed number like 1000 and clip
numbers that are below zero)

It's adding a bit more work again, but you can do some bit twiddling to avoid this: https://stackoverflow.com/a/43305015

per the inigo technique to check which hectohectant the current view is in and skip the sort altogether if that hasn't changed enough to warrant a re-sort at all.

Note that this technique expects all elements on a grid - we don't have that here. It will work, but you'll lose some correctness.
Because most scenes are in 2D (limited verticality), my idea was to sort it in 2D in about 2-8 directions for a rough estimation.. and then sort within those for final quality (also to support verticallity if necessary).
However, that was before I noticed that sorting probably isn't an issue, because sorting already sorted lists can be very fast. So you might not even need the precompution.

I implemented these changes in e7bc5f9
On my computer beforehand the sorting process was taking about 370ms.
Switching to the BigInt64Array made it about 100ms.
Curiously enough it seems like keeping the mostly sorted array between sorts has a negligible effect on performance.


feat(cascading): option to bypass non-latest minor branch

Proposed change
Option to bypass non-latest minor branch on cascading bot

Related issues

- No issue associated -

Is there a case where we want to skip non-latest minors? Wouldn't this create incoherence and confusion among the different branches / code levels?

Not sure to understand why.
As for the policy, we considered the latest minor branches as supported/maintained branches. Which mean that all the other branches are considered as archived (if they are not destroyed).
When we cascade, we want to do it only on the considered branched automatically, why would it create "incoherence" to skip unmaintained branches?


Support HAProxy SSL termination

Add SSL termination support on Rancher LB service.

I'm just checking out rancher now for the first time, but please support SNI here.

Requires UI and Rancher-Compose CLI to leverage this.


Pythonic refactoring

Some small refactoring to make some class definitions more pythonic:

Having public class attributes not just defined within the constructor helps in autospeccing when using mocking frameworks for unit testing (such as mock).
Replacing repeated declarations of the get_type() method by a generic class attribute with a super class method only.

Coverage decreased (-0.02%) to 90.509% when pulling 876474378f04214fa32e04f7c9adf51a56b51b0c on pohutukawa:pythonic-refactoring into 53c4107ccd12545295dedd1e250d4feac3c5b01f on trungdong:master.

Many thanks, @pohutukawa!


way to embed the menu in a view controller as opposed to a modal

Can you tell me if it is possible to embed the menu in a view controller either via storyboard or programmatically, rather than presenting it as a modal?

@ghubaddole It's not possible as of now, you should use actionsheet or alert menu presentation style for that.

Alternately, if the menu can be presented like an ios 13 style action sheet (even on ios 12), that'll be great.


Question about output extraction

I'm wondering is there a way I can extract the prediction value separately  instead of plot.

Hi @aazz7777 ,
Yes, this is possible. The inferences are stored in ci.inferences which is a pandas dataframe with all information extracted from the fitted model.
You also have access to ci.summary_data which relates to the summarization step, such as the cumulative information.
Let me know if this helps.

Thank you !


[develop]518221 [S] Webhooks: create python script

This PR adds an initial version of a CLI utility that allows you to manage webhooks, along with the library backing the utility and an example script showing how to use the library.

I'm doing it as part of the next task where I add an example server


doc: ug2html.py: Do not depend on robotframeworklexer.

Fixes #4264.
The RobotFramework lexer has long been included in Pygments now, and
the robotframeworklexer copy on PyPI wast last updated earlier 2019.

doc/userguide/ug2html.py: Do not import robotframeworklexer, check
its version nor use it.  Bump Pygments requirement to 2.8, as that's
old enough and is what was used to test.
requirements-build.txt: Drop robotframeworklexer requirement.
Update pygments required version.

It's true that Pygments has supported RF for long time, but I'm not sure does it contain the latest version of the lexer or not. Did you check that?
A related problem is that the external lexer is badly outdated and doesn't support latest RF features. The external lexer is from the time RF's own parser was different than now. Nowadays It should just use Robot's own lexer, but I'm not sure what would be the best way to handle that dependency. It would be probably best to talk more about this on Slack.

You seem to be right, I just tested that using Pygments on its own produces ok results.


Document Release Processes

We'll need to cut a new release to Ruby Gems to get this added to a project. - @willtcarey in #7

Let's document this.

Step 1 here is setting up a BNB RubyGems account (I can tackle that) as the gem currently lives under Joe's RubyGems account

To release a new version of the BNB blazer gem:

Clone the project and make your changes
Bump the version in version.rb.
Build the gem: gem build bnb_blazer.gemspec. This will create a bnb-blazer-[version].gem
If you aren't already: Make sure you are an owner of the bnb_blazer gem. An existing owner can add you: gem owner bnb_blazer --add<EMAIL_ADDRESS>If you don't have one already: Create a RubyGems account with your BNB email
If you haven't already: login to RubyGems in your terminal: gem signin
Push the gem: gem push bnb-blazer-[version].gem
You should see the new version here

More info about the RubyGems build/release process is here.


Support larger cache sizes

The maxbytes parameter to New() is an int, which effectively limits the cache size to ~2GB. It would be nice if it supported int64 so we could create very large caches

int in Go is equivalent to int64 on 64-bit architectures. As for 32-bit archs, they usually cannot address more than 2^32 bytes in process memory.


Add a “Recently Played” Section

Description: Allow users to quickly access audiobooks they recently listened to by creating a "Recently Played" section below the "Top Audio Books" section.
Tasks:
Use localStorage to store the last few audiobooks the user played.
Create a new section on the homepage to display these books.
Ensure it updates dynamically as users interact with new books.

Hello! @Ctoic
I am a git beginner and I am eager to learn and grow with the help of such wonderful projects. Please assign me this particular task for Hacktoberfest 2024. Really looking forward to contribute.
Thank you!

Sure @G-SaiVishwas I'm here to help those who are new to Git and Githtub. Go ahead I wish you good luck 🤞🏼 and Don't forget to ⭐ the repo. Happy Coding 😄

@Ctoic
I am done with it and I have generated a pull request. Please check the code and merge it.
Thank you!


Adding and removing AttributeSets has problems

I have a Character and Weapon classes in my project.
Weapon keeps its stats, such as MagazineAmmoCount on an attributeset that it owns. My "shoot" ability costs "1 MagazineAmmoCount" attribute to shoot. However, because these attributes are not initialized properly and instead I add them through SpawnedAttributes.AddUnique (as shown in this documentation), system crashes somewhere and my project closes.
Crash tells me that faulty line is following
	UFUNCTION() 	void OnRep_MagazineAmmoCount() { GAMEPLAYATTRIBUTE_REPNOTIFY(UZWeaponAttributeSet, MagazineAmmoCount); }
I have come up with following solution to work around this problem:
AbilitySystem->InitStats(UZWeaponAttributeSet::StaticClass(), Weapon->StatInitializer); Weapon->InitializeAttributesFromThis(AbilitySystem);
Idea is to basically initialize attributes as standard, and then updating them manually using attribute set located at weapon. I havent tested the unequip function yet.

I've actually concluded that using separate AttributeSets on weapons is not a great idea. The main issues are that changes to an Attribute from a GE must go through an ASC and we can't target a specific AttributeSet if more than one of the same class exists on the ASC or the AttributeSet is removed from the ASC but still valid in the world. This causes:

Complications with multiple weapons in your inventory.

If you remove an AttributeSet before an Attribute aggregator change is replicated, the replicated change will not find its AttributeSet and crash the game. I ran into this with a rocket launcher killing myself and removing the AttributeSet on Unequip/Death before the ammo change replication came in.

There are probably ways around these issues, but I find it was much easier just to use replicated raw floats on the weapons for magazine ammo with an Ammo AttributeSet on the Character for reserve ammo (this allows weapons to share reserve ammo of the same ammo type). I'm led to believe that Fortnite also does its ammo this way with raw floats on the weapons (I'm unsure about their reserve ammo).
My next sample project coming soon is an advanced FPS that shows this functionality and how to work with it in GameplayAbilities. I'll update the README when its released.


Support ignoring custom app name for Spark engine

Why are the changes needed?

support to disable custom app name for spark engine, for either share level of USER/GROUP/SERVER. As for CONNECTION share level, custom app name for spark engine is still allowed.

How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before make a pull request

Codecov Report

Merging #5016 (e1c92df) into master (04ee20b) will not change coverage.
The diff coverage is 0.00%.

@@          Coverage Diff           @@
##           master   #5016   +/-   ##
======================================
  Coverage    0.00%   0.00%
======================================
  Files         563     563
  Lines       31167   31177   +10
  Branches     4070    4073    +3
======================================
- Misses      31167   31177   +10

Impacted Files
Coverage Δ

...in/scala/org/apache/kyuubi/config/KyuubiConf.scala
0.00% <0.00%> (ø)

...ain/scala/org/apache/kyuubi/engine/EngineRef.scala
0.00% <0.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

I'm curious to know why users set the custom app name in the first place if they want the default one?

It happened to set a custom spark app name for one time task on the user/client side without notifying the Kyuubi's administrator. Plus, he's using USER / SERVER share level and the app name was contaminated as the one time data task name. From the perspective of Kyuubi's administrator, it's hard to tell it's a shared spark SQL engine on YARN with such custom app name.
Therefore, this feature mainly enables the administrator of Kyuubi and the resource provider, to reject or ignore custom app name with the share level of USER / SERVER / GROUP.

@bowenliang123 Thanks for your input! It might apply to Flink as well. Perhaps I should make a related PR for the Flink engine later.

No problem, it was a real case in my production environment. It's embarrassinging that our administrator found it difficult to identify the target YARN app of engine instance in the list of mixed app names of MR/Spark/and etc when he was investigating other engine issues.
And let's see how other users interested in this minor feature.

Seems we have kyuubi.session.conf.ignore.list, the only difference is it applies to all share levels?

Yes. The CONNECTION level should not be limited to custom app name.
And this feature may be extended to combine share level and the customer app name as the final app name, if needed in the future.

How about introducing a kyuubi.session.conf.sharable.engine.ignore.list?

How about introducing a kyuubi.session.conf.sharable.engine.ignore.list?

It's a considerable approach for this.
It's introducing a new concept sharable engine or sharable share level. And this config may mix configs for different engine types as well.


SIFT2 parameters

This is a question, not a bug, but I can't change the type.
If I have a recon pipeline with "sift: {}", I know that it's using SIFT2 to filter the streamlines (using the streamlines, the normalized FOD image, and outputting the CSV with streamline weights).  What parameters/options is SIFT2 using when called this way in the recon spec?
Thanks

The parameters are named after the options defined here. Sift2 won't filter streamlines, but it will calculate weights for each. The original sift would delete steamlines, but we don't have that implemented in qsiprep for now


chore(docs): delete obsolete code samples

The "hello world" example for Go is here: https://github.com/GoogleCloudPlatform/golang-samples/tree/main/bigtable/helloworld and this code sample references several unmaintained repos and uses OpenCensus (we are encouraging OpenTelemetry these days).

adding @cshaff0524 to take a quick look

Discussed with @cshaff0524 in chat and confirmed it's ok to move forward with this. Thanks!


Lock six version to ~=1.12

Temporarily lock six version to ~=1.12 until next Sprint due to upgrade of package six from ~1.12.0 to 1.13.0 in Nov 6, 2019 breaks dependency, which failed Travis CI
Related PR:  #11051

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

[ ] I adhere to the Command Guidelines.

Astroid issue: https://github.com/PyCQA/astroid/issues/718


[FEA] Enable zstd integration tests for parquet and orc

We do not currently test parquet with zstd codec.  Once CUDF PR https://github.com/rapidsai/cudf/pull/10851 is merged, we should enable zstd testing by changing this line in parquet_test.py:
-parquet_compress_options = ['none', 'uncompressed', 'snappy', 'gzip']
+parquet_compress_options = ['none', 'uncompressed', 'snappy', 'gzip', 'zstd']

This change was reverted by #5916 because it was done too early.

Edited description to include update to orc_test.py.


feat: new stuff

feat: start the styling of components
feat: add most required parts of individual review tiles
feat: worked more on styling
fix: move add review button outside of AddReview component

looks wonderful!


Fix IOS keychain critical bug ⚠

add archiver on data writing/reading to hide data from being attacked and retrieved from malicious tools

the case is when i made dumping attack on the ios keychain over any app used flutter secure storage, i able to see what is the data hade and the screenshots i uploaded demonstrating the difference before and after my changes and it's very important as i assume .

this is the data showed after dumping

and this after i made my changes to hide the sensitive  user data

@juliansteenbakker if there's any thing i can help with in the tests don't hesitate to send me.

Thanks for your work @micheal1hany. it seems like tests are failing due to an unrelated issue right now, which im going to solve first.

Thanks for your work @micheal1hany. it seems like tests are failing due to an unrelated issue right now, which im going to solve first.

Ok fine, i just saw the logs and yes it's related to Gradle. Tyt

Can you please merge the latest changes from develop into your fork? @micheal1hany

Can you please merge the latest changes from develop into your fork? @micheal1hany

Ok no problem but maybe in the morning because it's late here in my time, I'll inform you once done.

Can you please merge the latest changes from develop into your fork? @micheal1hany

Ok no problem but maybe in the morning because it's late here in my time, I'll inform you once done.

No problem, here in my time it's also way to late to be working 😉

Can you please merge the latest changes from develop into your fork? @micheal1hany

Ok no problem but maybe in the morning because it's late here in my time, I'll inform you once done.

No problem, here in my time it's also way to late to be working 😉

Hello @juliansteenbakker , i just pulled develop to my fix-keychain forked branch. please inform me if you need anything else.

Is this change backward compatible in the sense that data stored with the current version of this plugin can be read after this PR has landed. If not, there should be a migration, as the stored data could be one-time generated data such as private keys or similar. If this data can no longer be read, it could break apps and possibly lead to data loss.

Is this change backward compatible in the sense that data stored with the current version of this plugin can be read after this PR has landed. If not, there should be a migration, as the stored data could be one-time generated data such as private keys or similar. If this data can no longer be read, it could break apps and possibly lead to data loss.

Don't worry i tested it many times and it worked perfectly storing/reading data, the purpose of these changes is to fix critical bug reletaed to the storing algorithm in the IOS keychain. With the old way (as string value) can be cracked and stole the user data by ssh connection or direct attack on a specific app.

Is this change backward compatible in the sense that data stored with the current version of this plugin can be read after this PR has landed. If not, there should be a migration, as the stored data could be one-time generated data such as private keys or similar. If this data can no longer be read, it could break apps and possibly lead to data loss.

i think it will not work directly and need to store those data from the beginning to achieve the new way, as a proposal you can make a patch for the app to delete all the keychain with your app/service name and store the data from the beginning

Is this change backward compatible in the sense that data stored with the current version of this plugin can be read after this PR has landed. If not, there should be a migration, as the stored data could be one-time generated data such as private keys or similar. If this data can no longer be read, it could break apps and possibly lead to data loss.

i think it will not work directly and need to store those data from the beginning to achieve the new way, as a proposal you can make a patch for the app to delete all the keychain with your app/service name and store the data from the beginning if it possible for your case.

Your proposal for a patch doesn't make sense. If this PR landed I can't read the data from the keychain to save them in the new way and without this PR I can read the data but can not save them in the new way. So either way it's not working.
Also I don't have a special case in mind, but rather the general case that all currently stored data must be readable after this PR landed.
So I think there should be a migration function. I propose the following function
Future<List<(String key, bool success)>> migrateKeychainValues();

And on the native side(pseudo code):
func migrate(_ args: [(key: String, options: Options)], _ result: @escaping FlutterResult) {
    let res: [(key: String, success: bool)] = []

    for t in args {
        read(...) // using the old way
        if (readSuccessfull) {
            write(...) // new way
            res.add((t.key, writeSuccessfull))
        } else {
            res.add((t.key, false))
        }
    }

    result(res)
}

I consider this PR as a breaking change since every developer has to run the migration at least once before accessing data in the keychain. Also, if the migration partially fails readAll doesn't work properly.
@juliansteenbakker What do you think of my concerns and how should we address them?

I share the thoughts of @bierbaumtim. This would otherwise break all current implementation. Also, if we want to implement this on iOS we should also implement it on macOS, since that is almost the same codebase.


Install from local file (if specified)

Hey Ryan, I added the ability for dynamodb-local to install from a local file if specified... Not sure if you want the code, but it would be nice... since some systems must perform their tasks behind strict firewalls....

Hey Scott - absolutely. Thanks for the PR. Merged and will update version here in a minute

ok published v0.0.22.  Thanks!

Thank you Ryan!!


MLPClassifier and MLPRegressor help files need to link to bigger examples

Because these objects do so much it bloats the help file to introduce audio classification / parameter regression in them.
To help smooth this over we can link to those example patches within the patch.

Solved via docs in https://github.com/flucoma/flucoma-docs/commit/a71d467b09f0a9a1fcebf3cac59a3190d576e928