exit doesn't reliably close the tab

Environment
Windows build number: Microsoft Windows [Version 10.0.18363.720]
Windows Terminal version (if applicable): Version: 0.10.781.0

Steps to reproduce
Use terminal preview for a while, then type exit to close the current tab
Expected behavior
The tab closes. This happens sometimes, particularly on a recently opened tab that hasn't been used much
Actual behavior
A "process exited" message is printed but the tab remains.
c:\src>exit
[process exited with code 2]

I don't know what triggers this - I cannot repro it on demand. I'll watch for it.

I was going to give the usual closeOnExit pitch:

Alas, this is a behavior that's highly conserved across multiple shells and terminal emulators on most mainstream operating systems; more info in https://github.com/microsoft/terminal/issues/4223#issuecomment-574834181 and additionally https://github.com/microsoft/terminal/issues/5040#issuecomment-601855840. If you'd rather it just exit every time, set closeOnExit to always. :smile:

But then after re-reading, it seems peculiar to me that this would happen sometimes and not other times. Maybe the process that's run immediately prior to exit is returning a non-zero exit code, and then we're picking that up from cmd when cmd exits?

Entering a nonexistent command then typing exit yields code 9009. I believe Terminal's broadly incapable of picking up somebody else's exit code -- it's holding a process handle for CMD, after all :smile:

Ah. Exit codes indeed. Trying to run a nonexistent command and then type exit gives 9009. Typing "where asdfasdf" and then typing exit gives exit code 1. And so on.
"where asdfasdf" followed by "where notepad" followed by "exit" works (closes the tab)

I can here to search in regards to this one, as have been getting this for a little bit, just as "randonascii" puts it.

Okay, I figured it out, found the resources online. Looks like exit codes can be used to evaluate the state that a run program exited as, maybe good for scripting. I just didn't know. Obviously you do, just saying ;) But yeah, seems like exiting via 'exit' when %ErrorLevel% != 0, then you get stuck with that error message you cannot break out of. You can still close the tab via the X.
To the original poster, looks whatever you last did before trying to exit triggered a 'code 2':
"The system cannot find the file specified."

I plan to tell people that this command sequence guarantees a clean exit:
cmd.exe /c echo ExitOnError=1 >%temp%\errorhandler.val && exit
It's true, and also deliciously misleading.
Vaguely related: twice I've seen a command prompt exit and print:
[process exited with code 1]
This is without my typing exit. I guess it has crashed?

Using exit 0 avoids this behaviour of the Terminal.  Yes, cmd is using the exit status of the last external command as the value for exit if you do not supply one.

This also happens sometimes with WSL.

I copied it this time. Strange thing though, I couldn't run any exe from WSL either.
wsluser@somecomputer:~$ cmd.exe
wsluser@somecomputer:~$ echo $PATH
/home/wsluser/.local/bin:/home/wsluser/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/WINDOWS/System32/OpenSSH/:/mnt/c/Program Files/OpenSSH-Win64:/mnt/c/Program Files/PowerShell/7/:/mnt/c/Program Files/Microsoft VS Code Insiders/bin:/mnt/c/ProgramData/chocolatey/bin:/mnt/c/WINDOWS/system32:/mnt/c/WINDOWS:/mnt/c/WINDOWS/System32/Wbem:/mnt/c/WINDOWS/System32/WindowsPowerShell/v1.0/:/mnt/c/Program Files/PuTTY/:/mnt/c/Users/user/.vcxsrv/vcxsrv.exe:/mnt/c/Users/user/AppData/Local/Microsoft/WindowsApps:/mnt/c/Users/user/AppData/Local/Programs/Microsoft VS Code Insiders/bin
wsluser@somecomputer:~$ ssh.exe
wsluser@somecomputer:~$ explorer.exe .
wsluser@somecomputer:~$ exit
logout

[process exited with code 1]

Since no errors popped up, I'm assuming the commands were piped to Windows but Terminal didn't handle them correctly.

Since no errors popped up, I'm assuming the commands were piped to Windows but Terminal didn't handle them correctly.

Windows Terminal doesn't "handle" commands. It gets text out of a stream and puts it on the screen, and gets user input from the keyboard and puts it into a different stream.
It looks like you're hitting an interop failure in WSL, which almost certainly warrants a bug filed on their repository.
The exit code from bash is, as always, the last exit code of the last command you tried to run with it. This is by design, and this behavior is highly conserved across many shells on a heap of different operating systems.

Usually I don't get exit codes to exit the tab. This time I did and got this interop error. It's working better than before at least (wsl.conf wasn't being read and interop was completely dead before unless I manually mounted it in the CLI). Using an imported distro is more painful than one would think.

Can we have the window close no matter what the code is? I've got an SSH client as one of my shells that connects to a machine and when I type "bye" on the remote machine, my window always says
[process exited with code<PHONE_NUMBER>]
and won't close. I honestly don't care about the code. Just close the window. Can we do that?

Yeah, you can. Set closeOnExit to always.

You do not need to be patronizing to make your point.

Yeah, wasn't trying to be patronizing. Just didn't know about the always option. Weird that you can set a field to either true or false, or a string value.
Just my opinion, I think "always" would be the preferred default option for most people. I've been a Windows user and software developer for over 25 years and couldn't think of a single instance in that time I wanted to know what exit code a shell or program closed with. I'm sure there's reasons people would want it, but just seems unnecessary by default.
Thanks for the quick response, though. :)

So, the main thing we were combatting is shells exiting immediately on open; it was one of the huge perceived reliability issues in versions pre-0.5 that was largely out of our control. We’ve got a plan on the books to make it more like ConEmu and only show you the message if the process eats it within five seconds of launching, but that one didn’t make the cut for 1.0.
Sorry for being so trigger-happy to call you out. 😄

(If you’re coming from a version of Terminal settings where closeOnExit is a Boolean value, it might be a great time to refresh your settings by moving them aside and letting us regenerate it. The new defaults are a lot more sane than the old ones, but they might also change some things you’re relying on. YMMV.)

Perhaps related: a couple of times a cmd window in the Windows Terminal (Preview) has spontaneously printed "[process exited with code 1]" without me typing exit. This seems to happen when I'm not even interacting with it. It feels like it has just randomly quit.
I have WER configured to get crash dumps and I haven't seen any so I don't know what's going on. It doesn't seem worth filing a bug because I have no real information, but I thought I'd mention it here.

Huh.. that could be a crash in the PTY host. Would you mind keeping your eyes peeled for OpenConsole dumps? This could be one of our known issues if it’s related to resizing (which I can’t immediately evoke by name on my phone.)

I don't know if I should open a new issue for this, but I'm starting to have issues with 'exit' causing WT as a whole to freeze. Before today's update it had started to randomly freeze on a 'exit' then after a few seconds the whole WT session would disappear. Now it's just freezing and staying that way.
As I see it, it's still a "exit doesn't reliably close the tab" kind of thing, so if I need to open a different issue, let me know.

@Firehawke We should definitely track that one as a separate issue! Would you mind capturing a dump file from Task Manager (WindowsTerminal.exe process) when it's hung like this? You can e-mail me with the address on my GitHub profile.

Okay, I'll open a new issue as soon as I'm able to trigger it again.

This also happens for me. It can reliably be reproduced with WSL (Ubuntu) for me.
How to see the issue:

Without typing any command, press CTRL + D -> tab closes
Open another WSL tab
Type an invalid command ('frwgufgwe' should do)
Press CTRL + D -> "[process exited with code 127]"

This is after the latest update the FIRST (without Preview) release.
And the exit command still doesn't appear to be fixed.

@raxraj as mentioned earlier in this thread:

Using exit 0 avoids this behavior of the Terminal. Yes, cmd is using the exit status of the last external command as the value for exit if you do not supply one.

You can also just set closeOnExit to always in your settings.json.

You can also just set closeOnExit to always in your settings.json.

For those that may not know, closeOnExit is a profile-specific setting. I had to go look up the settings schema docs to verify.

@HolisticDeveloper @zadjii-msft  Thanks, I understood. Solved the problem :)

Just got the same thing when closing a WSLv1 tab

What's bad is that it's completely unclear in the UI if WSLv1 is still running or not and what process exited with this exit code. I thought that some wrapper process is still running and typed Ctrl+C multiple times which does not work. Enter also doesn't work to close this. If it's Terminal that handles printing this message, the message should better include an explanation what happened and what's running / what's closed.

That's fair. The message should say "the root process, wsl.exe, exited and reported an error code. everything underneath it has been either terminated or detached from the console".
Sorry, that's the best explanation we can give. :smile:

Well, this explanation is way better than the message that I got just minutes ago. At least it's clear that WSL is dead and that I should close the tab.
I thought some WSLv1 process (like Ubuntu's command_not_found that loads up python and does search for modules in a gazillion of site locations) got hung so badly that doesn't respond to Ctrl+C and blocked the WSL instance.

Not to beat a dead horse but check the docks:
https://docs.microsoft.com/en-us/windows/terminal/customize-settings/profile-settings#how-the-profile-closes-when-exiting
closeOnExit to always specifically under the profiles.defaults section of settings.json fixed it for me.
Cheers,

I've read that closeOnExit is a profile especific feature. Seems not anymore, I've edit the settings.json and defined the setting in general (not inside the profiles part) and worked. That should be default behavior.

I've read that closeOnExit is a profile especific feature. Seems not anymore, I've edit the settings.json and defined the setting in general (not inside the profiles part) and worked. That should be default behavior.

You will probably find, next time you Save your settings using the UI, that it deletes that key from the general section. It is not expected there, and it will be ignored. :)


Add PyQtChart.

Signed-off-by: Adrian Fiergolski<EMAIL_ADDRESS>

Thanks, in master-next now 0bbd0ebe..caf09cd5

Perfect, could you move it to zeus-next as well (link)?

Perfect, could you move it to zeus-next as well (link)?

Yes, plan to, after initial testing in master-next.

In master now, thanks
23914740..abb42351  master -> master

@shr-project, will you move it to zeus-next?


Add enhancements to sriov controller

Transfer enhancements and fixes for sriov controller to networkservicemesh repo
Signed-off-by: Serguei Bezverkhi<EMAIL_ADDRESS>

@mestery Addressed your comments PTAL


[PT-BR] New sentences and some non-translated are now translated

As requested by @tetele in https://github.com/home-assistant/intents/discussions/1768#discussioncomment-7830669
based on https://github.com/home-assistant/intents/pull/1767
waiting for: https://github.com/home-assistant/core/pull/105524

I hope it's ok now.

There are currently 3 PRs for PT-BR with several overlapping intents:
https://github.com/home-assistant/intents/pull/1798
https://github.com/home-assistant/intents/pull/1783
https://github.com/home-assistant/intents/pull/1413
I'm flagging them here so we can clear those upon merge.
PS: they are all missing the label lang:pt-br, don't know why the bot missed them.

Hi guys, when will this PR be merged?
Who needs to approve

I can't find it atm, but I remember telling @jmceara somewhere that he shouldn't place all of the Brazilian Portuguese translations in this PR. If I haven't, I apologize and I'm saying it now.
So... the untranslated-state branch is strictly for exposing the untranslated weather and person states to the sentences, for better handling in some limited situations. That will get merged into main depending on what happens to the upstream PR.
But until then, if you want to have pt-br translated regardless of what happens in the core, you should create a PR with the translations against the main branch, not against the untranslated-state branch. If you do, @synesthesiam can take a look and decide to merge the PR and grant someone pt-br language leader status, so they can handle this stuff on their own.
Also, regarding this specific PR, I've approved the CI pipeline run and linting fails, so it can't be merged regardless. Run script/lint locally to fix the issues and push another commit.

Hi @tetele , thank you !
@llluis @jmceara
Can we do this ?
If you guys need agree, I can send this PR to the main branch

Hi @tetele , thank you !
@llluis @jmceara
Can we do this ?
If you guys agree, I can send this PR to the main branch

Agree!

@jmceara Hi, the issue here is that there are 16 space characters left in the 7th line of the sentences/pt-br/homeassistant_HassNevermind.yaml file.
If there are errors remaining after running the script/lint, it is advisable to reconfigure the environment using the script/setup command.

Oh, I see now, this isn't even going into the main branch :P Sorry

My vote doesn't count but I reviewed and approved anyways. :)

If everything is perfect, then @luyzfernando08 it's your job to press the Approve and Run button :) You are the language leader.
Unfortunately, I don't understand Portuguese :) :D

If everything is perfect, then @luyzfernando08 it's your job to press the Approve and Run button :) You are the language leader. Unfortunately, I don't understand Portuguese :) :D

Well, I guess my appliance was not accepted LOL

Thanks @jmceara! That was a quick response. :)
I am just going through more details here. Looks like this will not be merged as it is now.
According to @tetele, we need to make this changes against the main branch, not untranslated-state.
Regarding #1848 for weather and person, this PR already contains the changes for pt-br, should this be split in two different PRs? @tetele

This branch into which the PR was merged is the "untranslated-state" branch, which as you also write, is intended for the handling of weather and person in "untranslated-state" to function if introduced.
What is currently in there will fit, it won't bother anyone :). But what you would like to use from this that is not related to person and weather, you will need to implement into the main as well.
It's important that any potentially missing parts of the _common.yaml file should definitely be added to the main branch too, to make the future merging easier.

Thanks @v1k70rk4 ,
Ok, then we can just create a new PR against main, without the changes in HassGetState.yaml and HassGetWeather.yaml.
Did I understand it right?

@gohlas As you say :) If there is no conflict, there is nothing preventing this.
But you can also put the weather sentences in reduced mode into the main, so it will only display the temperature, but it will also work there.
For example, in the responses\pt-br\HassGetWeather.yaml file, this is the response:
language: pt-br
responses:
  intents:
    HassGetWeather:
      default: >
        {{ state.attributes.get('temperature') }} {{ state.attributes.get('temperature_unit') }}


docs(zh): keep updated

Ref: https://github.com/vuejs/vue-cli/compare/686ec259...78c68772
Deployment 未完全写好，固暂时保留了英文
/ping @Justineo
Thanks.

@Justineo do you mind to take a review? 😅
Updated to b1b04b8a
https://github.com/vuejs/vue-cli/compare/686ec259...b1b04b8a


Deploy Day-3 Infrastructure via Argo CD

Change the lab so the student deploys NGINX-Ingress, Grafana, and Prometheus via Argo CD using manifests that drive Helm deployments.
We also need Argo CD to update the deployment once a modification to the underlying helm chart is committed in the student's repo.

Sorry about issue 2! I wasn't paying attention to this https://github.com/f5devcentral/modern_app_jumpstart_workshop/blob/6e6480b5e54d99cea6bbfae2d9dd4915b912f815/docs/ingress/setup.md?plain=1#L44


Run commands with the user's shell

This updates the logic to use ENV["SHELL"], if set, and falling back to /usr/bin/env bash otherwise when opening a new window and new tabs. Based on this table, many shells support the -l option, so this shouldn't have any adverse effects.
The changes to the gemspec were required to get the gem to build properly.

Thanks! I've pushed this into 2.1.0.

Awesome, thanks for the quick turnaround!


tf.contrib.slim evaluation

authors, thanks for opensourcing your code.
I would like to know why did you choose tf.contrib.slim instead of tf.layers or just tf.keras?
I'm trying to reimplement your using tf.keras which could be more simpler for more expremintation with tf.eager_execution to debug easily? what are implecations I should take to use tf.keras?
Thanks!!

Our internal framework is still dependent on older TF versions. We're currently migrating to newer versions and plan to adopt appropriate coding style in the future, but for now, we decided that releasing our code was more important than waiting for the migration to complete.

HI，I made a few modification，and it can run on windows now.  But how to use the models I trained?

do u have a demo about how to use this models？thanks.
my email<EMAIL_ADDRESS>

Hi @tony2278 @shurain
I tried to train for MModel, but never finished.
Please let me know valid training.


Add support for ignore patterns

The web app config should be set per target, using a schema like: HashMap<Regex, Config>, where the worker would pick the first config that match the job name of the given target.


How to not make sidebar close with links?

Nice directive you have here! I have removed the ui-turn-off='uiSidebarLeft' but the sidebar still closes when a link is clicked inside the sidebar. Is there a way to remove this effect? Thank You.

Yes, you can use ui-state="uiSideBarLeft" instead as you probably did and remove the  ui-track-as-search-param='true' attribute.
But if you use links (href) to navigate, it will still close on navigation.
This is the default behaviour for now, and is not easily changed.


Support passing multiple patterns through file_pattern

Will this break existing users with the change of the input name?

I only chagned the variable name inside the script. The action still uses file_pattern as the input and passes it as argv[5] to the entrypoint


net: ZWP timeout explicit type on bit shift calculation

Fix tcp_send_zwp() retransmission timeout calculation to avoid sign extension on bit shift operation:
the uint16_t value was implicitly promoted to 32-bit signed type with sign extension to uint64_t.

The problem shouldn't have side effects with the current ZWP_MAX_DELAY_MS value of 120000.
It was detected by the Coverity scan with the following explanations:
Unintended sign extension (SIGN_EXTENSION)
sign_extension: Suspicious implicit sign extension: conn->rto with type uint16_t (16 bits, unsigned)
is promoted in conn->rto << conn->zwp_retries to type int (32 bits, signed),
then sign-extended to type unsigned long long (64 bits, unsigned). If conn->rto << conn->zwp_retries is greater than 0x7FFFFFFF, the upper bits of the result will all be 1.

Bad bit shift operation (BAD_SHIFT)
large_shift: In expression conn->rto << conn->zwp_retries, left shifting by more than 31 bits has undefined behavior.
The shift amount, conn->zwp_retries, is as much as 255.

Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN)
overflow_before_widen: Potentially overflowing expression conn->rto << conn->zwp_retries with type int (32 bits, signed) is evaluated using 32-bit arithmetic,
and then used in a context that expects an expression of type uint64_t (64 bits, unsigned).
To avoid overflow, cast conn->rto to type uint64_t.


NativeCallableAttribute Issue

While brainstorming ideas on how to workaround Marshal.GetFunctionPointerForDelegate() being unimplemented, I came up with the idea of using NativeCallableAttribute to give the method in question a C name, and then use a C stub method (exposed to C# as [DllImport("*")] method) to convert the name of the function into its pointer, which then can (hopefully) be passed safely into Win32.
Unfortunately, ILC is not emitting the entry-point thunk for the method in question. The symbol does not appear in the emitted object file, and as such the linker fails when I try to link in the name-to-pointer stub method. I can prove the method itself is compiled (both by running dumpbin /symbols on my object and by examining the verbose ILC output); the NativeCallableAttribute is just not honored. Weirdly, though, the NativeCallable functions in the CoreRT runtime are exported (and visible in dumpbin /symbols) -- good thing, too, or else my app would emit far more linker errors than it does.
If it helps, I am defining System.Runtime.InteropServices.NativeCallableAttribute in a separate assembly than System.Private.CoreLib, because this makes it easier for me to reference the type from my application code. Having reviewed ILC's handling of this type, I do not think that the assembly containing the attribute is considered, but I could be wrong.

It should work.
I would set at breakpoint here to check why your custom methods are not getting picked up: https://github.com/dotnet/corert/blob/d85dc4f6b9912e002d0a41ce08c06cf394c7ee08/src/ILCompiler.Compiler/src/Compiler/CompilationModuleGroup.cs#L97

I figured it out, actually. It seems that only the input module and System.Private.CoreLib are scanned for methods marked with NativeCallableAttribute. @jkotas I can prepare a PR with a fix for this, if desired.

Closing as resolved; please see #1510 for the fix.


Rotation Tool: display predefined rotation angles and current angle

Enhance the rotation tool introduced in scope of #2090, so that user can see predefined rotation angles' marks as well as current rotation angle and can easier rotate the selection on needed angle. The rotation tool should work as follows:

Once user clicks the rotation handle and rotation mode is activated, the rounded rectangle should be replaced with a circle. Center of the circle should be in the center of rotation, circle diameter equal to the height of replaced rectangle. The rotation handle should not change its position while switching from selection to rotation mode and back.
The following predefined angles should be marked on the rotation circle: -90, -60, -45, -30, 0, 30, 45, 60, 90
Once user starts rotating the structure, the set of angles shown should always cover [-90 - +90] range relatively to the handle position, e.g. if the structure is rotated for 60 degrees, user should see -30, 0, 30, 45, 60, 90, 120, 135, 150
Current rotation angle should be displayed at starting position (0 degree).
Once user releases left mouse button, the rotation mode is deactivated and selected fragment stays in its new position (rotated). Rotation circle should be replaced with rounded rectangle (handle on top).

#184121704

Once user starts rotating the structure, the set of angles shown should always cover [-90 - +90] range relatively to the handle position, e.g. if the structure is rotated for 60 degrees, user should see -30, 0, 30, 45, 60, 90, 120, 135, 150

One question here, according to ''the set of angles shown should always cover [-90 - +90] range relatively to the handle position", angles of the image above should be like

Once user starts rotating the structure, the set of angles shown should always cover [-90 - +90] range relatively to the handle position, e.g. if the structure is rotated for 60 degrees, user should see -30, 0, 30, 45, 60, 90, 120, 135, 150

One question here, according to ''the set of angles shown should always cover [-90 - +90] range relatively to the handle position", angles of the image above should be like

This looks a bit wierd huh... Which one is correct?

It should show "nice" degrees between [currentDegree - 90, currentDegree + 90]


TorrentGalaxy downtime/maintenance missing torrents

I noticed that there are gaps in torrentio TorrentGalaxy scraping around 20/09/2024 - 21/09/2024 and torrents are missing from the torrentio db. Hope it's possible to re-scrape the missing torrents.
Thank you and keep up the good work!

I think it indexed everything, unless you can share an example?

From TGx id 16173525 (20/09/2024 07:16) to 16174723 (21/09/2024 13:25).

Reindexed that id range again and some new netries were added. Thanks

@TheBeastLT There is a new gap in the TGx id 16224xxx (13/11/2024) area. Hope you can re-scrape.

@Joerg87 I've reindexed this range, but no new entries were added, so don't think anything is missing fro tgx. Some entries can be indexed from other providers first, so they might appear under a different provider.

@TheBeastLT Is TorrentGalaxy scraping currently down? Since the site came back up a lot of torrents are missing from the torrentio DB. Example ids 16252394 16254311. Can you re-scrape from 16252000 to now?


Add README.md as a source for the description for preview page

Description and overview of changes

Addresses https://github.com/chanzuckerberg/napari-hub/issues/1023
I ran black on this file. To view the functional changes, see this commit: https://github.com/chanzuckerberg/napari-hub/commit/beac6602e28d83f9d43bcd14333c8182cf78422b

Notes

The reason we appear to render the README.md as the description on the napari hub site currently is that pypi uses that file to populate its description field. However, we don't specifically examine the github README.md ourselves (this PR will change that).
I am assuming the reason for the discrepancy between the napari hub plugin page, and the preview page (in terms of what is available for the description field) is due to what is available in the wheel's metadata at the time of the preview page action. Haven't verified that though.

I am assuming the reason for the discrepancy between the napari hub plugin page, and the preview page (in terms of what is available for the description field) is due to what is available in the wheel's metadata at the time of the preview page action.

This is correct. Although the long description is present in the wheel's METADATA file, it's not parsed as part of the package metadata for some reason. PyPI have their own logic to grab this long description, and they store it in the description field.
This means in the current (on main) implementation for the hub, we always have the PyPI description stored in the metadata, and it's overwritten by the content in DESCRIPTION.md (if present). The approach in this PR would also overwrite the PyPI description if the DESCRIPTION.md is present, but it will also re-overwrite the description field with the README.md content if DESCRIPTION.md is not present.
This is unlikely to be an issue, as I would imagine there are probably only a handful of packages on PyPI that point to a different file than the README.md for their description, but it is possible, and in that case, we would overwrite the desired description with README.md content.
It's really annoying that the long_description is in the METADATA file in the wheel, but we have (as far as I can tell) no blessed way to access it before it hits PyPI. The closest I've found is build's project_wheel_metadata, which has the description available, but under a private attribute 🤔 .
I think if we're explicit that we're rendering the README.md and not the PyPI description field (even though they're in 99.9999% of cases the same), then this implementation should be ok and I can look at adding tests. Happy to chat more when you're back @richaagarwal

Thanks for looking into this, @DragaDoncila!

but it will also re-overwrite the description field with the README.md content if DESCRIPTION.md is not present.

Ah good catch! I'd be a little hesitant to move forward with this change in that case, as we'd probably want to continue for the hub plugin page to display any PyPI-specific description in the absence of a napari-specific description, even if it's not a common scenario that it would deviate from what's in the README.
If there was an easy way to isolate this change to the preview page only, that might be worth considering.
@DragaDoncila I'll reach out to discuss further, but you can hold off on testing for now.
cc @neuromusic


Does route support FastRoute\cachedDispatcher?

I was not a fan of Front Controller pattern in PHP, but I am trying to force myself to follow the movement of open source frameworks. So I think FastRoute\cachedDispatcher could reduce the overhead of dispatching process as much as possible (of course, I know this is a premature optimization). I wonder, does league's route support it? I cannot found any declaration of that class in the source code.

Hi, the cached dispatcher is not currently implemented but it is in mind for the next feature release 2.1.0

(y) Hope version 2.1.0 could be released soon.

Having had a look at the simplicity of this it may be something that we can introduce in to the upcoming 2.0.0 release. I will take a look at the best way to implement it when dispatching and attach results to this issue.

Hi @philipobenito,
I'm happy to help with introducing this caching mechanism. Do you have some thoughts already on the approach?
Currently my proposition would be to introduce an indirection in the getDispatcher() method.
Thanks,

@wtracz It is actually a little more complex than originally though, we cannot just enable caching via FastRoute as league/container adds extra functionality. It will have to be a completely new caching system that I'm aiming to look at after the 2.0.0 release.

This is now out of the scope of what can be done so closing for now. However, I will come back to the idea of caching or compiling a route collection at some point.


Suite Setup failure: Associate TC suite failure

@gkeishin Feel free to assign to either Sridevi and Swetha as per our scrum discussion today. We can assign them based on the work priority.

Pls refer saturday's driver of GHE CT execution.

Most of the suite failure are now fixed via few smaller other commits for different module suite fixes.. closing this one.


Add auto-fix for ARG

def a(b):
    pass

could be auto-fixed like this
def a(_b):
   pass

This is similar to the B007 auto-fix.

Not sure this is safe unless the argument is positional only. This could be called as a(b=1), so autofixing as above, even if unused in the definition, could break at the call sites.

Yeah I think this is too risky. It also may "mask" the error in some cases where the correct fix is too remove the argument entirely.


Can't disable JFXDialog dropshadow

Hello,
I'm building an embedded touchscreen application on Raspberry Pi, using JFoenix to design the UI.
Currently, I'm running into a performance problem: When a JFXDialog is opened up, the application lags (i.e. Button Presses on the Numpad-Popup happen with about half a second of delay).

The relevant code for loading this Numpad is this:
public class PinController implements Initializable {

    public static void show(int digits, Consumer<String> handler) {
        JFXDialog dialog = new JFXDialog();
        EventHandler<ActionEvent> close = (e) -> dialog.close();
        PinController controller = new PinController(digits, close, (pin) -> {
            dialog.close();
            handler.accept(pin);
        });

        Region content = loadFxml(controller);
        dialog.setTransitionType(JFXDialog.DialogTransition.CENTER);
        dialog.setDialogContainer(StageController.getRoot());
        dialog.setOverlayClose(false);
        dialog.setContent(content);
        dialog.show();
    }

    private static Region loadFxml(PinController controller) {
        try {
            FXMLLoader loader = new FXMLLoader();
            loader.setLocation(PinController.class.getResource("/views/fragment/Numpad.fxml"));
            loader.setCharset(StandardCharsets.UTF_8);
            loader.setController(controller);
            return loader.<Region>load();
        }
        catch (IOException ex) {
            throw new RuntimeException(ex);
        }
    }

It works without problem on Desktop, but as I said above, it lags on the Raspberry Pi.
I looked for a way to disable the Dropshadow, which I suspect causes this performance problem, but I found this in the JFXDialog class:

    private void initialize() {
        this.setVisible(false);
        this.getStyleClass().add(DEFAULT_STYLE_CLASS);
        this.transitionType.addListener((o, oldVal, newVal) -> {
            animation = getShowAnimation(transitionType.get());
        });

        contentHolder = new StackPane();
        contentHolder.setBackground(new Background(new BackgroundFill(Color.WHITE, new CornerRadii(2), null)));
        JFXDepthManager.setDepth(contentHolder, 4); // <-- Relevant Line
        // ...

So the dropshadow is apperantly hardcoded with JFXDepthManager.setDepth(contentHolder, 4);, and the private StackPane contentHolder; is private, so I can't even extend the class and remove the dropshadow this way.
It'd be great to have a way to disable this JavaFX effect, i.e. if the -fx-effect style was picked up by the JFXDialog.
Thank you and have a nice day,
Alexander Dahmen

Small update:
Using reflection, I made a small quick-and-dirty workaround:
public class CustomJFXDialog extends JFXDialog {

    private final StackPane contentHolder;

    public CustomJFXDialog() {
        super();
        this.contentHolder = accessPrivate("contentHolder", StackPane.class);
    }

    public void setDropshadow(boolean dropshadow) {
        if (dropshadow)
            JFXDepthManager.setDepth(contentHolder, 4);
        else
            contentHolder.setEffect(null);
    }

    private <T> T accessPrivate(String name, Class<T> type) {
        try {
            Field field = JFXDialog.class.getDeclaredField(name);
            field.setAccessible(true);
            Object value = field.get(this);
            return type.cast(value);
        }
        catch (IllegalAccessException |
                IllegalArgumentException |
                NoSuchFieldException |
                SecurityException ex)
        {
            throw new AssertionError(ex);
        }
    }
}

Having a property on the standard JFXDialog and not having to use reflection would still be really neat, though :)

you can use the following css:
.jfx-dialog-overlay-pane > StackPane{
    -fx-effect: null;
}

Regards,


Error: write after end

Hi,
we are using RethinkDBdash as our rethinkDB driver along with Nodejs. Its working great. But i started a bit of stress testing on my dev machine, and when I send 500+ concurrent requsts per second, the server is crashing with the following error:
events.js:160
throw er; // Unhandled 'error' event
^
Error: write after end
    at writeAfterEnd (_stream_writable.js:166:12)
    at Socket.Writable.write (_stream_writable.js:217:5)
    at Socket.write (net.js:651:40)
    at Connection._sendProof (/home/chattes/dev/LocationTracker/prototype/node_modules/rethinkdbdash/lib/connection.js:281:19)
    at Connection._computeSaltedPassword (/home/chattes/dev/LocationTracker/prototype/node_modules/rethinkdbdash/lib/connection.js:247:10)
    at Socket.<anonymous> (/home/chattes/dev/LocationTracker/prototype/node_modules/rethinkdbdash/lib/connection.js:184:18)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at Socket.Readable.push (_stream_readable.js:134:10)
    at TCP.onread (net.js:543:20)

We are using Node 6.5.0 and "rethinkdbdash": "^2.3.23"
Request you to kindly help with the same.
Thanks

My guess is that you're hitting your max open files limit and can't properly open connections after that. Looking at the stacktrace, it's probably a bug introduced with the new handshake, 2.2.x should work for now.
It should be relatively straightforward to fix.

Okay let me try out with 2.2.x. I will keep you posted. although i increased my max pool to 10000 after your suggestion and it disappeared for a while. But it has come back again today.
Thanks

Hi quick question this error 'Fail to create a new connection for the connection pool. Error:{"message":"Failed to connect to localhost:28015 in less than 20s.","isOperational":true}'
Will it try and create connections once sockets are available again?

@neumino any updates on this? I'm getting the exact same error. I only started getting this once the number of connections per second increased. The connection is made, and then after a few seconds it is lost, and no new connections can seem to be made.
Fail to create a new connection for the connection pool. Error:{"msg":"Failed to connect to cluster.local:28015 in less than 5s","message":"Failed to connect to cluster.local:28015 in less than 5s.","isOperational":true}

Here is my driver file:
driver.js
const driverConfig = {
  authKey: process.env.RETHINK_DB_AUTH_KEY || '',
  buffer:  process.env.NODE_ENV === 'production' ? 50 : 3,
  db:      process.env.RETHINK_DB_NAME,
  host:    process.env.RETHINK_DB_HOST || 'localhost',
  min:     process.env.NODE_ENV === 'production' ? 50 : 3,
  pool:    true,
  port:    process.env.RETHINK_DB_PORT || 28015,
  timeout: process.env.RETHINK_DB_DRIVER_TIMEOUT || 5
}
const r = require('rethinkdbdash')(driverConfig)
module.exports = r

I changed my driver config to:
const driverConfig = {
  authKey: process.env.RETHINK_DB_AUTH_KEY || '',
  buffer:  process.env.RETHINK_DB_DRIVER_BUFFER || 50,
  db:      process.env.RETHINK_DB_NAME,
  host:    process.env.RETHINK_DB_HOST || 'localhost',
  pool:    true,
  port:    process.env.RETHINK_DB_PORT || 28015,
  timeout: process.env.RETHINK_DB_DRIVER_TIMEOUT || 20,
  timeoutGb: process.env.RETHINK_DB_DRIVER_TIMEOUTGB || 60 * 60 * 1000
}

And the issue has gone away for the moment.


Performance Tests

This is more of a question than an issue.  Looking at the unit tests I don't see any performance tests, is it possible to add some to validate that the number of generated ids do not collide, i.e. they are unique for a given time interval within the mask resolution?

CreateId_Throws_OnSequenceOverflow tests specifically for overflow with the mask resolution.
You could test for other masks but that doesn't make much sense; you can't test every possible combination (within reason); just as you also don't test the + operator for all combinations of operands.
Suggestions for specific tests are welcome though.


GPU DPS build

Building GPU enabled image on the regular MAAP build infrastucture (CPU) should be possible.
We might need to insert some env variables to ensure conda solves to the GPU versions of packages.
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-virtual.html#overriding-detected-packages
CONDA_OVERRIDE_CUDA=12 ENV variable
or potentially specify the cuda version of a conda package with packagename=*=*cuda*
noting isce3 could be one of the following, which might avoid the need to set the ENV variable above
- isce3-cuda=0.24.1
- isce3>=0.24.1=*cuda120*

https://github.com/MAAP-Project/sar2-d2/blob/main/environment.yml
https://github.com/MAAP-Project/sar2-d2/blob/dps/environment-dev.yml

Note maap-py is only in the dev yml right now, does that need to be moved to the regular one @nemo794 ?

@nemo794 can you ask the ISCE3 team if it safely falls back on CPU if no GPU is available so we only have to worry about building 1 image/algorithm?

https://github.com/MAAP-Project/sar2-d2/blob/gcov_pipeline/environment.yml is good for maap-py but needs the isce3 change noted above to make sure to pull the GPU compatible version.

Looks like this is handled the correct deps are included:
cuda-cudart                12.6.77       h5888daf_0                   21.9 KiB   conda  cuda-cudart-12.6.77-h5888daf_0.conda
cuda-cudart_linux-64       12.6.77       h3f2d84a_0                   184.2 KiB  conda  cuda-cudart_linux-64-12.6.77-h3f2d84a_0.conda
cuda-version               12.6          h7480c83_3                   20.4 KiB   conda  cuda-version-12.6-h7480c83_3.conda

Now I just need to write a test DPS algorithm that checks the GPU status based on this private ticket https://github.com/NASA-IMPACT/veda-analytics/issues/130 which uses Tensorflow to test since the ISCE3 test is not good. https://github.com/isce-framework/isce3/blob/release-v0.23/tests/python/packages/isce3/core/gpu_check.py
@chuckwondo if you have ideas of another way to test let me know.

@wildintellect, in VEDA, I launched a TF instance, created a custom conda env with only isce3 installed w/cuda (isce3>=0.24.1=*cuda120*), activated the custom env and simply ran the following, showing that 1 GPU is available:
$ python -c "import isce3; print(isce3.cuda.core.get_device_count())"
1

Doing the same in a non-NVIDIA instance, produced this error, as expected:
$ python -c "import isce3; print(isce3.cuda.core.get_device_count())"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
RuntimeError: Error in file /home/conda/feedstock_root/build_artifacts/isce3_1731982147617/work/cxx/isce3/cuda/core/Device.cu, line 41, function int isce3::cuda::core::getDeviceCount(): cudaError 35 (CUDA driver version is insufficient for CUDA runtime version)

In the MAAP ADE, I followed the same steps and also produced the RuntimeError.
I think in DPS, an algorithm run bash script with the following line should suffice for determining the availability of a GPU:
# Adjust conda env name, if necessary
conda run -n isce3 python -c "import isce3; print(isce3.cuda.core.get_device_count())"

If the job fails, then no GPU is available. If is succeeds, you have at least 1 GPU available.

@chuckwondo
Do we really want the job to fail, or should we wrap that in a try so that it passes either way?
Since this is an ISCE3 specific way, should it go in this repo as alternate algorithm?

@chuckwondo Do we really want the job to fail, or should we wrap that in a try so that it passes either way, and log the results? Could also pass a T/F arg to the alg on if a fail at this step should abort the run.

We could make the script succeed no matter what, if we want to avoid failing the job. All we need to do is append || true to the line in the bash script:
conda run -n isce3 python -c "import isce3; print(isce3.cuda.core.get_device_count())" || true

That should ensure that we always get an exit code of 0, but if there's no GPU, we'll see the error message captured in the _stderr.txt file.
Of course, there are alternatives, including your suggestion for exposing a boolean input. We could even do as you said and wrap things in a try/except block and avoid generating a traceback altogether, and simply output 0 as the number of GPUs in that case.
Up to you, really. What are you wanting to achieve more generally? Do you want users to be able to use this as a means of checking that a particular queue does or does not launch GPU instances?

Since this is an ISCE3 specific way, should it go in this repo as alternate algorithm?

This is isce3-specific only because that's how this all began, but if we want to not use isce3, then perhaps we find a more general library for testing for the availability of a GPU.
Either way, I'd lean towards not adding this as an alternate algorithm in this repo, particularly if you want to be able to use this more generally as a GPU test, especially since the logic is not tied to this repo in any way, other than that it currently uses isce3 to perform the GPU test, but that's not a necessity.


L4-L7 traffic and emulation support

We need to extend existing model to support vendor-agnostic L4-L7 traffic and emulation testing testing.
Model structure for L4-L7 protocol support is as follows:

The L4-7 session configuration shall be part of root config, sitting next to device and flows.
The interface properties (ethernet and IP) shall be configured as part of device..
The L4-7 session configuration, aka app shall be used to configure protocols such as:

L4 protocols : TCP and UDP
L7 protocols : HTTP

flows shall be used to configure end points of data traffic configurations and parameters as test objectives, test durations, etc.

Each flow will have end points(tx,rx) in  traffic.
For example if we have 2 flows then 2 tx and rx will be configured.

Sample of the script using the above model will be represented like below:

Please refer to updated proposal and code snippet in : https://github.com/open-traffic-generator/models/issues/238#issuecomment-1176097943
Sample script for HTTP config
import snappi
api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>5")
confObj = api.config()
port_1 = confObj.ports.port(name="p1", location='<IP_ADDRESS>/1/1')[-1]
port_2 = confObj.ports.port(name="p2", location='<IP_ADDRESS>/1/2')[-1]

rx_device, tx_device  = confObj.devices.device(name="client").device(name="server")

rx_device.port_name = port_1.name
tx_device.port_name = port_2.name
#Client Configuration
rx_eth = tx_device.ethernet
rx_eth.name = "rx_eth"
rx_eth.mac = "00:00:00:00:65:01"
rx_eth.name = "eth_r1"
rx_eth.mtu = 1500

rx_ipv4 = rx_eth.ipv4
rx_ipv4.name = "rx_ipv4"
rx_ipv4.address = "<IP_ADDRESS>"
rx_ipv4.prefix = "24"
rx_ipv4.gateway = "<IP_ADDRESS>"

rx_tcp = rx_ipv4.tcp
rx_tcp.keep_alive_time = 700
rx_tcp.receive_buffer_size = 4096
rx_tcp.transmit_buffer_size = 4096
rx_tcp.fin_timeout = 60
rx_tcp.syn_retries = 5
rx_tcp.source_port_range.min = 1024
rx_tcp.source_port_range.max = 65535

http_client = rx_tcp.http[-1]
http_client.role = "Client"
http_client.name = "HTTP Client1"
http_client.enable_ssl = "True"

#server configurations

tx_eth = tx_device.ethernet
tx_eth.name = "tx_eth"
tx_eth.mac = "00:00:00:00:70:01"
tx_eth.name = "eth_S1"
tx_eth.mtu = 1500

tx_ipv4 = tx_eth.ipv4
tx_ipv4.name = "tx_ipv4"
tx_ipv4.address = "<IP_ADDRESS>"
tx_ipv4.prefix = "24"
tx_ipv4.gateway = "<IP_ADDRESS>"

tx_tcp = tx_ipv4.tcp
tx_tcp.keep_alive_time = 700
tx_tcp.dest_port_range.min = 1024
tx_tcp.dest_port_range.max = 65535

http_server = tx_tcp.http()[-1]
http_server[0].role = "Server"
http_server[0].name = "HTTP Server"
http_server[0].enable_ssl = "True"
http_server[0].method.get(name="get", response=200, url="/b.html").post(name="post", response=201, url="/128k.html")

http_client.method.get(name="get").post(name="post")

#Flow Configurations
flow = confObj.flows.flow()
flow.tx_rx.tx(name=rx_device.name)
flow.tx_rx.rx(name=tx_device.name)

#Objective and Traffic configurations
flow.test_type = "ConnectionPerSecond"
flow.test_value = 100
flow.duration.ramp_up = 10
flow.duration.sustain_time = 180
flow.duration.ramp_down = 180
api.set_config(confObj)
api.control(state= start)

Sample script for TCP:-
import snappi
api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>5")
confObj = api.config()
port_1 = confObj.ports.port(name="p1", location='<IP_ADDRESS>/1/1')[-1]
port_2 = confObj.ports.port(name="p2", location='<IP_ADDRESS>/1/2')[-1]

rx_device, tx_device  = confObj.devices.device(name="client").device(name="server")

rx_device.port_name = port_1.name
tx_device.port_name = port_2.name
#Client Configuration
rx_eth = tx_device.ethernet
rx_eth.name = "rx_eth"
rx_eth.mac = "00:00:00:00:65:01"
rx_eth.name = "eth_r1"
rx_eth.mtu = 1500

rx_ipv4 = rx_eth.ipv4
rx_ipv4.name = "rx_ipv4"
rx_ipv4.address = "<IP_ADDRESS>"
rx_ipv4.prefix = "24"
rx_ipv4.gateway = "<IP_ADDRESS>"

rx_tcp = rx_ipv4.tcp
rx_tcp.keep_alive_time = 700
rx_tcp.receive_buffer_size = 4096
rx_tcp.transmit_buffer_size = 4096
rx_tcp.fin_timeout = 60
rx_tcp.syn_retries = 5
rx_tcp.src_port.increment.start = 5000
rx_tcp.src_port.increment.step = 2
rx_tcp.src_port.increment.count = 10

#server configurations

tx_eth = tx_device.ethernet
tx_eth.name = "tx_eth"
tx_eth.mac = "00:00:00:00:70:01"
tx_eth.name = "eth_S1"
tx_eth.mtu = 1500

tx_ipv4 = tx_eth.ipv4
tx_ipv4.name = "tx_ipv4"
tx_ipv4.address = "<IP_ADDRESS>"
tx_ipv4.prefix = "24"
tx_ipv4.gateway = "<IP_ADDRESS>"

tx_tcp = tx_ipv4.tcp
tx_tcp.keep_alive_time = 700
tx_tcp.fin_timeout = 60
tx_tcp.syn_retries = 5
tx_tcp.dst_port.decrement.start = 6000
tx_tcp.dst_port.decrement.step = 2
tx_tcp.dst_port.decrement.count = 10

#Flow Configurations
flow = confObj.flows.flow()
flow.tx_rx.tx(name=rx_device.name)
flow.tx_rx.rx(name=tx_device.name)

#Objective and Traffic configurations
flow.test_type = "TcpThroupgut"
flow.test_value = 1000
flow.duration.ramp_up = 10
flow.duration.sustain_time = 180
flow.duration.ramp_down = 180
api.set_config(confObj)
api.control(state= start)

Sample script for UDP:-
import snappi
api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>5")
confObj = api.config()
port_1 = confObj.ports.port(name="p1", location='<IP_ADDRESS>/1/1')[-1]
port_2 = confObj.ports.port(name="p2", location='<IP_ADDRESS>/1/2')[-1]

rx_device, tx_device  = confObj.devices.device(name="client").device(name="server")

rx_device.port_name = port_1.name
tx_device.port_name = port_2.name
#Client Configuration
rx_eth = tx_device.ethernet
rx_eth.name = "rx_eth"
rx_eth.mac = "00:00:00:00:65:01"
rx_eth.name = "eth_r1"
rx_eth.mtu = 1500

rx_ipv4 = rx_eth.ipv4
rx_ipv4.name = "rx_ipv4"
rx_ipv4.address = "<IP_ADDRESS>"
rx_ipv4.prefix = "24"
rx_ipv4.gateway = "<IP_ADDRESS>"

udp_rx = rx_ipv4.udp
udp_rx.src_port.increment.start = 5000
udp_rx.src_port.increment.step = 2
udp_rx.src_port.increment.count = 10

#server configurations

tx_eth = tx_device.ethernet
tx_eth.name = "tx_eth"
tx_eth.mac = "00:00:00:00:70:01"
tx_eth.name = "eth_S1"
tx_eth.mtu = 1500

tx_ipv4 = tx_eth.ipv4
tx_ipv4.name = "tx_ipv4"
tx_ipv4.address = "<IP_ADDRESS>"
tx_ipv4.prefix = "24"
tx_ipv4.gateway = "<IP_ADDRESS>"

udp_tx = tx_ipv4.udp
udp_tx.dst_port.decrement.start = 6000
udp_tx.dst_port.decrement.step = 2
udp_tx.dst_port.decrement.count = 10

#Flow Configurations
flow = confObj.flows.flow()
flow.tx_rx.tx(name=rx_device.name)
flow.tx_rx.rx(name=tx_device.name)

#Objective and Traffic configurations
flow.test_type = "UdpThroupgut"
flow.test_value = 1000
flow.duration.ramp_up = 10
flow.duration.sustain_time = 180
flow.duration.ramp_down = 180
api.set_config(confObj)
api.control(state= start)

The app object should be used for test_type, test_value etc instead of the flow node.  The flow object should be reserved for L2L3 stateless traffic while the app object can be used for L4L7 stateful traffic properties.
# L4L7 stateful flow configurations
app = confObj.apps.add(name="Application Flow 1")
app.tx_device(name=rx_device.name)
app.rx_device(name=tx_device.name)

# objective and traffic configuration that are specific to L4L7
app.test_type = app.TEST_TYPE_UDP_THROUGHPUT
app.test_value = 1000
app.duration.ramp_up = 10
app.duration.sustain_time = 180
app.duration.ramp_down = 180

Updated script and design based on Andy, Razvan and Lyle.

import snappi
api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>5")
conf_obj = api.config()
port_1 = conf_obj.ports.port(name="p1", location='<IP_ADDRESS>/1/1')[-1]
port_2 = conf_obj.ports.port(name="p2", location='<IP_ADDRESS>/1/2')[-1]

#rx and tx varible naming is to differentiate between client and server configs.
rx_device, tx_device  = conf_obj.devices.device(name="client").device(name="server")

rx_device.port_name = port_1.name
tx_device.port_name = port_2.name
#Client Configuration
rx_eth = tx_device.ethernet
rx_eth.name = "rx_eth"
rx_eth.mac = "00:00:00:00:65:01"
rx_eth.name = "eth_r1"
rx_eth.mtu = 1500

rx_ipv4 = rx_eth.ipv4
rx_ipv4.name = "rx_ipv4"
rx_ipv4.address = "<IP_ADDRESS>"
rx_ipv4.prefix = "24"
rx_ipv4.gateway = "<IP_ADDRESS>"

rx_tcp = rx_ipv4.tcp
rx_tcp.keep_alive_time = 700
rx_tcp.receive_buffer_size = 4096
rx_tcp.transmit_buffer_size = 4096
rx_tcp.fin_timeout = 60
rx_tcp.syn_retries = 5
rx_tcp.source_port_range.min = 1024
rx_tcp.source_port_range.max = 65535

http_client = rx_tcp.http[-1]
http_client.role = "Client"
http_client.name = "HTTP Client1"
http_client.enable_ssl = "True"

#server configurations

tx_eth = tx_device.ethernet
tx_eth.name = "tx_eth"
tx_eth.mac = "00:00:00:00:70:01"
tx_eth.name = "eth_S1"
tx_eth.mtu = 1500

tx_ipv4 = tx_eth.ipv4
tx_ipv4.name = "tx_ipv4"
tx_ipv4.address = "<IP_ADDRESS>"
tx_ipv4.prefix = "24"
tx_ipv4.gateway = "<IP_ADDRESS>"

tx_tcp = tx_ipv4.tcp
tx_tcp.keep_alive_time = 700
tx_tcp.dest_port_range.min = 1024
tx_tcp.dest_port_range.max = 65535

http_server = tx_tcp.http()[-1]
http_server.role = "Server"
http_server.name = "HTTP Server"
http_server.enable_ssl = "True"
#changing from http_server.method.get() to http_server.content.add()
#adding support of file payload
http_server.content.add(name="/b.html", response=200, payload_type="file", file_name="C:\ProgramData\Ixia\IxLoadGateway\sample.pdf").add(name="/1024.html", response=200, payload_type="range", fixed_size="1048576")
#changing from http_server.method.get() to http_server.traffic.get()
http_client.traffic.get(name="/b.html").post(name="/1024.html")

app = conf_obj.apps.add(name="Application Flow 1")
app.tx_device(name=rx_device.name)
app.rx_device(name=tx_device.name)

# objective and traffic configuration that are specific to L4L7
#test_type-> test_objective_type
app.test_objective_type = app.concurrent_connections
app.test_objective_value  = 100
app.duration.ramp_up_time = 10  #in seconds
app.duration.sustain_time = 180 #in seconds
app.duration.ramp_down_time = 180 #in seconds
api.set_config(conf_obj)
api.control(state= start)

Sample script for TCP test with review comment changes

import snappi
api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>5")
confObj = api.config()
port_1 = confObj.ports.port(name="p1", location='<IP_ADDRESS>/1/1')[-1]
port_2 = confObj.ports.port(name="p2", location='<IP_ADDRESS>/1/2')[-1]

rx_device, tx_device  = confObj.devices.device(name="client").device(name="server")

rx_device.port_name = port_1.name
tx_device.port_name = port_2.name
#Client Configuration
rx_eth = tx_device.ethernet
rx_eth.name = "rx_eth"
rx_eth.mac = "00:00:00:00:65:01"
rx_eth.name = "eth_r1"
rx_eth.mtu = 1500

rx_ipv4 = rx_eth.ipv4
rx_ipv4.name = "rx_ipv4"
rx_ipv4.address = "<IP_ADDRESS>"
rx_ipv4.prefix = "24"
rx_ipv4.gateway = "<IP_ADDRESS>"

rx_tcp = rx_ipv4.tcp
rx_tcp.keep_alive_time = 700
rx_tcp.receive_buffer_size = 4096
rx_tcp.transmit_buffer_size = 4096
rx_tcp.fin_timeout = 60
rx_tcp.syn_retries = 5
rx_tcp.src_port.increment.start = 5000
rx_tcp.src_port.increment.step = 2
rx_tcp.src_port.increment.count = 10

#server configurations

tx_eth = tx_device.ethernet
tx_eth.name = "tx_eth"
tx_eth.mac = "00:00:00:00:70:01"
tx_eth.name = "eth_S1"
tx_eth.mtu = 1500

tx_ipv4 = tx_eth.ipv4
tx_ipv4.name = "tx_ipv4"
tx_ipv4.address = "<IP_ADDRESS>"
tx_ipv4.prefix = "24"
tx_ipv4.gateway = "<IP_ADDRESS>"

tx_tcp = tx_ipv4.tcp
tx_tcp.keep_alive_time = 700
tx_tcp.fin_timeout = 60
tx_tcp.syn_retries = 5
tx_tcp.dst_port.decrement.start = 6000
tx_tcp.dst_port.decrement.step = 2
tx_tcp.dst_port.decrement.count = 10

#Traffic  Configurations
app = conf_obj.apps.add(name="Application Flow 1")
app.tx_device(name=rx_device.name)
app.rx_device(name=tx_device.name)

app.test_objective_type = app.tcpthroughput
app.test_objective_value  = 1000
app.duration.ramp_up_time = 10  #in seconds
app.duration.sustain_time = 180 #in seconds
app.duration.ramp_down_time = 180 #in seconds
api.set_config(conf_obj)
api.control(state= start)

Sample script for 1 ARM test
import snappi
api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>5")
conf_obj = api.config()
port_1 = conf_obj.ports.port(name="p1", location='<IP_ADDRESS>/1/1')[-1]

#rx and tx varible naming is to differentiate between client and server configs.
rx_device = conf_obj.devices.device(name="client")

rx_device.port_name = port_1.name
tx_device.port_name = port_2.name
#Client Configuration
rx_eth = tx_device.ethernet
rx_eth.name = "rx_eth"
rx_eth.mac = "00:00:00:00:65:01"
rx_eth.name = "eth_r1"
rx_eth.mtu = 1500

rx_ipv4 = rx_eth.ipv4
rx_ipv4.name = "rx_ipv4"
rx_ipv4.address = "<IP_ADDRESS>"
rx_ipv4.prefix = "24"
rx_ipv4.gateway = "<IP_ADDRESS>"

rx_tcp = rx_ipv4.tcp
rx_tcp.keep_alive_time = 700
rx_tcp.receive_buffer_size = 4096
rx_tcp.transmit_buffer_size = 4096
rx_tcp.fin_timeout = 60
rx_tcp.syn_retries = 5
rx_tcp.source_port_range.min = 1024
rx_tcp.source_port_range.max = 65535

http_client = rx_tcp.http[-1]
http_client.role = "Client"
http_client.name = "HTTP Client1"
http_client.enable_ssl = "True"

#DUT Server config
server_ip = "<IP_ADDRESS>"

http_client.traffic.get(destination= server_ip, name="/b.html")

app = conf_obj.apps.add(name="Application Flow 1")
app.tx_device(name=rx_device.name)
app.rx_device(name=server_ip)

# objective and traffic configuration that are specific to L4L7
#test_type-> test_objective_type
app.test_objective_type = app.concurrent_connections
app.test_objective_value  = 100
app.duration.ramp_up_time = 10  #in seconds
app.duration.sustain_time = 180 #in seconds
app.duration.ramp_down_time = 180 #in seconds
api.set_config(conf_obj)
api.control(state= start)

The definition of the http object under tcp should not be a dual purpose object that is governed by the .role property.
There should be specific models for tcp.http_client and tcp.http_server that have explicit properties as opposed to runtime,  dynamic, dependent properties.

Specifying a filename and file location for dynamic content must be avoided and the content must be provided by a user as a byte stream.
If there is static content then it should be explicitly modeled.

@waseembaig thanks for the script.  Few questions/comments (looking at the most recent script) :

Typo?

rx_eth = tx_device.ethernet

Should this be rx_device.ethernet?

Inadvertently setting rx_eth.name and tx_eth.name  twice?

rx_eth.name = "rx_eth"
rx_eth.mac = "00:00:00:00:65:01"
rx_eth.name = "eth_r1"

tx_eth.name = "tx_eth"
tx_eth.mac = "00:00:00:00:70:01"
tx_eth.name = "eth_S1"

Which is the one script that incorporates all the traffic (UDP from IxNetwork, TCP/HTTP from IxLoad and additional background traffic from IxNetwork) for the DASH HERO test?  Do we have such a script?  If so, could you please point me to it?

Hi Ankur and Andy,
I have updated the script based on review comments.
Ankur,
Which is the one script that incorporates all the traffic (UDP from IxNetwork, TCP/HTTP from IxLoad and additional background traffic from IxNetwork) -- TCP/UDP of IXNetwork falls under flows where as TCP of IxLoad falls under deviceand  traffic of IxNetwork is configurable under flow and traffic of IxLoad is configurable under app,  following is the scripts.
import snappi
#IxLoad Snippet of TCP and HTTTP  configs under device object.
ixload_api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>5")
conf_obj = ixload_api.config()
port_1 = conf_obj.ports.port(name="p1", location='<IP_ADDRESS>/1/1')[-1]
port_2 = conf_obj.ports.port(name="p2", location='<IP_ADDRESS>/1/2')[-1]

#rx and tx varible naming is to differentiate between client and server configs.
rx_device, tx_device  = conf_obj.devices.device(name="client").device(name="server")

rx_device.port_name = port_1.name
tx_device.port_name = port_2.name
#Client Configuration
rx_eth = rx_device.ethernet
rx_eth.name = "rx_eth"
rx_eth.mac = "00:00:00:00:65:01"
rx_eth.mtu = 1500

rx_ipv4 = rx_eth.ipv4
rx_ipv4.name = "rx_ipv4"
rx_ipv4.address = "<IP_ADDRESS>"
rx_ipv4.prefix = "24"
rx_ipv4.gateway = "<IP_ADDRESS>"

rx_tcp = rx_ipv4.tcp
rx_tcp.keep_alive_time = 700
rx_tcp.receive_buffer_size = 4096
rx_tcp.transmit_buffer_size = 4096
rx_tcp.fin_timeout = 60
rx_tcp.syn_retries = 5
rx_tcp.source_port_range.min = 1024
rx_tcp.source_port_range.max = 65535
#Modified tcp.http[-1] to tcp.http_client
http_client = rx_tcp.http_client
http_client.role = "Client"
http_client.name = "HTTP Client1"
http_client.enable_ssl = "True"

#server configurations

tx_eth = tx_device.ethernet
tx_eth.name = "tx_eth"
tx_eth.mac = "00:00:00:00:70:01"
tx_eth.mtu = 1500

tx_ipv4 = tx_eth.ipv4
tx_ipv4.name = "tx_ipv4"
tx_ipv4.address = "<IP_ADDRESS>"
tx_ipv4.prefix = "24"
tx_ipv4.gateway = "<IP_ADDRESS>"

tx_tcp = tx_ipv4.tcp
tx_tcp.keep_alive_time = 700
tx_tcp.dest_port_range.min = 1024
tx_tcp.dest_port_range.max = 65535
#Modified tcp.http[-1] to tcp.http_server
http_server = tx_tcp.http_server
http_server.role = "Server"
http_server.name = "HTTP Server"
http_server.enable_ssl = "True"
#changing from http_server.method.get() to http_server.content.add()
#adding support of file payload
http_server.content.add(name="/b.html", response=200, payload_type="file", file_name=file_path).add(name="/1024.html", response=200, payload_type="range", fixed_size="1048576")
#changing from http_server.method.get() to http_server.traffic.get()
http_client.traffic.get(name="/b.html").post(name="/1024.html")
#IxLoad traffic configurations
app = conf_obj.apps.add(name="Application Flow 1")
app.tx_device(name=rx_device.name)
app.rx_device(name=tx_device.name)

# objective and traffic configuration that are specific to L4L7
#test_type-> test_objective_type
app.test_objective_type = app.concurrent_connections
app.test_objective_value  = 100
app.duration.ramp_up_time = 10  #in seconds
app.duration.sustain_time = 180 #in seconds
app.duration.ramp_down_time = 180 #in seconds
api.set_config(conf_obj)
api.control(state= start)

IxNetwork code snippet of TCP config under flow object.
ixnet_api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>5")
ixn_conf_obj = ixnet_api.config()
f = ixn_conf_obj.flows[0]
size = 1518
packets = 100
f.packet.ethernet().ipv4().tcp()
eth, ip, tcp = f.packet[0], f.packet[1], f.packet[2]
eth.src.value = "00:CD:DC:CD:DC:CD"
eth.dst.value = "00:AB:BC:AB:BC:AB"
ip.src.value = "<IP_ADDRESS>"
ip.dst.value = "<IP_ADDRESS>"
tcp.src_port.increment.start = 5000
tcp.src_port.increment.step = 2
tcp.src_port.increment.count = 10
tcp.dst_port.decrement.start = 6000
tcp.dst_port.decrement.step = 2
tcp.dst_port.decrement.count = 10

IxNetwork code snippet of UDP config under flow object.
ixn_conf_obj = ixnet_api.config()
f = ixn_conf_obj.flows[0]
packets = 1
size = 74
f.packet.ethernet().ipv4().udp()
eth, ip, udp = f.packet[0], f.packet[1], f.packet[2]
eth.src.value = "00:0c:29:1d:10:67"
eth.dst.value = "00:0c:29:1d:10:71"
ip.src.value = "<IP_ADDRESS>"
ip.dst.value = "<IP_ADDRESS>"
udp.src_port.value = 3000
udp.dst_port.value = 4000

Dash-Hero Script
import snappi

def configure_ixnetwork(api):
    conf_obj = api.config()
    port_1 = conf_obj.ports.port(name="p1", location='<IP_ADDRESS>/1/1')[-1]
    d1 = conf_obj.devices.device(name="d1")

    eth1 = d1.ethernets.add()
    eth1.name = "eth1"
    eth1.port_name = port_1.name
    eth1.mac = "00:ad:aa:13:11:01"

    ip1 = eth1.ipv4_addresses.add()
    ip1.name = "ipv41"
    ip1.address = "<IP_ADDRESS>"
    ip1.gateway = "<IP_ADDRESS>"

    rx1_bgpv4 = ip1.bgpv4
    rx1_bgpv4.name = "rx1_bgpv4"
    rx1_bgpv4.as_type = "ebgp"
    rx1_bgpv4.dut_address = "<IP_ADDRESS>"
    rx1_bgpv4.as_number = "65200"
    rx1_rr = rx1_bgpv4.bgpv4_routes.bgpv4route()[-1]
    rx1_rr.name = "rx1_rr"

    flow = conf_obj.flows.flow(name="flow1")[-1]
    flow.tx_rx.device.rx_names = [rx1_rr.name]

    eth1, ip1, udp1 = flow.packet.ethernet().ipv4().udp()

    # set source and destination MAC addresses
    eth1.src.value, eth1.dst.value = "00:AA:00:00:04:00", "00:AA:00:00:00:AA"

    # set source and destination IPv4 addresses
    ip1.src.value, ip1.dst.value = "<IP_ADDRESS>", "<IP_ADDRESS>"

    # set incrementing port numbers as source UDP ports
    udp1.src_port.increment.start = 5000
    udp1.src_port.increment.step = 2
    udp1.src_port.increment.count = 10
    # assign list of port numbers as destination UDP ports
    udp1.dst_port.values = [4000, 4044, 4060, 4074]

    print("Pushing traffic configuration ...")
    api.set_config(conf_obj)

def configure_ixload(api):
    conf_obj = api.config()
    port_1 = conf_obj.ports.port(name="p1", location='<IP_ADDRESS>/1/1')[-1]
    port_2 = conf_obj.ports.port(name="p2", location='<IP_ADDRESS>/1/2')[-1]

    #rx and tx varible naming is to differentiate between client and server configs.
    rx_device, tx_device  = conf_obj.devices.device(name="client").device(name="server")

    rx_device.port_name = port_1.name
    tx_device.port_name = port_2.name
    #Client Configuration
    rx_eth = rx_device.ethernet
    rx_eth.name = "rx_eth"
    rx_eth.mac = "00:00:00:00:65:01"
    rx_eth.mtu = 1500

    rx_ipv4 = rx_eth.ipv4
    rx_ipv4.name = "rx_ipv4"
    rx_ipv4.address = "<IP_ADDRESS>"
    rx_ipv4.prefix = "24"
    rx_ipv4.gateway = "<IP_ADDRESS>"

    rx_tcp = rx_ipv4.tcp
    rx_tcp.keep_alive_time = 700
    rx_tcp.receive_buffer_size = 4096
    rx_tcp.transmit_buffer_size = 4096
    rx_tcp.fin_timeout = 60
    rx_tcp.syn_retries = 5
    rx_tcp.source_port_range.min = 1024
    rx_tcp.source_port_range.max = 65535
    #Modified tcp.http[-1] to tcp.http_client
    http_client = rx_tcp.http_client
    http_client.role = "Client"
    http_client.name = "HTTP Client1"
    http_client.enable_ssl = "True"

    #server configurations

    tx_eth = tx_device.ethernet
    tx_eth.name = "tx_eth"
    tx_eth.mac = "00:00:00:00:70:01"
    tx_eth.mtu = "1500"

    tx_ipv4 = tx_eth.ipv4
    tx_ipv4.name = "tx_ipv4"
    tx_ipv4.address = "<IP_ADDRESS>"
    tx_ipv4.prefix = "24"
    tx_ipv4.gateway = "<IP_ADDRESS>"

    tx_tcp = tx_ipv4.tcp
    tx_tcp.keep_alive_time = 700
    tx_tcp.dest_port_range.min = 1024
    tx_tcp.dest_port_range.max = 65535
    #Modified tcp.http[-1] to tcp.http_server
    http_server = tx_tcp.http_server
    http_server.role = "Server"
    http_server.name = "HTTP_Server"
    http_server.enable_ssl = "True"
    #changing from http_server.method.get() to http_server.traffic.get()
    http_client.traffic.get(page_name="/b.html", destination=http_server.name)

    #IxLoad traffic configurations
    app = conf_obj.apps.add(name="Application Flow 1")
    app.tx_device(name=rx_device.name)
    app.rx_device(name=tx_device.name)

    # objective and traffic configuration that are specific to L4L7
    #test_type-> test_objective_type
    app.test_objective_type = app.concurrent_connections
    app.test_objective_value  = 1000
    app.duration.ramp_up_time = 10  #in seconds
    app.duration.sustain_time = 180 #in seconds
    app.duration.ramp_down_time = 10 #in seconds
    api.set_config(conf_obj)
    api.control(state= "start")

if  __name__=="__main__":
    ixnet_api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>9")
    ixload_api = snappi.api(host="<IP_ADDRESS>", version="<IP_ADDRESS>9", ext="ixload")
    configure_ixnetwork(ixnet_api)
    configure_ixload(ixload_api)

minor nits
 http_client.role = "Client" - is this needed as an http_client object us being created (same goes for "Server")?
http_client.enable_ssl = "True" - True should not be in quotes
app.tx_device(name=rx_device.name) - should this be more explicit like app.tx_name and should it take the http_client.name?

http_client.role = "Client" - is this needed as an http_client object us being created (same goes for "Server")? --> Updated the script
http_client.enable_ssl = "True" - True should not be in quotes --> Updated the script
app.tx_device(name=rx_device.name) - should this be more explicit like app.tx_name and should it take the http_client.name? --> In the earlier discussion, we decided to link  device to app instead of http  or tcp (for  tcp testcase).
Following are reference diagram.

@ajbalogh @ankur-sheth @waseembaig
initial thoughts:

does http or other things above L4 really belong in the device stack config?  we should probably work on an alternate view of including apps/protocols beyond the transport be properties of the app config node?

test_objective and test_type sounded very vendor/domain specific.  is there a better and more neutral way to describe it?

consider rename variables like ixnet, ixload to something more neutral.

@winstonliu1111

does http or other things above L4 really belong in the device stack config? we should probably work on an alternate view of including apps/protocols beyond the transport be properties of the app config node? -> We had discussions on that in early review discussions. Following are the old proposed  models.

New proposed models.

test_objective and test_type sounded very vendor/domain specific. is there a better and more neutral way to describe it? -> Will work on it

consider rename variables like ixnet, ixload to something more neutral. --> Updated the script with L_23_api and L_47_api.

@winstonliu1111

does http or other things above L4 really belong in the device stack config? we should probably work on an alternate view of including apps/protocols beyond the transport be properties of the app config node? -> We had discussions on that in early review discussions. Following are the old proposed  models.

New proposed models.

test_objective and test_type sounded very vendor/domain specific. is there a better and more neutral way to describe it? -> Will work on it
consider rename variables like ixnet, ixload to something more neutral. --> Updated the script with L_23_api and L_47_api.

have we thought about
Device -> Ethernet -> IP -> UDP/TCP
Apps -> HTTP/grpc/quic -> Traffic config

@winstonliu1111

does http or other things above L4 really belong in the device stack config? we should probably work on an alternate view of including apps/protocols beyond the transport be properties of the app config node? -> We had discussions on that in early review discussions. Following are the old proposed  models.

New proposed models.

test_objective and test_type sounded very vendor/domain specific. is there a better and more neutral way to describe it? -> Will work on it
consider rename variables like ixnet, ixload to something more neutral. --> Updated the script with L_23_api and L_47_api.

have we thought about
Device -> Ethernet -> IP -> UDP/TCP Apps -> HTTP/grpc/quic -> Traffic config

Yes Winston, we had discussions on those modelling as well, following are the proposed model.

@winstonliu1111

does http or other things above L4 really belong in the device stack config? we should probably work on an alternate view of including apps/protocols beyond the transport be properties of the app config node? -> We had discussions on that in early review discussions. Following are the old proposed  models.

New proposed models.

test_objective and test_type sounded very vendor/domain specific. is there a better and more neutral way to describe it? -> Will work on it
consider rename variables like ixnet, ixload to something more neutral. --> Updated the script with L_23_api and L_47_api.

have we thought about
Device -> Ethernet -> IP -> UDP/TCP Apps -> HTTP/grpc/quic -> Traffic config

Yes Winston, we had discussions on those modelling as well, following are the proposed model.

@ankur-sheth @ajbalogh @waseembaig @ashutshkumr
can we have the pros/cons?  aside from the DASH use case, can we think of the more complicated what if use cases to see how it will look overall?  think in terms of environments such as KNE or containerlab...
application/attack clients  -> [some network] -> border router of an AS (say an emulator) -> border router of another AS (say a router running a real NOS) -> [some network] -> middle box -> application server/services
should look at how L23/L47 concepts will be configured in the most natural/understandable way.

@winstonliu1111

does http or other things above L4 really belong in the device stack config? we should probably work on an alternate view of including apps/protocols beyond the transport be properties of the app config node? -> We had discussions on that in early review discussions. Following are the old proposed  models.

New proposed models.

test_objective and test_type sounded very vendor/domain specific. is there a better and more neutral way to describe it? -> Will work on it
consider rename variables like ixnet, ixload to something more neutral. --> Updated the script with L_23_api and L_47_api.

have we thought about
Device -> Ethernet -> IP -> UDP/TCP Apps -> HTTP/grpc/quic -> Traffic config

Yes Winston, we had discussions on those modelling as well, following are the proposed model.

@ankur-sheth @ajbalogh @waseembaig @ashutshkumr can we have the pros/cons? aside from the DASH use case, can we think of the more complicated what if use cases to see how it will look overall? think in terms of environments such as KNE or containerlab...
application/attack clients -> [some network] -> border router of an AS (say an emulator) -> border router of another AS (say a router running a real NOS) -> [some network] -> middle box -> application server/services
should look at how L23/L47 concepts will be configured in the most natural/understandable way.

@winstonliu1111 from the complex use case front, are you looking for below configs.
"application(Ixload) -> IxNetwork ->  border router of another AS (say a router running a real NOS) -> IxNetwork -> DUT -> application server/services (IxLoad)"
if Yes. We can achieve the above config with existing L23 and proposed L47 models.
Please let me know if you are looking for other configs.

Waseem, i'm looking for pro vs. con of the different approaches, and need to see UX of the above use case i mentioned by actually showing how it is configured in the different approaches.


Skip CI for RFCs

Skip CI tests for RFCs, as we already do for frontend code.
Edit and rename skipping script, update CircleCI config to use new name.

I think this error is real here

@bkase I've seen that test fail in other PRs, it seems to be a Dockerhub issue. Usually, rerunning the test succeeds (as it did here). It's unrelated to anything in this PR, certainly.


Timeouts and Aborted Connections after running code for long times

I am using Twitter API to scan through tweets and users, but because of Twitter's API limits, scanning thousands of users/tweets can take hours. Fine. However, when I run Twitter API code through Python using Tweepy, after a few hours of the code running the code gets interrupted and I get one of two error messages:
tweepy.error.TweepError: Failed to send request: ('Connection aborted.', error(54, 'Connection reset by peer'))
Another error:
tweepy.error.TweepError: Failed to send request: HTTPSConnectionPool(host='api.twitter.com', port=443): Read timed out. (read timeout=60)
It seems Twitter has a timeout limit that stops access after a certain point? Or not? Is this a Twitter API issue, or a Tweepy one?
This doesn't happen for small sizes, only when the code has to spend more than about an hour running. Of course one sees the problem with walking away from script that should take hours to run, only to come back and find that it's aborted partway through.
No one seems to know why this happens. Posted on several forums and spoken to several people..Could be a bug.

Any luck with this? I'm having the same problem

Also having this problem unfortunately

Bump, since I am too QQ

I'm getting this error also

Me too. Any help?

Besides these errors, I have also gotten this error code;
RATE LIMIT - waiting 15 minutes...
Traceback (most recent call last):
File "C:/Users/eabanoz/Desktop/Python_Scripts/Tweepy_Followers.py", line 73, in
print e.message[0]["code"]
TypeError: string indices must be integers, not str
TweepError Failed to send request: ('Connection aborted.', error(10054, 'An existing connection was forcibly closed by the remote host'))
Process finished with exit code 1

Just so everyone knows, the sixohsix twitter library works great. No issues with retry failing.

Also getting this error. Anyone figure out a fix?

I just got the same error two days ago. You can quickly reproduce the error by turning off your network connection. After briefly check the source codes of tweepy, I think the error is from request lib because the default timeout value in tweepy is set to be 60s. So just let your program sleep some time and make sure the network connection is Okay, and the codes should be able to be resume.

I agree with scubjwu. To me it happened when the connection went off. I have "fixed" by catching ReadTimeoutError and ConnectionError within the Timeout except.
except (Timeout, ssl.SSLError, ReadTimeoutError, ConnectionError) as exc:
Note that ReadTimeoutError belongs to urllib3 packages
from requests.exceptions import Timeout, ConnectionError
from requests.packages.urllib3.exceptions import ReadTimeoutError

SORRY. This was for a client and Tweepy apparently couldn't care less about responding to the issue, plus the project was compete, so I stopped following the thread.
Our workaround was to divide the tweets up into smaller batches. We ran the code several times at once using different keys and smaller batches, then join the files later.
For future projects I probably won't use Tweepy, due to lack of support. We were just in too deep to turn to another library this time.
Though, after reading the thread, it looks like it could be a connection issue. We were working on company server so maybe connection dropped every once in a while, but I seem to recall this happening on a home computer as well and someone with higher expertise saying this was not the case.
Solutions so far:
-@jcapde87 's Try/Except solution also seems like a good idea.
-Use another library (Twitter)
-Divide data/calls into smaller batches and combine results later.
:(

Hi @tednopple. I am just curious why using smaller batches would help ;-) Is it because smaller batches will shorter your time to process the data, so that the codes won't be timed out? Actually my codes just fetch the data from twitter and store it into database, so the data processing time is negligible. My feeling is the time out error may be raised by the rate limitation problem. You could always try to catch the exception and let the codes resume after sleeping for a while.

Yes, I think smaller batches just cut the processing time down. We were scanning followers. People with hundreds of thousands of followers caused the time out. So we divided the list to process less than 100,000 at a time
We wrote an exception to catch rate limits and wait the required 15 minutes. Despite this, the code would just abort all together with large data after hours of running.

Thanks @jcapde87. Big help.

Thanks, @jcapde87 and @ccphillippi.

I've got another error message to add:
File "/usr/local/lib/python2.7/dist-packages/tweepy/binder.py", line 149, in execute
    raise TweepError('Failed to send request: %s' % e)
tweepy.error.TweepError: Failed to send request: [Errno 110] Connection timed out

Another wish:
Can you split the TweepError into more detailed Exceptions? Like TweepyTimeout(TweepError), so it can be caught for a retry?

I was having this ReadTimeOutError issue. I resolved this issue by catching the TweepError. @allo- you can access the TweepError's attribute (reason, response, api_code) to more specifically handle TweepError. In the particular error of 'Failed to send request' (e.g., raised in line 189), TweepError object only has reason.
In the try except block, I added:
except TweepError as e:
if 'Failed to send request:' in e.reason:
print "Time out error caught."
time.sleep(180)
continue
I have a while loop that wraps around the try except block. So whenever this TweepError is raised, the code sleeps for 180 sec. and continue executing the while loop. You can test this by running the code without internet access. Within that 180 sec, re connect the internet, and it should work.

I was having this ReadTimeOutError issue. I resolved this issue by catching the TweepError. @allo- you can access the TweepError's attribute (reason, response, api_code) to more specifically handle TweepError. In the particular error of 'Failed to send request' (e.g., raised in line 189), TweepError object only has reason.
In the try except block, I added:
except TweepError as e:
if 'Failed to send request:' in e.reason:
print "Time out error caught."
time.sleep(180)
continue
I have a while loop that wraps around the try except block. So whenever this TweepError is raised, the code sleeps for 180 sec. and continue executing the while loop. You can test this by running the code without internet access. Within that 180 sec, re connect the internet, and it should work.

Interestingly, I've found that @jcapde87's solution isn't working in python3, anyone else have the same experience?

A quick query, can't we specify a longer timeout in api.timeout in Tweepy object rather than manually handling the error first and then using time.sleep? Since in 188th line in binder.py the following line is specified:
resp = self.session.request(self.method, full_url, data=self.post_data, timeout=self.api.timeout,auth=auth,
proxies=self.api.proxy)

I still observe this with python 2.7 and tweepy 3.5

What's the best way to deal with broken connections?
I'm seeing this error when using the stream API (stream.filter() ) with tweepy 3.5.0 and python 3.5.2 after the program has run for a while.

@neuhaus It seems it's a timeout issue, so I would suggest setting a timeout for now.

@ZeerakW
Yes I did a month ago and commented there that it's an improvement but not a complete fix.

Is there any other alternative to Tweepy?

@kjoth : try google.
asking it here is not going to help you or anyone else.

@kjoth I answered this in a comment in this thread: https://github.com/tweepy/tweepy/issues/617#issuecomment-133950914

@ThaWeatherman Thats great. Thanks dude. I will check it out.

A code example how to catch a timeout and re-establish connection etc would be great.

@jcapde Can you provide an example on how to catch a timeout and re-establish connection?

@greysonevins Apparently not, but you can follow the 'old' method (without using Cursor) in this link and it works perfectly.

In my case I didn't want to mess up the code inside the package and my main thread was not doing anything, so I did the following after reading thorough the lib code:
from ssl import SSLError
from requests.exceptions import Timeout, ConnectionError
from urllib3.exceptions import ReadTimeoutError

listener = TweetsStreamListener(...)
stream = tweepy.Stream(...)

# create a zombie
while stream.running is False:
    try:
        # start stream synchronously
        logging.info("Started listening to twitter stream...")
        stream.filter(follow=target_accounts, async=False)
    except (Timeout, SSLError, ReadTimeoutError, ConnectionError) as e:
        logging.warning("Network error occurred. Keep calm and carry on.", str(e))
    except Exception as e:
        logging.error("Unexpected error!", e)
    finally:
        logging.info("Stream has crashed. System will restart twitter stream!")
logging.critical("Somehow zombie has escaped...!")

Hope it helps.

It will be best if there's a PR for this one generalizing the caught exception at here. I've seen somebody already sent related PRs like https://github.com/tweepy/tweepy/pull/675 and https://github.com/tweepy/tweepy/pull/797. Maybe maintainers have different opinions or there's some tricky issues that only insiders really understand. Anyway, my example may be an easy workaround if it suits.

I incorporated a fix for the problem when using the cursor (tested when collecting followers from a user with too many followers to successfully collect before). It's in my Tweepy fork here:
https://github.com/lwahedi/tweepy.git@master

Nevermind, it's not working, I just got lucky with timeouts. Will post again when I do manage to fix it.

vtzs solution worked. in the try statement I added in time.sleep(5) so that it doesn't try 100's of times in a few minutes

I tried to initiate api = tweepy.API(timeout=600) #default=60 every time after time.sleep(15*60). It seems to work.

HEY @jcapde  Can you plss write the entire catch block.

I wonder why Tweepy has not addressed this issue yet. It persists even now. Will try solve timing out myself when I reach the boundary.

2021, I still have the same issue.

2021, I still have the same issue.

This seems to be for streaming (which should now be resolved with 68e19cc) while this issue seems to be about API methods.

@KhoaDuongUQ

This seems to be for streaming (which should now be resolved with 68e19cc) while this issue seems to be about API methods.

@KhoaDuongUQ

I am facing this error while collecting the followers' information. Any suggestion will be helpful.

@dheeman00 I'm currently facing the same issue while collecting follower and friend lists for my thesis. By any chance, were you able to come up with a solution yet?

@dheeman00 I'm currently facing the same issue while collecting follower and friend lists for my thesis. By any chance, were you able to come up with a solution yet?

Well one way to resolve the issue by running your extraction in a While True loop with a try and except block in it. In that way if you encounter any issues it will pause for a moment and continue the extraction again.

@dheeman00 Yeah, I was able to get it working by now. Thanks tho. Unfortunately, now I'm facing a new issue that I haven't seen posted anywhere else before.
When I get this list of a person's follower_ids and then use the look_up users functions for 100 users at a time, both lists don't have the same amount of entries.
I figured out why by looking up every user_id individually. Apparently one out of 11k user ids seem to give an error. I did get the tweep error that the id does not match to a user. However, the issue is that when doing a user lookup for multiple ids , it does not return an error. In that case it only returns 99 lookup requests, resulting in a different length of lists.
Any idea why that is the case? Did you perhaps face the same issue at some point?

@dheeman00 Yeah, I was able to get it working by now. Thanks tho. Unfortunately, now I'm facing a new issue that I haven't seen posted anywhere else before.
When I get this list of a person's follower_ids and then use the look_up users functions for 100 users at a time, both lists don't have the same amount of entries.
I figured out why by looking up every user_id individually. Apparently one out of 11k user ids seem to give an error. I did get the tweep error that the id does not match to a user. However, the issue is that when doing a user lookup for multiple ids , it does not return an error. In that case it only returns 99 lookup requests, resulting in a different length of lists.
Any idea why that is the case? Did you perhaps face the same issue at some point?

Could you post your code where you are facing this issue. I can give you a better understanding when I can see this issue.
But running followers is extraction and look up is will work out one after the another. As you can extract max of 15k id per 15 mins and in the case of look up it is just 100.

Closing this in favor of #1606


Added AES256 in CBC mode to string

To test the code
    var test = String("Test");
    var key = String("12345678901234567890123456789012");

    var encrypted = test.encrypt(key);

    var cnt = 0;
    var buf = ""

    for i in encrypted:
        cnt = cnt + 1;

        buf = buf +  str(i) + " ";

        if(cnt == 16):
            cnt =1;
            buf = buf + "\n"

    print (buf);

    print ("**DECRYPTED**");

    var decrypted = test.decrypt(encrypted,key);

    print ("**END DECRYPTED**");

    for i in decrypted:
        print (i);

    print ("**DECRYPTED PRINTED**");

This seems to fix #1969.

Could you squash the commits together? (at least the last two ones should be squashed in the first one before this can be merged).

Question:
Should I turn the decrypted back into a string?
Also should I work on HMAC?

@Catprog Yeah, if you input a string for encryption, you would usually expect the output to be a string as well. And, in the same way, inputting a RawArray/ByteArray for encryption should also result in a RawArray decrypted.

@bojidar-bg
The problem with turning the encrypted raw array into a string is that it not really a printable string anymore. It is very easy to get into >127 or <32.
The biggest use I see for the methods are network communications.  I do not think using unicode to make a string would be the best idea.
I was actually talking about this method
    var decrypted = test.decrypt(encrypted,key);

not the encrypt method.

@bojidar-bg is also talking about the decryption method. He means that if you input a String as a user, encrypt it, decrypt it again, you'd likely expect to get your input back, i.e. a String.

@akien-mga
right.
I think I need to use something like
		return utf8(data,data.size() - paddingNumber);

But that takes a const char *.
How do I convert that from a Vector<uint8_t>

Thinking about it. String is not the best place to put this. It needs to be in it's own class.

Why that? It's about String encryption no?

@akien-mga
I am thinking about a raw array.

https://github.com/Catprog/godot/commit/9a5adecf917e0f501de9c6ce5f4e41f65eb8d5c5 is the new attempt at it.

@Catprog any progress with this? Why not add it gradually - first just add ability to encrypt a string, then do another commit for arrays

Is this pretty much dead in the water or what :D

The problem was the interface was not right(stopping it from being merged).
I am waiting for an interface then I will build from that.
The difference between a raw array and a string is not a lot. (It makes no
sense to not do the two together)
On 4 October 2017 at 23:00, Todor Imreorov<EMAIL_ADDRESS>wrote:

Is this pretty much dead in the water or what :D
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/pull/5732#issuecomment-334147109,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFV4PlAvIWJyckJ-9KbHdm61MYFoLheCks5so4F2gaJpZM4JN5I5
.

--
Please visit my interactive story site
http://catprog.org/story.html
anti-EULA:
READ CAREFULLY. By reading the electronic correspondence attached
above the anti-EULA, you agree, on behalf of your employer, to
release me from all obligations and waivers arising from any and
all NON-NEGOTIATED agreements, licenses, terms-of-service,
shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure,
non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I
have entered into with your employer, its partners, licensors,
agents and assigns, in perpetuity, without prejudice to my ongoing
rights and privileges. You further represent that you have the
authority to release me from any BOGUS AGREEMENTS on behalf of your
employer.


Unable to receiving incoming calls when app is closed

Hi everything is working fine & supab. new version is awsome. but one issue is there i am not able to receive call when my app is closed. it's working fine when app is open or in forground state. but when app is killed the CallActivity is not invoking..

Anybody here can give me any issue fix?

In current Q-mun version 2.7, background calls work.


feat: support vllm in controller

Reason for Change:

support vllm runtime deployments
add a feature gate to enable vllm runtime
able to select runtime by annotation tag

Requirements

[x] added unit tests and e2e tests (if applicable).

ready @Fei-Guo


Uncaught OSError in pathlib.py line 1044

Aider version: 0.54.7
An uncaught exception occurred:
Traceback (most recent call last):
  File "aider", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "main.py", line 355, in main
    io = InputOutput(
         ^^^^^^^^^^^^
  File "io.py", line 214, in __init__
    self.append_chat_history(f"\n# aider chat started at {current_time}\n\n")
  File "io.py", line 548, in append_chat_history
    with self.chat_history_file.open("a", encoding=self.encoding) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pathlib.py", line 1044, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 30] Read-only file system: '.aider.chat.history.md'

Thanks for trying aider and filing this issue.
A fix is available in the main branch. You can get it by installing the latest version from github:
aider --install-main-branch

# or...

python -m pip install --upgrade git+https://github.com/paul-gauthier/aider.git

If you have a chance to try it, let me know if it works better for you.


MS SQL Server Sink Kamelet

Fixes #364

I'll rework the other connector by adding expecting type, query example and json mapping.

Let me merge and open other PRs.


Hotfix for Kotlin syntax highlighting

It looks like there was a bug in the grammar that causes major issues. See https://github.com/nishtahir/language-kotlin/issues/5. I've put up a patch to fix the problem and tested it on Lightshow and it seems to be working.
Preliminary Steps
Please confirm you have...

[x] reviewed How Linguist Works,
[c] reviewed the Troubleshooting docs,
[x] considered implementing an override,
[x] verified an issue has not already been logged for your issue (linguist issues).

Problem Description
It looks like the cause of the problem is a difference in how VSCode and Github parse and render the grammar since it works with VSCode. Unfortunately I don't know if there are any other tools to aid in debugging the exact cause of the issue on GitHub besides using Lightshow 😕.
URL of the affected repository:
Specific repositories affected were listed https://github.com/nishtahir/language-kotlin/issues/5. I can confirm that more have been affected
Last modified on:
2018-11-14
Detected language:
Kotlin

Thanks. Your changes will be pulled in automatically the next time I perform a Linguist release. I've just made a release today and am out of the office until the end of the week so won't be able to make another release until early next week at the earliest.
There isn't really an easy way to check how things will behave on GitHub short of ensuring the compiler we use in Linguist doesn't report any problems and checking on Lightshow.
Another tip is your grammar is a TextMate-compatible grammar: it's probably worth checking how it behaves with TextMate just in case VSCode is doing something clever to catch inconsistencies which GitHub isn't.
@Alhadis has a fair amount of experience with writing grammars so he may be able to also offer some assistance in determining why the current grammar version we're shipping isn't playing nicely.

I aim for a release approximately once a month, with exceptions. I won't have the time for another release until late next week at the earliest, though it could possibly the week after.

@lildude How is that acceptable? Isn't this impacting thousands of developers?

This may indeed be affecting a lot of developers, however this is no different from any other aesthetic bug that occurs in a third-party grammar used by GitHub (and GitLab) or any other third-party library used by any other piece of software for that matter. There is only one of me and maintaining Linguist isn't my primary role at GitHub. It is also impractical for me (or anyone else) to drop everything they're doing to make a new release every time we're notified of fix for a bug in a third party grammar as the time it takes to build, test and deploy and actually release a new version of Linguist is quite significant so I need to actively schedule time to do it.  This is why we don't make a release more frequently.
The opportunity I thought I was going to have this week was taken up by higher-priority tasks. If I get the opportunity to schedule time sooner than anticipated, I will.

Thank you for providing this context.
Is there any way to involve another developer or team in this process? Any opportunity to prioritize this issue would be greatly appreciated.
Even though it's aesthetic, we feel that this issue is starting to take a toll on our development velocity at Shopify. We don't use a QA team in our process and so the code review process is a significant and irreplaceable part of how we work. The lack of syntax highlighting is impacting our ability to effectively review code.

Is there any way to involve another developer or team in this process? Any opportunity to prioritize this issue would be greatly appreciated.

Possibly, and I might get a gap tomorrow or Friday thanks to most of my colleagues being off for thanks giving (I'm not in the 🇺🇸 so I won't be taking the time off) so won't be needing much of my attention, however that comes with the caveat that most of the peeps needed to approve deploys to GitHub.com will also be off. 😞
I'll certainly try to squeeze it in tomorrow and/or Friday.

Thank you very much @lildude that'd be greatly appreciated

🎉

This release is now live on GitHub.com and the sample in the OP of #4336 is now rendering correctly.


Improve handling of hostcall errors

In a lot of places, we currently turn a hostcall error code into a JS exception containing the error code and the C++ source location of the hostcall. That's very rarely helpful in debugging JS code, and we should improve on it :)

@tschneidereit I just came across the mapping of hostcall error codes within the fastly-shared crate and thought I'd have a go at contributing to this project -- I've started a draft pull-request with the error messages from the crate


Handling of non-ASCII characters (mostly in references)

Looking for any opinions on how non-ASCII characters can/should be handled.
Generally, the actual data json files should all be ASCII only. The citations, however, are a different matter, particularly with author names
I would think the non-ascii characters should be included in some way in the citation data, but never in the reference keys.

Python 3 handles everything quite well. Python 2 may require some prodding
Encoding them with latex is probably not a great idea - we want to support citation formats other than bibtex

Should be fixed in 6801507fa1a1a5c81b9c4ec9737a740c0e170f56 and 0b146857d4729c056ef070dcd4556169ef3e8605. References and basis set json files can now contain unicode, and should be in UTF8 encoding.