Add access to system role.

This pull request adds access to the system role for better description of how a model is supposed to respond.
In support of #68
This is probably just a first stage implementation because it doesn't allow for the creation, storage, and recall of custom roles as exampled in #68. Instead it simply adds a --system argument that can be used to append a message with the role system to the query delivered to the API.
If no --system argument is provided it delivers a default system message of:
DEFAULT_SYSTEM_MESSAGE = "You are shell-gpt, a helpful programming and system administration assistant."

This is editable in the config file.
The --code and --shell arguments have also been converted over to deliver their instructions as the system role instead of a prompt prefix. As mentioned in a TODO on line 77 of client.py
        # TODO: Move prompt context to system role when GPT-4 will be available over API.

Known Problems:
This solution interacts with chat history strangely, because a chat history can contain previous system role messages, and a new query on an old chat will just append the new system request to the chat. More testing is needed to see how this will interact, but preliminary gpt-3.5 seems to be fine with receiving multiple system messages and just obeys the most recent one.
Interactions with other active pull requests
GPT-4 appears to follow system role commands better. My pull request #85 interacts well with this upgrade. Clone My Fork:main if you want to use both at the same time.

I want to add --save-role <role_name>, --recall-role <role_name>, --list-roles, and --show-role operators, but I think we should have an issue chain to see if that is how people want it implemented and might be best in a different pull request.
To be used like
How to save a role as a custom role:
$ sgpt --role "You are a terminal emulator. Respond as if prompts received are directly executed on a bash shell" --save-role virtual_shell

Custom role 'virtual_shell' saved:
"You are a terminal emulator. Respond as if prompts received are directly executed on a bash shell"

How to use a custom role:
$ sgpt --recall-role virtual_shell "ls -la"

file1.txt
Documents
...

How to view available roles:
$ sgpt --list-roles

Base roles:
'default'
'shell'
'code'

Custom roles:
'virtual_shell'

How to see what a role is:
$ sgpt --show-role virtual_shell

Custom role 'virtual_shell':
"You are a terminal emulator. Respond as if prompts received are directly executed on a bash shell"

How to update a role:
$ sgpt --role "You are a terminal emulator. Respond as if prompts received are directly executed on a PowerShell" --save-role virtual_shell

Custom role 'virtual_shell' saved:
"You are a terminal emulator. Respond as if prompts received are directly executed on a PowerShell"

I was testing these changes today, and seems current GPT-3.5 model doesn't pay much attention to system role, as it is described in OpenAI documentation:

gpt-3.5-turbo-0301 does not always pay strong attention to system messages. Future models will be trained to pay stronger attention to system messages.

I was asking for generating few shell commands, and it doesn't perform well while generating clean shell commands. For example what is in current folder will return a command along with description which we don't want.
Last couple days I spent a lot of time to find best performing prompt to get a clean shell command, even for complicated queries. With few tricks and without defining huge list of rules I managed to make a prompt which most of the time getting clean shell comamnd or code (currently on main). Could you test these new changes with system role? If it will perform well we can merge this feature.

Don't review yet. Just testing the force pushing.
I just re-engineered how roles work in support of #110 and my desires to have custom roles be savable. Currently working on testing, debugging, documentation, and refinement.

I am planning to merge my PR https://github.com/TheR1D/shell_gpt/pull/130 soon. You will need to rebase it once more. However, this should not be difficult since I notice that you are not modifying core modules.

Could you please rebase your branch with main. I will try to find a solution for your issue with ChatSession (aka ChatCache) later today.

Working on it.

It is very annoying to get black and pylint to agree.

It is currently failing integrational_tests.test_code because it is adding markdown syntax to the code response, and integrational_tests.test_validation_code_shell because this test isn't relevant as with the --role system you can't put two arguments in at the same time.

Sry, I did a force push since I was chaging a lot of logic regarding to roles. We can't affect any core features of sgpt like --code and --shell, they should stay, we can't just remove them. I run more tests and seems current GPT-3.5 model doesn't really care about system role. In more cases it is just acting like there was not any system messages (especially with low temperatures like 0.1). I think we should wait with this feature until GPT-4 will be available for every one. I have pushed one commit where outlined some crucial logic, and made it easy to implement.

Np, I just have to figure out how to pull that into my version.
I know GPT-3.5 doesn't pay attention to the system role. That was why I implemented the PROMPT_STRUCTURE key. For shell and code it delivers completions to the API the same as what is currently in main (with just an added system prompt for gpt-3.5 to ignore).
I don't think the role system should be just the system role, because when I was playing around (even with gpt-4) on #108 the system role wasn't enough to explain to the model what it was and what it could do. It took some extra prompt structuring and conversation lead in before it understood that it could ask to execute commands and see the result.
I also think the role system should be more complex to avoid community separation from this project getting forked for specialized purposes #110 . It would be better if Shell GPT was an extensible foundation with a user interface where people wanted it (in their terminal). To support this, the role system will need the complexity where shell and code can be easily included as other types of roles.
To avoid repeated edge case detection in ChatHandler and DefaultHandler shell and code should be removed as arguments. Then they can be specified as just another type of role, but if you want the --shell -s and --code arguments to still exist they could be handled in app.py with something like:
    # Check if --shell or --code is called and set role accordingly while only allowing
    # one of the three to be called
    # If role and shell or role and code are called, error
    if role and (shell or code):
        raise typer.BadParameter("Cannot use --role and --shell or --code together.")
    # If neither is called, then the role is set to the default role
    if shell and code:
        raise typer.BadParameter("Cannot use --shell and --code together.")

    if shell:
        role = "shell"

    if code:
        role = "code"

I think it would be weird to have different layers of roles where some like shell or code are defined in python while other custom ones are in files. I think it would be better to make a fully extensible system.
Finally, the reason I want to implement this role system is because I want to get back to building a semi autonomous system administration and coding assistant, and Shell GPT could become that. I want to have a way to 'pre-train' a model in conversation about things like:
Introducing Autonomous Task Orchestration Manager (ATOM) Framework
Heuristic Imperatives
Rolling Episodic Memory Organizer (REMO) for autonomous AI systems
And also teach it what interfaces it has access to with the system to understand its environment.
To do this the role system needs to be more complicated than just exposing the system role. It needs to be a customizable conversation initiation system and prompt structuring database. At that point why not just role shell and code inside?

I figured I'd chime in as I've thought some more about my use case/ similar interest in an extensible ShellGPT.

I think that the default --code and --shell roles are the right ones to introduce users to sgpt functionality, and especially the -se interface is something I'd like to keep using and appreciate it's robustness for CLI command generation.
My test cases so far have been adding more to the "context window" ahead of what the user types as their prompt. E.g., after the default shell instructions there is some info about CLI tools 1,2,..,N and their syntax. This has enabled me to use natural language to generate mostly accurate commands in the tools of choice, which are too niche for GPT3.5 and even GPT4 to reliably get correct. I agree that we can't just use the system role, at least for GPT3.5, the user role context needs to be added as well. (Maybe this is the main drift from the original intent of this PR?)
As far as I've gotten with the project, I feel like I will probably want to do similar with the --code role. I.e., the desired output of commented python code with no other returned context/ commentary is the same. However it would also reference specific documentation that I've provided in the context window.
I also think it's interesting to modify the DEFAULT_SYSTEM_MESSAGE above. So, the CLI commands and flags to sgpt would be exactly the same, but somewhere I can configure custom context/ roles for each of: "chat," "code," "shell" so that my desired info is included.

So most of what @lpurdy01 describes here would be immediately useful to me, but I also agree on keeping the current CLI structure as it is both a sensible jumping off point and at least in my case the desired long-term usage.
In fact, I could probably get by with just these 3 custom role types if context windows were unlimited. There is a chance I'd create multiple roles for sub-domains (one is an image retrieval expert, one plans satellite trajectories, etc.) however it looks like the slick way to do it would be embeddings & a layer that adds only the needed context to the window during prompt generation. If this first step has basic ability for saving/ retrieving custom roles and pointing them to one of the three sgpt archetypes I would be very happy! :)

@lalligagger Thank you much for jumping in!
I like the auto context window idea. I have been wanting to do something like that including system context (like the result of ls or cat commands), but I haven't hypothesized a way to implement it yet.
In the last force push some context got wiped out.
Have a look at: https://github.com/TheR1D/shell_gpt/blob/892380185f33ea2178e54d4c962153af95301ead/README.md#the-role-system
And: https://github.com/TheR1D/shell_gpt/blob/892380185f33ea2178e54d4c962153af95301ead/README.md#custom-system-roles

This is what my shell.json looks like:
{
    "EXECUTABLE_RETURNS": true,
    "SYSTEM_MESSAGE": "\nYou are Command Line App ShellGPT, a programming and system administration assistant.\nYou are managing {os} operating system with {shell} shell.\nProvide only {shell} commands for {os} without any description.\nIf there is a lack of details, provide most logical solution.\nEnsure the output is a valid shell command.\nIf multiple steps required try to combine them together.\n",
    "PROMPT_STRUCTURE": "###\nProvide only {shell} commands for {os} without any description.\nIf there is a lack of details, provide most logical solution.\nEnsure the output is a valid shell command.\nIf multiple steps required try to combine them together.\nPrompt: {prompt}\n###\nCommand:"
}

@TheR1D I can work on rebasing this capability using the outline you provided today, but I don't want to have both of us working on a duplicate implementation. Where are you at with this?

@TheR1D I can work on rebasing this capability using the outline you provided today, but I don't want to have both of us working on a duplicate implementation. Where are you at with this?

Well I already have working code https://github.com/TheR1D/shell_gpt/pull/183, but big part of this feature is acutally to test it properly.

https://github.com/TheR1D/shell_gpt/pull/183


[internals] improve options module

Hello,
Since using default export of Options as option initializer, I have a question that is it a good way to use a Class-like naming style to name a function?
I found members in options.js like Modes and defaultOptions are also put in Options now, maybe leave Options as a pure Object, and rename the init function to Options.from(_options) is a better choice?
https://github.com/nuxt/nuxt.js/blob/d7d85d3b4763eacf4a25155c08a01fab3ee5746a/lib/core/nuxt.js#L18

Ping @pi0

@shirohana Yeap you are right. Indeed it would be even better-converting Options to a class like other nuxt internals. Anyway, that's an internal enhancement. PRs welcome :)

I think this should be closed :)


Exchange Hybrid mode - Remote\Office 365 mailbox: Set-Mailbox cmdlet throws no error when updating only the primary SMTP address

Create Remote\Office 365 mailbox.
Add second smtp address.
Wait for synchronization to complete.
Connect to Exchange Online via PowerShell.
Use cmdlet Set-Mailbox to set the second smtp address as Primary\Reply address.
Result: No error. The second smtp address is set as Primary\Reply address.
Run synchronization and wait till it completes.
Result: The changes made on step 5 will be reverted and the Primary\Reply address specified in the on-premises AD will be set.

Note: I understand that I need to update the Remote\Office 365 mailbox in on-premises AD or using on-premises Exchange. And I have scripts which execute updates in on-premises Exchange when update in Online Exchange fails with a specific error. But when updating only the primary SMTP address no errors occur. It is by design or a BUG?
Note: You can also change the Primary\Reply address to another one in the Online Exchange admin center and the ADSync problem described above will reoccur.

Facing the very same issue. Could anyone provide any insight on this point? Seems to be quite a major bug.

@yogkumgit Any updates? The same issue happens when manually updating proxy addresses of AADConnect-synced users in hybrid environments.

Add a new smtp address to a user mailbox in Exchange Online admin center.
Run AAD Ccnnect delta sync cycle.
The proxyAddresses attribute reverts to old value from on-premises. New smtp address is GONE!

Exchange should forbid changing proxyAddresses of synced users online. Maybe this writeback mechanism is supposed to handle the case, but it doesn't work https://docs.microsoft.com/en-us/azure/active-directory/hybrid/reference-connect-sync-attributes-synchronized#exchange-hybrid-writeback.

@TalismanM and @mr-spacecat - thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Here we are limited to documentation issues and improvements.
Please follow this link to contact support for business products:  https://docs.microsoft.com/office365/admin/contact-support-for-business-products.
Please keep us posted here on the resolution to feed whatever information you discover into the content.

@dariomws  Thank you very much for the contribution and sharing this explanation. @TalismanM Hope this comment is helpful for you. Considering the nature of the issue, as advised, please open a service ticket in your tenant and follow with them for the resolution. We proceed here to close it.
We will be very happy if you can share the outcome or resolution with us if you see documentation update is required. We will certainly update this documentation with that valuable information. Thanks again, for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


fix: url sitemap for post-processor

Please ensure your pull request adheres to the following guidelines:

[ ] make sure to link the related issues in this description
[ ] when merging / squashing, make sure the fixed issue references are visible in the commits, for easy compilation of release notes

Related Issues
jira.corp.adobe.com/browse/SITES-18817
Fix the baseURL for post-processor audit

:tada: This PR is included in version 1.10.11 :tada:
The release is available on:

v1.10.11
GitHub release

Your semantic-release bot :package::rocket:


Import data sources of an imported module

I noticed that since the recent changes to the data source importing, the tests/test_paths.py in lambdapdk fails.
I think this might be the reason: We're trying to get the 'package' information from the subgroup ('pdk', 'fpga', ...) instead of at the top level of the schema.

@ubfx it appears there were no CI testing for this so I added a test in #2225


Uncaught SyntaxError: Unexpected end of JSON input

Happens when I open a Julia project in Atom.
Atom: 1.15.0 x64
Electron: 1.3.13
OS: Mac OS X 10.12.3
Thrown From: linter-julia package 0.7.1
Stack Trace
Uncaught SyntaxError: Unexpected end of JSON input
At file:///Applications/Atom.app/Contents/Resources/app.asar/static/index.html:1

SyntaxError: Unexpected end of JSON input
    at Object.parse (native)
    at /packages/linter-julia/lib/linter-julia.coffee:93:26)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at Pipe._handle.close [as _onclose] (net.js:493:12)

Commands

Non-Core Packages
2-dark-syntax 1.1.0
actual-solarized-syntax 0.1.0
atom-dark-fusion-syntax 2.0.3
atom-material-syntax 1.0.2
atom-material-ui 1.3.9
atom-solarized-dark-ui 0.28.0
autocomplete-bibtex 0.7.0
autoupdate-packages 1.3.1
color-picker 2.2.5
compare-files 0.8.0
export-html 3.1.0
focus-dark 1.1.0
idle-theme 0.1.0
idledark 0.3.1
idlefingers 0.2.0
indent-detective 0.1.0
ink 0.6.5
julia-client 0.5.12
language-julia 0.10.0
language-matlab-only 0.2.0
language-r 0.4.1
latex-completions 0.3.2
linter 1.11.23
linter-julia 0.7.1
one-dark-shade-ui 0.1.0
polaris-syntax 0.1.0
python-jedi 0.3.8
python-tools 0.6.9
razor-atom-syntax-theme 2.2.3
tomorrow-night-bright-razor-atom-syntax-theme 1.0.0
solarized-dark-ui 0.3.2
solarized-light-syntax-modified 0.6.2
solarized-light-ui 0.1.0
solarized-seti-ui 0.8.6
solo-one-dark-ui 0.1.1
todo-show 1.11.0
tomorrow-night-bright-razor-atom-syntax-theme 1.0.0
tomorrow-night-bright-syntax 0.1.0
tool-bar 1.1.0
twilight 0.3.0
twilight-syntax 0.7.0
twilight-textmate-syntax 0.1.19
uber-juno 0.1.1
wordcount 2.10.4

Have you run Pkg.checkout("Lint")?  Also restarting Atom (Ctrl + Shift + F5) may help. Also Ctrl + Shift + I, opens developer tools and from there the output is interesting: namely is the lintserver running.

I have the same problem. I ran Pkg.checkout("Lint") but the result is the same. The dev tools say
linter-julia.coffee:72 connection ended with error ArgumentError("stream is closed or unusable")

I have the same error. I guess it's because the recent update of the Lint.jl package?

I had the same error.
[Enter steps to reproduce:]

...
...

Atom: 1.15.0 x64
Electron: 1.3.13
OS: Mac OS X 10.11.6
Thrown From: linter-julia package 0.7.1
Stack Trace
Uncaught SyntaxError: Unexpected end of JSON input
At file:///Applications/Atom.app/Contents/Resources/app.asar/static/index.html:1

SyntaxError: Unexpected end of JSON input
    at Object.parse (native)
    at /packages/linter-julia/lib/linter-julia.coffee:93:26)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at Pipe._handle.close [as _onclose] (net.js:493:12)

Commands
     -1:46.6.0 core:backspace (input.hidden-input)
  5x -1:45.4.0 core:save (input.hidden-input)
     -1:35.2.0 core:backspace (input.hidden-input)
     -1:34.8.0 core:save (input.hidden-input)

Non-Core Packages
atom-beautify 0.29.17
atom-latex 0.7.1
autocomplete-python 1.8.60
busy-signal 1.3.0
docblockr 0.9.1
Hydrogen 1.6.0
intentions 1.1.2
language-julia 0.10.0
language-latex 1.0.0
last-cursor-position 0.9.0
latex-completions 0.3.2
linter 2.1.0
linter-julia 0.7.1
linter-pylint 1.2.2
linter-ui-default 1.1.0
minimap 4.26.8
project-manager 3.3.3
Remote-FTP 0.10.7
sync-settings 0.8.1
termination 0.5.0

The number of errors thrown by this package went through the roof. I will reinstall in a few months. Keep up the good work, it looks like it will become extremely useful with time!

I will look into the issue today.

It looks like it may actually the linter-ui-default package that is responsible for much of the trouble.

This is a real problem. Here is the current work around:
cd ~/.julia/v0.5/Lint
git checkout 1dfcfd4
cd ~/.atom/packages/linter-julia
git checkout v0.5.5

Above is the solution to go back before the JSON change is implemented.

Does that also work if you've installed linter-julia inside Atom (rather than cloning the repo and apm)? I get
git checkout v0.5.5
fatal: Not a git repository (or any of the parent directories): .git

Here is the guideline for Atom packages: https://discuss.atom.io/t/how-do-you-download-a-specific-version-of-a-package/11124/4
apm install <package_name>@<package_version>

Thus it should be:
apm install<EMAIL_ADDRESS>

zn11265:linter-julia michael$ apm install<EMAIL_ADDRESS>Installing<EMAIL_ADDRESS>to /Users/michael/.atom/packages ✗
Package version: v0.5.5 not found

@mkborregaard can you try one more:
apm install<EMAIL_ADDRESS>

Now I get :-)
Installing<EMAIL_ADDRESS>to /Users/michael/.atom/packages ✓
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
zn11265:linter-julia michael$

I have now tested with both linux and windows computers. linter-julia works after Pkg.free("Lint") and update to latest linter-julia version 0.7.1. On linux my mistake was not to run the Pkg.free("Lint") properly, which meant that the requirements was not updated, my problem was caused an old version of Compat.jl. Windows I had to set the path to julia.exe, because the juno default is still julia version 0.4.
After updates on both platforms I had to restart Atom at least one time by pressing Ctrl + Shift + F5, to get the linter-julia working (see #40)
I will develop some tests (specs) for linter-julia and try to improve the error messages as well as the logic of auto-installer or possible auto-updater. Although I am little worried that too much automation without user action is unwanted feature some developers, who want to control the updates manually.
If the problem persists, please post Pkg.status() output.

@TeroFrondelius Thank you for the information. I just tried it on my Ubuntu machine, which turned out to work now! I will later try it on macOS and update the info.

@xukai92 thanks a lot for testing.

I can confirm that this seems to fix the problem on MacOS, thanks!

Mine are consistent, but seem to be from linter-ui-default so I commented on an issue there.

I can probably close this now? Or leave open so people can find the fix?

@dpsanders I still experienced this problem on my macOS. Can you plz tell me your linter-julia version and Lint.jl version please?

Correction, I still have an error.

index.html:1 Uncaught SyntaxError: Unexpected end of JSON input

/Users/dpsanders/.atom/packages/linter-julia/lib/linter-julia.coffee:72 WARNING: could not import Base.iszero into Main

/Users/dpsanders/.atom/packages/linter-julia/lib/linter-julia.coffee:72 connection ended with error EOFError()

SyntaxError: Unexpected end of JSON input
    at Object.parse (native)
    at Socket.<anonymous> (/Users/dpsanders/.atom/packages/linter-julia/lib/linter-julia.coffee:93:26)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at Pipe._handle.close [as _onclose] (net.js:493:12)

Versions:

Lint.jl 0.3
linter 2.1.0
linter-julia 0.7.1
linter-ui-default 1.2.1

My versions (Windows):

Lint.jl 0.3.0
linter 1.11.21
linter-julia 0.7.1
linter-ui-default not installed

This would indicate to me that maybe the reason to this bug is the change to linter 2.1.0.

My fail on macOS used:

Lint 0.3.0
linter 2.1.0
linter-julia 0.7.1
linter-ui-default 1.2.1

FWIW, I'm still seeing the error after updating to Lint.jl v0.4.

If someone could try to make clean installation of 0.7.2, it would be really appreciated. You probably need to uninstall linter and linter-ui-default before updating linter-julia.

I did a clean install of linter-julia, linter, and linter-ui-default. The problem seems to still be present. I currently have:

Lint 0.3.0
linter 1.11.21
linter-julia 0.7.2
linter-ui-default 1.2.1

SyntaxError: Unexpected end of JSON input
    at Socket.<anonymous> (/home/cynddl/.atom/packages/linter-julia/lib/linter-julia.coffee:93:26)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at Pipe._handle.close [as _onclose] (net.js:493:12)

@cynddl please uninstall linter-ui-default (if you won't need it for other linter), I think that is causing the problem. linter-ui-default is not yet needed with linter-julia.
I tried clean install on my windows machine, with these steps:

update Lint.jl v"0.4.0"
run Pkg.test("Lint") (in order to pre-compile Lint.jl)
uninstall linter, linter-julia
set correct julia path to juno (julia-client settings)
install linter-julia from settings, which asked to install linter 1.11.21

I also noticed that my autoinstaller #30 isn't working.

linter-ui-default was not causing the issue, as I tried with and without. I installed Lint.jl v0.4.0 and did a clean install of linter-julia and linter (1.11.21, from settings). The error message is still there.

@cynddl can you see the lintserver running in console, something like Server running on port/pipe \\.\pipe\lintserver6504nKjRJXcaGX2Ssock ... if the error comes after this message, then press Ctrl + Shift + F5 to restart Atom (or should it down and restart). That problem is issue #40, although not so well explained.

Yes, I get both:
Server running on port/pipe /tmp/lintserver22650PAngS3vuAHCIsock ...

and:
Uncaught SyntaxError: Unexpected end of JSON input

Also, it's followed by this message that didn't seem to be there before:
connection ended with error ArgumentError("stream is closed or unusable")

After restarting Atom (with F5), the errors messages are still happening (altough the connected ended comes before the SyntaxError)

@cynddl can you try to change linter settings: Lint As You Type Interval for something bigger than default 300, for example 1000. BTW Ctrl + s starts the linting, sometimes when I did debugging I pressed Ctrl + s multiple times to get the server - client to sync. I am not sure but I have the feeling that error message is because Lint.jl lintserver expects reading, but linter-julia is trying to write to the server (or vice versa). So far I have always get rid of this error when I get it working once.

I'm currently experiencing this exception on atoms #46

Could you try the latest version? Does this error still exit?
https://atom.io/packages/linter-julia

Most probably Lint.jl precombillation causes this issue. The lintserver is not up and running yet. Pressing Ctrl+s (saving the file) few times may resolve it or restarting Atom.

Sorry for the noise. This was a really old issue.

I do not have any issues. Are on the master repository of Lint.jl?

Opening a project (a folder) gives the error.

Fixed in #119


Why don't naming conventions seem to be applied at init?

https://github.com/laminlabs/lnschema-core/blob/ee8919ffa76d89a7d1d435af34de00e7da589b68/lnschema_core/dev/sqlmodel.py#L17-L23
They're clearly active at time of import...

This seems to behave correctly now. @falexwolf Could we close it?


Netflix API has changed, breaks extension

I've identified two problems with the current NVA extension:

The constant window.netflix.reactContext.models.serverDefs.data.API_BASE_URL does not exist anymore. It seems to be replaced with window.netflix.reactContext.models.serverDefs.data.API_ROOT = "https://www.netflix.com/api" and the object window.netflix.reactContext.models.services.data.api:
{
    hostname: "www.netflix.com",
    name: "api",
    path: [ "api", "shakti"]
    protocol: "https"
    secure: false
}

In order to fix this we need to fix the way we build our api uri. This is the current situation:
const url = `https://www.netflix.com/api${apiBaseUrl}/` +
            `${buildIdentifier}/${type}` +
            `?pg=${pageNum}&pgSize=${pageSize}&_=${timestamp}` +
            `&authURL=${authUrl}`

After some testing it seems that changing apiBaseUrl = "/shakti" fixes the API calls.

CSV export is broken, only the first twenty entries are visible in exported sheets. JSON seems to be fine.

@lesander update?

@lesander Can I submit a pull request to fix this?

@plotka Yes! Or just fork it!

Hi @plotka, of course! Your pull request is more than welcome!

Done #8

Thank you @plotka for your fix, I'll deploy the changes to the chrome store shortly.

Version 1.1.1 has been submitted to the chrome store for review.

The latest version is now published on the chrome store.


Use RX semantics for fragment interactions?

Yet another stylistic/preferential question--forgive me for having so many of these but once they're firmly answered I'll move forward without causing any trouble!
We've recently had some discussions about naming subjects that track user interactions appropriately such as (addFeatureClicks) etc. Most of these currently (or will) live in viewmodels as subjects. However, I think it may be more appropriate to track interactions (clicks, etc) in the fragment, granting the view models some level of independence.
This is already the case in some sense, but we currently often mix RX methods and approaches with more traditional direct VM method calls etc. Instead, we can stick to RX as much as possible, creating observables/subjects to track fragment interactions and capturing responses to view model data as transformations on these interactions.
see 4491816d5d29f3c52b7ce8b3d2699bc22c7da4a1 experimental/scolsen/observable-views for an example using this approach.
For reference, here's a relevant snippet from the previously mentioned commit:
First, we specify the appropriate fragment reaction in the onCreate method and subscribe to the stream:
  @Override
  public void<EMAIL_ADDRESS>Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    saveClicks = PublishSubject.create();
    singleSelectDialogFactory = new SingleSelectDialogFactory(getContext());
    multiSelectDialogFactory = new MultiSelectDialogFactory(getContext());
    viewModel = getViewModel(EditRecordViewModel.class);

    Observable<Boolean> saveClickReactions =
        saveClicks
            .map(__ -> viewModel.onSaveClick())
            .doOnNext(
                result -> {
                  if (!result) {
                    EphemeralPopups.showFyi(getContext(), R.string.no_changes_to_save);
                    navigator.navigateUp();
                  }
                });

    saveClickReactions.subscribe();

Second, when a user clicks, drags, w/e, we emit:
  @OnClick(R.id.save_record_btn)
  void onSaveClick() {
    saveClicks.onNext(new Object());
  }

Note that this is mostly a stylistic difference and it may be totally unnecessary if we're able to switch every fragment/VM data coupling to LiveData bindings. This approach does grant us some additional flexibility in that it's now fairly straightforward to accumulate further transformations/interactions on the initial stream, and it eliminates the need to track any state for dependent reactions (if any exist, we can simply model them as transformations, merges, etc.).
It also frees up

forgive me for having so many of these

Nothing to forgive, these are very productive foundational questions!

I think it may be more appropriate to track interactions (clicks, etc) in the fragment, granting the view models some level of independence.

Iiuc, in MVVM the idea of the View Model is to abstract the implementation details of the view, but not the details of interactions; conversely, the View Model is intended to be a model of the interactions between the user and the UI, modulo Android-specific "View" details. So in my understand, hiding some of the view behaviors might be considered a bug and not a feature.
Also, in many cases, user interactions require some UI change after some asynchronous event completes. In your example, we'd be navigating away from the Fragment and saying "Saved" before the save action actually completes. Since the async operations are kicked off by VM, it seems somewhat arbitrary to have sync. side effects handled in the View, and async. ones in the VM. It also spreads out reactions to user interactions among the View and View Model, making the logic harder to follow, and also making interactions handled inside the view almost impossible to test.
Does that make sense?
Btw, the BasicRx example also seems to have the problem of accumulating subscriptions; just filed https://github.com/googlesamples/android-architecture-components/issues/615 to verify if this is in fact an issue.

Correction: Looking at the delta in your change, I see now that the message is shown when there's nothing to save, and that it's already done that way in the current implementation. The points in my previous reply still apply, and that we should eventually refactor that logic to pass through the VM as part of Issue #9.

Ahha! Thanks for the clarification. It would certainly be silly to go against the principles of our architecture!
This makes sense. What's nice is that we can use streams to capture interactions in either case. Based on your explanation, keeping the interactions in the VM makes sense to me now.
I suppose what I'm really hoping to do is simplify the connection between the View and View Model in some standardized fashion. It'd be nice if we can eliminate some of the <T>Subjects that act as intermediaries of sorts between the V and VM. I suppose the best and most appropriate way of doing that is to use LiveData binding when possible.

I wish there were a more succinct and readable way to model these. The reason we end up with lots of Subjects is because we're trying to avoid any references from the ViewModel to the View. Passing the ViewModel a callback or stream of clicks would violate this. Also, using LiveData instead of Rx wouldn't reduce the number of streams needed, since we'd just end up  replacing each Subject with an analagous MutableLiveData. So basically I think the number of Subjects is an artifact of the clean MVVM architecture, not Rx per se.
That said, Rx stream chains can get pretty hairy, so we should try to mitigate that, for example by always putting actual business logic in methods rather than on lambdas in the chains.

Would it be helpful to just declare Subjects and public final, and to allow Views to invoke onNext directly? We're not encapsulating away the fact that Rx is being used, so it wouldn't break any existing abstractions, and it would obviate the need to boilerplate methods that just call onNext. Then exposed Subjects would become quasi method objects, à la:
viewModel.addFeatureClicks.onNext(NIL);
Or even better, databinding could call onNext directly.
Another observation: we should take care not to pass Views or Fragment references into the ViewModel streams for the reasons mentioned above, so perhaps streams without an object should be strongly typed (we can create a Nil class or just use Boolean for example) to prevent accidental misuse.

Re Subjects with no value, see https://stackoverflow.com/questions/47992689/rxjava2-subject-with-no-data

Would it be helpful to just declare Subjects and public final, and to allow Views to invoke onNext directly? We're not encapsulating away the fact that Rx is being used, so it wouldn't break any existing abstractions, and it would obviate the need to boilerplate methods that just call onNext. Then exposed Subjects would become quasi method objects, à la:
viewModel.addFeatureClicks.onNext(NIL);
Or even better, databinding could call onNext directly.
Another observation: we should take care not to pass Views or Fragment references into the ViewModel streams for the reasons mentioned above, so perhaps streams without an object should be strongly typed (we can create a Nil class or just use Boolean for example) to prevent accidental misuse.

I do like that approach! Another option, I think, for the cases in which we pass no/dummy values is to use a connectable observable. I think the semantics of this are pretty close to what we're actually trying to do, namely, kick-off emissions at a certain point in time. I think a subject makes more sense when we cannot initialize a value in the VM (e.g. we emit some runtime value from a field or something from the View). If I'm understanding connectables correctly, the benefit to this is that we can handle "dummy values" entirely in the VM. The view doesn't need to know anything about some dummy value contract, it would just call viewModel.observable.connect() at the appropriate time. It should work appropriately as long as we're doing our subscriptions in the constructor and calling connect after VM construction.
Creating some kind of utility Signal enum is also a good solution! I'm open to either strategy.

Hm, scratch that re: connectables, that actually wouldn't work.
I believe the simplest and best case is what you've suggested, creating some dummy signal and calling onNext directly in the view.

Yeah, ConnectableObservable appears to serve a different purpose.
My initial thought would be to create a Nil enum with one void value, NIL, but that then requires the layout xml to import the symbol to reference it. Creating a helper class that extends PublishSubject<Nil> and provides a no-arg onNext() might be a sol'n. I general relying on inheritance for util functions can lead to headaches, but this might be a case where it's tightly scoped enough to be worth the "cost".

Keeping it as is and having the onFooClick() methods would ultimately be more flexible, I think. If we had to update what happens on a click with some additional logic, we would simply add a couple of lines to the method, whereas if we made direct calls to onNext in the view and used some enum we lose some of that flexibility--we'd have to implement additional logic in the subject stream itself which gets complicated if we want to fire off multiple streams based on one click or if we need to do some complex preparatory work on the argument to onNext (in those cases in which it's not nil)--basically, we'd wind up needing a wrapper method anyway in the view to call each onNext (one might suspect we probably shouldn't have situations in which we need to kick off two separate subjects from a single interaction anyway, but you never know).
We can still use some dummy enum in VMs to make things extra clear, or just Object or Result<Object> (if we have some situation in which the call can somehow yield an error).
So, I guess what we're looking at is something like:
enum Interaction {
  DONE
}

PublishSubject<Interaction> fooBarClicks/drags/etc)s = PublishSubject.create();

// do some stream manipulation.

public void onFooBar<Interaction>() {
  fooBarClicks(/drags/etc)s.onNext();
}

// A concrete example

PublishSubject<Interaction> addFeatureClicks = PublishSubject.create();

// so some stream manipulation

public void onAddFeatureClick() {
  addFeatureClicks.onNext(Interaction.DONE)
}

I don't know what the best name for the interaction enum and member would be, some ideas: Nil.NIL, Interaction.INTERACTION, Interaction.DONE, Nothing.NOTHING, Signal.SIGNAL, Interaction.SIGNAL, UI.Interaction

Another option is to go all in on defining an interaction types and wrappers. For example, we could create a Click which in certain cases carries additional data (such as coordinates). (though I imagine Android might already provide something like this).

Keeping it as is and having the onFooClick() methods would ultimately be more flexible, I think.

+1. Looks like we came full circle on, but I feel more confident it's the right path now that we've fleshed out the alternatives.

I don't know what the best name for the interaction enum and member would be,

I'm slightly partial to Nil.NIL, since it's clear, concise, doesn't conflict with other Java reserved symbols, and is at the right level of abstraction (Rx streams vs. View logic). It probably belongs in the ..gnd.rx package.

Another option is to go all in on defining an interaction types and wrappers.

I think this would lead to a small combinatoric explosion of Subject types x event types, which is already well served by generics.

For example, we could create a Click which in certain cases carries additional data (such as coordinates). (though I imagine Android might already provide something like this).

We have Point value object to represent these, so Subject<Point> is correct, clear, and to the point. 🙄


Fix logr.Error accepting key-value pair instead of error

This PR fixes possible panics when the logr.Error() has an error instead of key-value string pairs as its third and onwards arguments.
Signed-off-by: Michail Resvanis<EMAIL_ADDRESS>

Codecov Report
Base: 73.90% // Head: 73.90% // No change to project coverage :thumbsup:

Coverage data is based on head (6a12592) compared to base (a9f4879).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #190   +/-   ##
=======================================
  Coverage   73.90%   73.90%
=======================================
  Files          26       26
  Lines        2667     2667
=======================================
  Hits         1971     1971
  Misses        602      602
  Partials       94       94

Impacted Files
Coverage Δ

internal/filter/filter.go
85.09% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

/ok-to-test

/lgtm
/assign @yevgeny-shnaidman


Load externa libraries by specifying a path

I want to load an external library (a cmxa file) that is in a specific directory. What should I write in the .merlin file ?
I have try to do this but it does not work :
FLG -I "~/Git/project/_build"
Thanks !

A cmxa contains object and linking information but no typing information. They are useful for producing executable but not for Merlin. You need to provide *.cmi and, optionally, *.cmt(i) files for Merlin to work.
The second problem is that merlin cannot expand ~/ in paths (though that could be a reasonable feature request). The correct specification would be:
B /home/.../Git/project/_build

where /home/.../ is your $HOME directory (in macOS, it starts with /Users/.../).


Allow fetching of all Flutter Outline refactors for a given file in a single request (or include in Outline)

In VS Code we need to context menu information at the time of rendering a tree node (context menus must be declared up-front with filters and then we apply context information to a note to tell VS Code which nodes have which contexts). This means in order to start the Outline expanded by default we send a getAssists request for every single node.
I think there are a couple of options:

Do nothing and assume that because the requests are parallel that it won't be too bad
Do nothing but assume we only ever have outlines against nodes Widgets (which we've assuming are anything where dartElement == null)
Include refactor information in Flutter outline responses
Support request flutter outline refactors in a single request for the whole file

For now I'm doing 2 so I can progress - the collapsed-by-default tree is annoying. LMK what you think!
@devoncarew @scheglov

Do nothing and assume that because the requests are parallel that it won't be too bad

Server is single threaded, so all requests are handled sequentially, not in parallel.

Server is single threaded, so all requests are handled sequentially, not in parallel.

Oh yeah, I was thinking about my round-trips. That said, depending on how you compute quick fixes it might not make much difference in the server (I guess you'd walk the tree looking for Widgets then compute their fixes, which might not be much different to me asking you for each location at a time?). Still should be some savings in the overhead of each request (and if you have to search to find the element at the offset I give you I guess it saves a bunch of them).
Do you have a preference for these? I think 3 is most convenient for me, but I don't know what overhead it adds for other editors that don't care. It's only really the IDs of the assists I need, but supplying just them might be a little VS-Code specific!

I don't like this. If VS Code does not support dynamic menus, we should request this feature from VS Code, not to perform lots of computations to work this around.

I'm not sure how this would work - they can't stall the rendering of the menu while waiting for a response from us? Slow context menus make a poor user experience.
Does IntelliJ have these on the context menu, or just the buttons? If the context menu, how does it get the data before the menu is rendered?

When the user right clicks on a tree item, this first causes selection event, and when selection happens we ask for assists. Once we receive assists from Analysis Server, we update toolbar. In the context menu provider we wait up to 100 ms for receiving assists (scheduled on selection). This is a bit too generous, but we usually get response from the server in 3-5 ms. So, there is never any noticeable delay between right click and displaying the context menu. It is OK-ish even with full 100 ms, although if you know what to look for, you will notice it.

I've raised a request with Code (https://github.com/Microsoft/vscode/issues/45325) but I'm not hopeful it'll be accepted :( Honestly, I think it'd be fair to reject - laggy UIs really suck and it's a valid question about how we have the data to render the tree node but not the data to know which context menu commands are valid.
If it is accepted, it's going to be at least a month away (maybe two if they put a proposed API into the next release). We can decide what to do once we know though (we're already blocked in a bunch of things so I suspect it might be a little while before we have something we're happy to ship anyway).

This does seem expensive - that we'd need to request the available refactorings for every widget node, on most file changes. I'm not sure what workarounds I can think of. Perhaps we just depend in the source code as the best place for users to discover these actions?

@scheglov The case for dynamic context menus doesn't seem to be progressing well... Is there any possibility of including info about which refactors are available inside the FlutterOutline nodes (option 3 of the original list)? Is it expensive to look them up there, or is the logic for which are available simple enough that it wouldn't make it more expensive (we're only interested in those seven or so flutter refactors)?

I would not like computing assists for all nodes.
This would make it slow for all clients, including IntelliJ based.
It is 3-5ms per node, and it will get too high quickly if we do this for all nodes.

We could limit this to only the nodes that represent items in the outline, but even then there can be quite a few.

Ah, yes, I was talking only about Flutter widgets in outline, not about all AstNode(s) in the compilation unit (ouch). The calculator demo in flutter_gallery has about 50 widgets creations. Adding 150-250 ms to every change is quite a cost.

Could it be opt-in?
If we don't get dynamic context menus in Code (I'm not hopeful it'll happen soon - feel free to 👍 or add support to https://github.com/Microsoft/vscode/issues/45325!) then we'll have to decide between not having the context menu (which will make Flutter Outline less useful) or paying this price (either internally in the server, or by Code just sending lots of requests).
Btw, we're only interested in some specific refactors - is there no short-cut to figuring out which ones are applicable without computing all assists?

Just to chip in, my sense is that the improved discoverability of adding a context menu likely isn't paid for by the additional performance cost (and small bit of customization to the API). My 0.02 -

... is there no short-cut to figuring out which ones are applicable without computing all assists?

There isn't currently, but there could be. AssistProcessor could have a flutter-outline-specific (ahem, flutter-layout-view-specific) entry point.

Another approach would be to show all actions in the menu, but when invoked ask to the actual Quick Assist from the server, and report "unavailable" it is not returned, or timed out.

It doesn't feel like a great user experience to show items that then fail when clicked. I have added a comment to the Code case asking about whether they'd be more likely to allow us to enable/disable the items asynchronously if they won't let us control the menu asynchronously. It would be up to us to make sure it's fast (and handle the case if a user manages to invoke an item before we disable it).

It's not looking good for getting support in Code:

Blocking/delaying the right click with extension code won't happen

I've asked about allowing the menu to appear immediately, but allowing us to (asynchronously) mark some commands as disabled (so they'd become grey - I think we can do it fast enough that the user wouldn't notice the change, and we can gracefully reject them if they are able to click).

Is there a specific part of computing which refactors are available that is particularly expensive? If there was an option to do less expensive work to filter out nonsense refactors but perhaps leave a few false positives that could provide a good user experience.
There are some use cases outside of VSCode where getting an outline with all refactors might also be useful. For example, if we showed icons triggering some refactors directly in the code editor in IntelliJ it would be nice if we could get them with the outline.


Cow: Watch brownstowntilerepair.com

Cow requests the watch of the watch_keyword brownstowntilerepair\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

brownstowntilerepair\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Self-contained deployments to Azure

Fixes #19882

This is good, but I sort of think this would be good as its own article that we maintain on each release.

@bradygaster  see https://github.com/dotnet/AspNetCore.Docs/issues/20326 and update that issue to what you want and I'll get right on it.


Byteorder version is conflicting

I'm using another library that uses byteorder, and it seems that Crust uses an older version which is causing conflict issues. The library uses 1.2.1 and above, while Crust uses 1.1.0 only. Are there any simple solutions here?

Actually, Crust is not using byteorder anymore. So closing the issue.


Download the latest version of npm

Description

If you can't find your npm version in the link, download the latest version.
ex. fosslight_download -s "https://www.npmjs.com/package/json-schema/" -t test_wget -d test_logs
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Documentation update
[ ] Refactoring, Maintenance
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

@dd-jy , Please review this PR.


What should be passed to the purchaseData param of validateAndLogInAppPurchaseAndroid method?

I'd like to preface this by saying that I reached to support directly as mentioned in the README over a week ago but didn't get a response so I am going to try reaching out here as a last resort.
What is supposed to be passed to the purchaseData param of the method below? I'm not seeing anything referencing this property in the latest Google Play Billing library documentation (https://developer.android.com/google/play/billing/integrate).
  AppsFlyer.validateAndLogInAppPurchaseAndroid({
      additionalParameters: {aa: 'cc'},
      currency: 'usd',
      price: '20',
      signature: 'the_signature',
      publicKey: 'your_public_key',
      purchaseData: 'the_purchase_data'
    })
      .then(r => alert('validateAndLogInAppPurchase success: ' + r.res))
      .catch(e => alert('validateAndLogInAppPurchase error: ' + e));

I did finally hear back from support but was told that the Google Play Billing Library v4 isn't officially supported yet. We're likely going to look for another service as this back and forth has been way too slow.

Hi @mcfarljw ,
Thank you for reaching out to us.
We do support Google Play Billing Library v4 as our API is agnostic from Play Billing Library. The required data is identical in both V3 and V4. The only difference is how to extract the required data:

V3: extract from onActivityResult
V4: extract from the Purchaseobject  received in the onPurchasesUpdated callback

Hey @pazlavi the problem ended up not being related to v3 or v4 but the documentation wasn't worded clearly on what to pass for Android purchases. It just says purchaseData: 'the_purchase_data' which is wording that is more relevant to the v3 billing library but that terminology isn't used for the v4 billing library.  I was told by support I should use purchase. getOriginalJson for the purchaseData field.
https://developer.android.com/reference/com/android/billingclient/api/Purchase#getOriginalJson()
I recommend updating your documentation to make this more clear for future developers.


Refactor Observatorium Api dashoard

Observatorium API is deployed as
observatorium-observatorium-api - api is in observatorium-production/stage namespace and thanos components in observatorium-metrics-production/stage namespace
observatorium-observatorium-mst-api -Both  api and Thanos components deployed in observatorium-mst-production/stage namespace
Changing the drop down selector to pick up only on the api job

test dashoard - https://grafana.stage.devshift.net/d/7hXbx6_4k/api2?orgId=1&refresh=1m&var-datasource=telemeter-prod-01-prometheus&var-job=observatorium-observatorium-mst-api&var-handler=All

General comment unrelated to the PR, but good to have it documented: I think we should move these dashboards to somewhere inside the https://github.com/observatorium org.

added namespace selector - https://grafana.stage.devshift.net/d/3yjYCel4z/api4?orgId=1&refresh=1m&var-datasource=telemeter-prod-01-prometheus&var-namespace=observatorium-mst-production&var-job=observatorium-observatorium-mst-api&var-handler=All


Cache k8s 1.25 related components: kube-proxy, kube-binary and crictl

What type of PR is this?

What this PR does / why we need it:
Caching kube-binary, kube-proxy and crictl for k8s 1.25.
Which issue(s) this PR fixes:

Fixes #
Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

Changes moved from here: https://github.com/Azure/AgentBaker/pull/2177, as it needs to be from a branch in the original repo (instead of the fork)


Metrics for healthchecks

It would be nice to have metrics for healthchecks. I was thinking something like this:
karma_healthcheck_up{alertmanager="am", filter="prom1"} 1
karma_healthcheck_up{alertmanager="am", filter="prom2"} 1

Is this possible to implement this? Maybe with other metric naming and/or labels, this is just an example. Thank you!

https://github.com/prymitive/karma/blob/25f920fc950eb7b56043d507a8c0fe9a5a191f8c/cmd/karma/metrics.go#L54

There’s already a metric tracing status of last pull from alertmanager.

Thanks for you response. AFAIK these are 2 different metrics, alertmanager is for the state of the alertmanager instance itself, but healthchecks confirm the agents are reporting properly. I wouldn't mind if this specific alertmanager_up metric could be extended with filter label instead, however I think they serve different purposes. That's why I suggested an healthchecks metric instead. Otherwise I have to adapt (and keep using) the deadman switch daemon instead (which feels a bit redundant since karma already have this info).


Code signing of .dylib files for MacOS

On MacOS it is not possible to either open or run games that are using this extension, as the loaded .dylib files are considered insecure by the latest MacOS security policies. To work around this issue, the dylib files need to be manually approved within the security section of the operating system, however, this is not an acceptable solution when trying to bundle the game for people to play. Also, it makes it difficult for collaborators to clone your game and try it out on their MacOS, as they will face the same issue.
Possible solution
The solution to this problem is to use codesign https://stackoverflow.com/a/61176509 where the artifacts need to be signed. We should document this issue on the README and include steps for people to circumvent that by code signing their artifacts (including the FMOD dylibs)
Please note that M1 laptops might require special treatment here as the security on them has been tightened by Apple!

Maybe we can add a code sign step in scons. But the problem exists also with libraries provided by fmod.

@piiertho exactly. Also, we'd need an Apple Developer license for the certificate and all that jazz, so maybe for now I will just document how to solve this for MacOS.

I think for now it is the best option. It seems even for open source apple require an organisation to get a certificate.
Other thing I don't know is if we can distribute fmod librairies, and more if we can sign them.

I think there are ways, considering the fact that there is an official Unity and an official Unreal plugin available. The goal of this project could be to become an official plugin for Godot as well, however, I am wondering what is required to make it onto a page like this: https://www.fmod.com/resources/documentation-unity?version=2.02&page=welcome.html
Surely, Unity must have paid them in some way, unless there is some other way of getting featured on there. I created https://github.com/utopia-rise/fmod-gdnative/issues/110 to look further into this.

Do we know of anybody active in the Godot community who might be willing to share an Apple Developer license with us? Surely there's someone who could lend us a code signing, unless Apple specifically needs each organization to do so in which case... f*** Apple?

unless Apple specifically needs each organization to do so in which case... f*** Apple?

The difficult part I think is licensing here - there is an official plugin for Unity and it might be that FMOD themselves provide signed binaries when downloading the plugin for Unity.
As part of this we might need to look into https://github.com/utopia-rise/fmod-gdnative/issues/110 first, however, as @piiertho already discovered this seems to be more tricky than expected. My wild guess is as soon Godot for gains more attention by AAA companies they will push for this plugin to become official.

unless Apple specifically needs each organization to do so in which case... f*** Apple?

The difficult part I think is licensing here - there is an official plugin for Unity and it might be that FMOD themselves provide signed binaries when downloading the plugin for Unity.
As part of this we might need to look into #110 first, however, as @piiertho already discovered this seems to be more tricky than expected. My wild guess is as soon Godot for gains more attention by AAA companies they will push for this plugin to become official.
Plan B could be to include a CLI script with instructions to auto-sign FMOD binaries as part of the setup of this plugin. Devs using this plugin would then need to provide their own certificates for this.

I think this is possible. IIRC, godot does this for Mac builds.


Fixed checkstyle EnforceFinalFieldsCheck violations in KeyVault classes

Based on #5030.

I'm ecstatic to see us removing Checkstyle suppressions, hope to see more of these in the future.


chore(tabs): switch to OnPush change detection

Switches all of the tab-related components to OnPush change detection and sorts out the issues that come from the changes.
Relates to #5035.
Note: I'm not a fan of having to subscribe to changes in the tab labels just so we can trigger change detection, but if we want something more practical, we'll have to restructure the tabs so they don't have to reach into other components.

Addressed the feedback @jelbourn.


Compilation failure with ffmpeg 5.0

Environment

OS Version: macOS 10.15
Source or binary build? source build all major versions 1-5

Description
ffmpeg 5.0 has just been merged in https://github.com/Homebrew/homebrew-core/pull/94423

Expected behavior: ignition-common compiles with ffmpeg
Actual behavior: it fails to compile

Steps to reproduce

brew install ignition-common5
observer build failure

Output
cd /tmp/ignition-common5-20220203-58814-14kxgm5/events/src && /usr/local/Cellar/cmake/3.22.2/bin/cmake -E cmake_link_script CMakeFiles/ignition-common5-events.dir/link.txt --verbose=1
/tmp/ignition-common5-20220203-58814-14kxgm5/av/src/VideoEncoder.cc:132:10: error: cannot initialize return object of type 'AVCodec *' with an rvalue of type 'const AVCodec *'
  return avcodec_find_encoder(_codecId);
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang++  -stdlib=libc++ -O3 -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -dynamiclib -Wl,-headerpad_max_install_names  -Wl,-undefined -Wl,dynamic_lookup -compatibility_version 5.0.0 -current_version 5.0.0 -o "../../lib/libignition-common5-events.5.0.0~pre1.dylib" -install_name @rpath/libignition-common5-events.5.dylib CMakeFiles/ignition-common5-events.dir/Event.cc.o CMakeFiles/ignition-common5-events.dir/KeyEvent.cc.o CMakeFiles/ignition-common5-events.dir/MouseEvent.cc.o  -Wl,-rpath,/tmp/ignition-common5-20220203-58814-14kxgm5/lib "../../lib/libignition-common<IP_ADDRESS>~pre1.dylib" "/usr/local/lib/libignition-math<IP_ADDRESS>~pre1.dylib" /usr/local/lib/libignition-utils<IP_ADDRESS>.dylib -lpthread
/tmp/ignition-common5-20220203-58814-14kxgm5/av/src/VideoEncoder.cc:370:21: error: cannot initialize a variable of type 'AVOutputFormat *' with an rvalue of type 'const AVOutputFormat *'
    AVOutputFormat *outputFormat = av_guess_format(nullptr,
                    ^              ~~~~~~~~~~~~~~~~~~~~~~~~
[ 66%] Building CXX object graphics/src/CMakeFiles/ignition-common5-graphics.dir/Image.cc.o
cd /tmp/ignition-common5-20220203-58814-14kxgm5/graphics/src && /usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang++ -DMAC_OS_X_VERSION=1080 -DTINYXML2_MAJOR_VERSION_GE_6 -Dignition_common5_graphics_EXPORTS -I/tmp/ignition-common5-20220203-58814-14kxgm5/include -I/tmp/ignition-common5-20220203-58814-14kxgm5/graphics/include -I/tmp/ignition-common5-20220203-58814-14kxgm5/core/include -isystem /usr/local/include/ignition/utils1 -isystem /usr/local/include/ignition/math7 -isystem /usr/local/Cellar/pcre/8.45/include -isystem /usr/local/Cellar/glib/2.70.3/include -isystem /usr/local/Cellar/glib/2.70.3/include/glib-2.0 -isystem /usr/local/Cellar/glib/2.70.3/lib/glib-2.0/include -isystem /usr/local/opt/gettext/include -isystem /usr/local/Cellar/gts/0.7.6_2/include -isystem /usr/local/Cellar/tinyxml2/9.0.0/include -stdlib=libc++ -O3 -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC -fPIC -I/usr/local/Cellar/pcre/8.45/include -I/usr/local/Cellar/glib/2.70.3/include -I/usr/local/Cellar/glib/2.70.3/include/glib-2.0 -I/usr/local/Cellar/glib/2.70.3/lib/glib-2.0/include -I/usr/local/opt/gettext/include -I/usr/local/Cellar/gts/0.7.6_2/include -I/usr/local/Cellar/tinyxml2/9.0.0/include -std=c++17 -MD -MT graphics/src/CMakeFiles/ignition-common5-graphics.dir/Image.cc.o -MF CMakeFiles/ignition-common5-graphics.dir/Image.cc.o.d -o CMakeFiles/ignition-common5-graphics.dir/Image.cc.o -c /tmp/ignition-common5-20220203-58814-14kxgm5/graphics/src/Image.cc
cd /tmp/ignition-common5-20220203-58814-14kxgm5/events/src && /usr/local/Cellar/cmake/3.22.2/bin/cmake -E cmake_symlink_library "../../lib/libignition-common5-events.5.0.0~pre1.dylib" ../../lib/libignition-common5-events.5.dylib ../../lib/libignition-common5-events.dylib
2 errors generated.
[ 66%] Built target ignition-common5-events
make[2]: *** [av/src/CMakeFiles/ignition-common5-av.dir/VideoEncoder.cc.o] Error 1

Fixed by #325 , thanks @traversaro !


Improve response compression

Currently all methods and all responses will be compressed if the client sends the Accept-Encoding header. There are a number of situations where it doesn't make sense to compress the response
Don't compress:

All Error responses
Empty list returns
null returns
primitive types
strings less than 500 characters (500 is a guestimate)

To achieve this ResponseMessage should have a Compress property that is [JsonIgnore]

I ended up going with 700 characters for string (~500 byte header still fits in one packet)
ICollection<> is inspected for length, IEnumerable<> is not because I don't want to enumerate it till serialize


Data Explorer

There's likely some overlap in functionality with the JupyterLab Data Explorer.
It's not stable yet but it might be useful to keep an eye on that project to see if you could leverage off the work there.

interesting. that looks like its being developed as part of jupyterlab core
but im wondering what their definition of 'dataset' is as it looks like its primarily aimed at working with physical files. it would be nice if variables created within kernel session could be automatically registered with the registry so extensions like this one can grab them directly. time will tell I suppose...

This extension looks really nice. If this is going to work with temporary in-kernel datasets, matrices, dataframes et al. I will gladly switch our current matrix viewer for the data explorer.

but im wondering what their definition of 'dataset' is as it looks like its primarily aimed at working with physical files. it would be nice if variables created within kernel session could be automatically registered with the registry so extensions like this one can grab them directly.

Yeah the MVP is about describing things like CSV files or notebook outputs. But I think being able to read data from the kernel would make sense as an extension.


client prefix missing from resource group names after resource group change

resolved by https://github.com/Food-X-Technologies/foodx_devops_tools/pull/108


chore(ci): Update renku actions to v1.6.0

Automated changes by create-pull-request GitHub action
/deploy

You can access the deployment of this PR at https://renku-ci-nb-1637.dev.renku.ch


Speed up addHoverClass on large stylesheets

Multi-megabyte stylesheets were incredibly slow with addHoverClass. I was able to bring that back by >90% on large stylesheets at a negligible cost on tiny stylesheets.
Results included below:
Running "4.1M stylesheet" suite...

  existingAddHoverClass:
    0 ops/s, ±0.90%   | slowest, 100% slower

  newAddHoverClass:
    2 ops/s, ±2.98%   | fastest

Finished 2 cases!
  Fastest: newAddHoverClass
  Slowest: existingAddHoverClass

Running "248K stylesheet" suite...

  existingAddHoverClass:
    56 ops/s, ±2.10%   | slowest, 32.53% slower

  newAddHoverClass:
    83 ops/s, ±1.86%   | fastest

Finished 2 cases!
  Fastest: newAddHoverClass
  Slowest: existingAddHoverClass

Running "105B stylesheet" suite...

  existingAddHoverClass:
    22 031 ops/s, ±2.32%   | fastest

  newAddHoverClass:
    21 493 ops/s, ±3.22%   | slowest, 2.44% slower

Finished 2 cases!
  Fastest: existingAddHoverClass
  Slowest: newAddHoverClass

Benchmark ran in node.js on a MacBook Pro (16-inch, 2019), 2,3 GHz 8-Core Intel Core i9.
Similar results found when running tests in Safari and Chrome.
Visual charts of these benchmarks: results.zip

I ran into a bug in one of the integration tests so I came up with a couple new variants. addHoverClassFilterFirst seemed to have the most upside with the least downside so I ended up picking that one: It is only 5% slower on the smallest stylesheets and it's still 99% faster on the larger stylesheets.
Benchmarks
Download graphs and granular results.
4.1M stylesheet
  addHoverClassOriginal:
    0.02 ops/s, ±0.80%   | slowest, 99.04% slower

  addHoverClassFilterFirst:
    2.09 ops/s, ±4.93%   | fastest

  addHoverClassSortFirst:
    2.07 ops/s, ±4.88%   | 0.96% slower

  addHoverClassUnique:
    1.97 ops/s, ±4.90%   | 5.74% slower

Finished 4 cases!
  Fastest: addHoverClassFilterFirst
  Slowest: addHoverClassOriginal

248K stylesheet
  addHoverClassOriginal:
    56.3 ops/s, ±1.01%   | slowest, 33.45% slower

  addHoverClassFilterFirst:
    83.8 ops/s, ±1.67%   | 0.95% slower

  addHoverClassSortFirst:
    84.6 ops/s, ±1.09%   | fastest

  addHoverClassUnique:
    84.3 ops/s, ±0.72%   | 0.35% slower

Finished 4 cases!
  Fastest: addHoverClassSortFirst
  Slowest: addHoverClassOriginal

105B stylesheet
  addHoverClassOriginal:
    23 851 ops/s, ±2.20%   | fastest

  addHoverClassFilterFirst:
    22 672 ops/s, ±2.97%   | 4.94% slower

  addHoverClassSortFirst:
    22 322 ops/s, ±2.10%   | slowest, 6.41% slower

  addHoverClassUnique:
    22 339 ops/s, ±2.89%   | 6.34% slower

Finished 4 cases!
  Fastest: addHoverClassOriginal
  Slowest: addHoverClassSortFirst

Tests pass for each variant
  add hover class to hover selector related rules
    addHoverClassOriginal
      ✓ can add hover class when there is a multi selector with the same prefix
      ✓ will do nothing to css text without :hover
      ✓ can add hover class to css text
      ✓ can add hover class when there is multi selector
      ✓ can add hover class when :hover is not the end of selector
      ✓ can add hover class when the selector has multi :hover
      ✓ will ignore :hover in css value
    addHoverClassFilterFirst
      ✓ can add hover class when there is a multi selector with the same prefix
      ✓ will do nothing to css text without :hover
      ✓ can add hover class to css text
      ✓ can add hover class when there is multi selector
      ✓ can add hover class when :hover is not the end of selector
      ✓ can add hover class when the selector has multi :hover
      ✓ will ignore :hover in css value
    addHoverClassSortFirst
      ✓ can add hover class when there is a multi selector with the same prefix
      ✓ will do nothing to css text without :hover
      ✓ can add hover class to css text
      ✓ can add hover class when there is multi selector
      ✓ can add hover class when :hover is not the end of selector
      ✓ can add hover class when the selector has multi :hover
      ✓ will ignore :hover in css value
    addHoverClassUnique
      ✓ can add hover class when there is a multi selector with the same prefix
      ✓ will do nothing to css text without :hover
      ✓ can add hover class to css text
      ✓ can add hover class when there is multi selector
      ✓ can add hover class when :hover is not the end of selector
      ✓ can add hover class when the selector has multi :hover
      ✓ will ignore :hover in css value

  28 passing (8m)

Different test cases:
addHoverClassFilterFirst
const HOVER_SELECTOR = /([^\\]):hover/;
const HOVER_SELECTOR_GLOBAL = new RegExp(HOVER_SELECTOR, 'g');
export function addHoverClassFilterFirst(cssText: string): string {
  const ast = parse(cssText, {
    silent: true,
  });

  if (!ast.stylesheet) {
    return cssText;
  }

  const selectors: string[] = [];
  ast.stylesheet.rules.forEach((rule) => {
    if ('selectors' in rule) {
      (rule.selectors || []).forEach((selector: string) => {
        if (HOVER_SELECTOR.test(selector)) {
          selectors.push(selector);
        }
      });
    }
  });

  if (selectors.length === 0) return cssText;

  const selectorMatcher = new RegExp(
    selectors
      .filter((selector, index) => selectors.indexOf(selector) === index)
      .sort((a, b) => b.length - a.length)
      .map((selector) => {
        return escapeRegExp(selector);
      })
      .join('|'),
    'g',
  );

  return cssText.replace(selectorMatcher, (selector) => {
    const newSelector = selector.replace(HOVER_SELECTOR_GLOBAL, '$1.\\:hover');
    return `${selector}, ${newSelector}`;
  });
}

addHoverClassSortFirst
export function addHoverClassSortFirst(cssText: string): string {
  const ast = parse(cssText, {
    silent: true,
  });

  if (!ast.stylesheet) {
    return cssText;
  }

  const selectors: string[] = [];
  ast.stylesheet.rules.forEach((rule) => {
    if ('selectors' in rule) {
      (rule.selectors || []).forEach((selector: string) => {
        if (HOVER_SELECTOR.test(selector)) {
          selectors.push(selector);
        }
      });
    }
  });

  if (selectors.length === 0) return cssText;

  const selectorMatcher = new RegExp(
    selectors
      .sort((a, b) => b.length - a.length)
      .filter((selector, index) => selectors.indexOf(selector) === index)
      .map((selector) => {
        return escapeRegExp(selector);
      })
      .join('|'),
    'g',
  );

  return cssText.replace(selectorMatcher, (selector) => {
    const newSelector = selector.replace(HOVER_SELECTOR_GLOBAL, '$1.\\:hover');
    return `${selector}, ${newSelector}`;
  });
}

addHoverClassUnique
function unique(arr: string[]): string[] {
  if (!Array.isArray(arr)) {
    throw new TypeError('array-unique expects an array.');
  }

  const len = arr.length;
  let i = -1;

  while (i++ < len) {
    let j = i + 1;

    for (; j < arr.length; ++j) {
      if (arr[i] === arr[j]) {
        arr.splice(j--, 1);
      }
    }
  }
  return arr;
}

export function addHoverClassUnique(cssText: string): string {
  const ast = parse(cssText, {
    silent: true,
  });

  if (!ast.stylesheet) {
    return cssText;
  }

  const selectors: string[] = [];
  ast.stylesheet.rules.forEach((rule) => {
    if ('selectors' in rule) {
      (rule.selectors || []).forEach((selector: string) => {
        if (HOVER_SELECTOR.test(selector)) {
          selectors.push(selector);
        }
      });
    }
  });

  if (selectors.length === 0) return cssText;

  const selectorMatcher = new RegExp(
    unique(selectors)
      .sort((a, b) => b.length - a.length)
      // .filter((selector, index) => selectors.indexOf(selector) === index)
      .map((selector) => {
        return escapeRegExp(selector);
      })
      .join('|'),
    'g',
  );

  return cssText.replace(selectorMatcher, (selector) => {
    const newSelector = selector.replace(HOVER_SELECTOR_GLOBAL, '$1.\\:hover');
    return `${selector}, ${newSelector}`;
  });
}

@Mark-Fenng @Yuyz0112 any chance we could get this merged?

l will start to review the PR tonight.

Thanks, @Juice10! The implementation looks great to me and I can reproduce similar results in some stylesheet examples.


use upstream yaml parser

fixes #59
With this PR, the actual parameters are being set:
 ➭ ros2 param list
...
/controller_manager:
  joint_state_controller.type
  joint_trajectory_controller.type
  update_rate
  use_sim_time

While implementing this, I found actually a second bug in which all specified controllers in the loaded yaml file would be loaded. It didn't show in the example, however this would necessarily lead to resource conflicts when more than one controller use the same command interface.

You didn't launch the controllers.

That's on purpose. As mentioned above, not all controllers can be loaded/configured at the same time due to resource conflicts. Also I believe it's not good practice to launch them within the gazebo plugin. I'd rather use either the spawner functionality (once available) or load them explicitly via services through a launch file.

I will have a look to the PR https://github.com/ros-controls/ros2_control/pull/310
anyhow the conditional to get the udpate_rate is required.

you don't necessarily need the spawner PR. You should be able to use ros2 control load_start_controller in the launch file to start the controllers.

anyhow the conditional to get the udpate_rate is required.

done in 1047c22

@Karsten1987
I was testing the ros2 control commands. I was able to start the joint_state_controller
ros2 control load_start_controller joint_trajectory_controller`

But I'm not able to load the joint_trajectory_controller. Digging about what it's happening here. The /joint_trajectory_controller parameters are not set.
ros2 param get /joint_trajectory_controller  joints
String values are: []

You should set up the parameters for the controller in the code.

There's currently no clear way to set parameters to a gazebo plugin's node. So I had to do this manually by parsing the ros parameter's file and add the arguments to the node's context. That's certainly not a recommended way, but the only feasible at the moment.
/cc @chapulina @jacobperron

I just wanted to add my 2c and say that as someone fairly new to all this, I found it quite confusing that the plugin loaded my controllers on its own as I expected a flow similar to my real robot where I used ros2 control load_start_controller xxx. The new behaviour resulting from this PR seemed more intuitive to me.
This PR resolved an issue I was having where the controller_manager and its child controllers did not obey the use_sim_time parameter I specified in the yaml file (and as I understand it there is currently no way to globally enable use_sim_time).

@ahcorde friendly ping.

I created a PR which builds on top of this PR load the controllers in the launch files: https://github.com/ros-simulation/gazebo_ros2_control/pull/64


Remove crashy unwrap() in on_error

I got a crash here, and I don't want a crash :)
thread 'av1_decoder' panicked at 'called `Option::unwrap()` on a `None` value'
6f12e3f/src/decode.rs:4997

It seems like this unwrap was introduced in https://github.com/memorysafety/rav1d/pull/663 along with many others.
May I suggest enabling the clippy::unwrap_used lint? Any .unwrap() is a panic waiting to happen :/
EDIT: The crash happens during decode of the chunk starting at timestamp 114176 of this video: https://storage.googleapis.com/rerun-test-assets/video/Sintel_1080_10s_av1.mp4

I was running this video (the crash happens about half-way through it):
https://storage.googleapis.com/rerun-test-assets/video/Sintel_1080_10s_av1.mp4

Hi @emilk, I have some time to look into this bug now.  What specific command did you use on Sintel_1080_10s_av1.mp4 that triggered the bug?  Sorry, I'm not super familiar with all of the CLI options besides what's run in the tests.

@emilk please let us know if you can help us reproduce the issue so we can test your solution.


Update submit picks to check for duplicate Wild Card submissions

It's possible right now that a player could submit the same team twice for the Wildcard choice.  Need to re-write the submit picks to do validation that the picks are different.
This will also require that the viewmodel passes the error back and displays to the user when they make the error.

This should also include validation if the Wild Card choice was already chosen as a division winner.  All division winners automatically make the playoffs, so the Wild Card choice != division winner.


New Version: TexasInstruments.TI-Nspire v5.3.2

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Closes #28752
Microsoft Reviewers: Open in CodeFlow

@denelon I see you've approved; Are the other labels preventing the automatic merge, or is this something that needs a waiver?

@Trenly is there a better URL for this? I didn't see the Domain label when I approved it. Yes, it will need a waiver with this URL.

@Trenly is there a better URL for this? I didn't see the Domain label when I approved it. Yes, it will need a waiver with this URL.

I looked and it seems like every URL that I find ultimately redirects back to that one

Redirects should be fine. Is there one from ti.com?

Redirects should be fine. Is there one from ti.com?

I do see one from TI. I'll update the manifest in just a moment

/azp run


Understand why logs aren't being generated by repo2docker for cosmicds hub

https://github.com/cosmicds/hub-user-image/ is a standard hub user image in use at the cosmicds hub. However, it is not generating user logs into the home directory as repo2docker should. Let's investigate why this is happening to this hub, and if this is happening to all hubs.

https://github.com/2i2c-org/infrastructure/pull/3433 is the reason. Opened https://github.com/2i2c-org/infrastructure/issues/3434 for systematic fixes.


ARROW-6212: [Java] Support vector rank operation

Given an unsorted vector, we want to get the index of the ith smallest element in the vector. This function is supported by the rank operation.
We provide an implementation that gets the index with the desired rank, without sorting the vector (the vector is left intact), and the implementation takes O(n) time, where n is the vector length.

Codecov Report

Merging #5066 into master will increase coverage by 2.1%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5066      +/-   ##
==========================================
+ Coverage   87.61%   89.71%    +2.1%
==========================================
  Files        1009      670     -339
  Lines      144081    99567   -44514
  Branches     1418        0    -1418
==========================================
- Hits       126230    89330   -36900
+ Misses      17489    10237    -7252
+ Partials      362        0     -362

Impacted Files
Coverage Δ

cpp/src/arrow/json/converter.cc
90.05% <0%> (-1.76%)
:arrow_down:

cpp/src/arrow/json/chunked-builder.cc
79.91% <0%> (-1.68%)
:arrow_down:

cpp/src/arrow/vendored/xxhash/xxhash.c
73.6% <0%> (-0.38%)
:arrow_down:

r/src/recordbatch.cpp

r/R/Table.R

js/src/util/fn.ts

go/arrow/array/bufferbuilder.go

r/src/symbols.cpp

rust/datafusion/src/execution/projection.rs

rust/datafusion/src/execution/filter.rs

... and 335 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93688e8...942cbfc. Read the comment docs.

+1, thank you.


Updating backend docstrings

Description
This PR attempts to add docstrings to all functions in the canonicalization backend. Notably, this includes the new NumPy backend that has recently been added in #2186 . This branch is WIP but a good chunk has already been completed, so a first review isn't out of the question. I also found a neat way to refactor indexing in the kron functions.
P.S: Adding non-trivial docstrings is actually kind of challenging, please bare with all the redundancy and reiterations that I may have introduced.
Type of change

[ ] New feature (backwards compatible)
[ ] New feature (breaking API changes)
[ ] Bug fix
[x] Other (Documentation, CI, ...)

Contribution checklist

[x] Add our license to new files.
[x] Check that your code adheres to our coding style.
[x] Write unittests.
[x] Run the unittests and check that they’re passing.
[x] Run the benchmarks to make sure your change doesn’t introduce a regression.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Transurgeon seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Great work adding these docstrings! Having docstrings sets a good pattern for future developers.

Benchmarks are failing because of a new asv release. Merging here as it only added docs.


Update documentation templates to remove duplicate line

Change Summary

Remove blank line when conditions are not met
Set default message when missing
Fix some rendering issues

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes)
[x] Documentation content changes
[ ] Other (please describe):

Related Issue(s)

Fixes issue #428
Component(s) name
eos_cli_config_gen
Proposed changes

Update documentation templates as per #428
Update CI artifacts

How to test

Run molecule scenario
Checklist:

[x] I have read the CONTRIBUTING document.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] All new and existing tests passed (pre-commit, make linting and make sanity-lint).
[x] I have updated molecule CI testing accordingly

LGTM


Avoid setting a locale as part of loaddata (Locale.DoesNotExist)

Issue Summary
Originally from #8097 - When running manage.py loaddata setting locale to Locale.get_default() is wrong.

There is no reason at all the locale is not also in the database dump and setting it to Locale.get_default() is completely random and more likely than not wrong.
We get the following error because at the point this code is called, the default locale does not exist yet:

wagtail.core.models.i18n.Locale.DoesNotExist: Problem installing fixture '/tmp/db.json': Locale matching query does not exist.

Full stacktrace:
Traceback (most recent call last):
  File "manage.py", line 8, in <module>
    sys.exit(main())
  File "main.py", line 9, in main
    execute_from_command_line(sys.argv)
  File "django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
  File "django/core/management/commands/loaddata.py", line 102, in handle
    self.loaddata(fixture_labels)
  File "django/core/management/commands/loaddata.py", line 163, in loaddata
    self.load_label(fixture_label)
  File "django/core/management/commands/loaddata.py", line 253, in load_label
    if self.save_obj(obj):
  File "django/core/management/commands/loaddata.py", line 209, in save_obj
    obj.save(using=self.using)
  File "django/core/serializers/base.py", line 288, in save
    models.Model.save_base(self.object, using=using, raw=True, **kwargs)
  File "django/db/models/base.py", line 861, in save_base
    pre_save.send(
  File "django/dispatch/dispatcher.py", line 176, in send
    return [
  File "django/dispatch/dispatcher.py", line 177, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
  File "wagtail/models/i18n.py", line 485, in set_locale_on_new_instance
    instance.locale = Locale.get_default()
  File "wagtail/models/i18n.py", line 63, in get_default
    return cls.objects.get_for_language(settings.LANGUAGE_CODE)
  File "wagtail/models/i18n.py", line 36, in get_for_language
    return self.get(
  File "django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
wagtail.models.i18n.Locale.DoesNotExist: Problem installing fixture '/tmp/db.json': Locale matching query does not exist.

Steps to Reproduce
Unknown
Technical details

Python version: Run python --version.
Django version: Look in your requirements.txt, or run pip show django | grep Version.
Wagtail version: Look at the bottom of the Settings menu in the Wagtail admin, or run pip show wagtail | grep Version:.
Browser version: You can use https://www.whatsmybrowser.org/ to find this out.

Working on this
This Django documentation describes the situation, and might be nice to have as reference: How fixtures are saved to the database?. There may be a lot of code to reuse from #8097, which had started and stalled.
Further feedback on the PR:

The default Locale should be created by the migrations and it should pick this up when you load your fixtures.
This may not be the case if you've changed the LANGUAGE_CODE setting after running migrations and before loading data or if you've flushed the database. Please could you confirm if you have done either of these?
There is no reason at all the locale is not also in the database dump and setting it to Locale.get_default() is completely random and more likely than not wrong.
If your data does not have a locale, I think it's sensible that Wagtail picks a default for you as it's a required field. Loading fixtures is very much a developer-only feature so I think it's reasonable to expect developers to set this to the correct locale after import if needed.

The next step is – we need:

more info and why you see the current behaviour as conflicting with Django's way of doing things.

I've hit this issue myself, and opened a new PR #12649.
The location this hit me is TransactionTestCase, which can optionally serialize and re-apply data using fixtures in between each test case. This is needed so models created in migrations are persisted, since TransactionTestCase truncates tables by default.
When that happens, the locale is often not created before a model which needs it, and so results in an error from the signal.
Generally, I've found the solution to this to be not doing things like this during fixture loads, and assuming the fixture itself is complete.


[BUG] StreamOutput write_v_int is broken

To reproduce:
variable_header = StreamOutput()
variable_header.write_v_int(4)

Expected results:
The byte 0x04 appended to the output stream
Actual results:
ERROR:root:a bytes-like object is required, not 'str'
Traceback (most recent call last):
  File "/Users/widdisd/git/opensearch-sdk-py/samples/hello/hello.py", line 65, in handle_connection
    variable_header.write_v_int(4)
  File "/Users/widdisd/git/opensearch-sdk-py/src/opensearch_sdk_py/transport/stream_output.py", line 20, in write_v_int
    return self.write(result)
           ^^^^^^^^^^^^^^^^^^

Analysis:
This implementation appears to define a string and append characters to it.
# writes an int in a variable-length format
def write_v_int(self, i: int):
    result = ''
    while True:
        result = chr((i & 0x7f) | 0x80) + result
        i >>= 7
        if ((i & ~0x7F) == 0):
            break
    return self.write(result)

This looks like it maybe would work if I could convert the string to bytes at the end, except the usual bytes() fails on lack of encoding and doesn't work for ascii because the values are higher than 128.  And there are a few more errors:

it prepends the character to the string, while vint appends them
it never writes the final byte after the loop break

This doesn't seem to match the implementation in OpenSearch where:

if the result can fit in a byte it shortcuts and writes the byte
it writes values less than 7 bytes but or's the high bit, appending to the end of the array
the final value is written without or'ing the high bit, which is how the reader knows to end reading the int

I've tried a few ways to fix it using bytearray and append but it's failing.  Working around it for now.

Doh... realized why it's failing, it writes 0x84 because it always hits that first loop iteration.  Short-cutting single byte would probably fix that.


Create a timeline subcommand to print its walreceiver state

If walreceiver is stuck due to some reason, it might be simpler to send an HTTP GET query into pageserver management API instead of checking the logs for error messages.
New HTTP endpoint should return:

all safekeepers whose messages were recently received from the broker
Safekeepers' metadata, related to timeline: LSNs, last message time, etc.
current active walreceiver connection metadata: SK id, time since connected, etc. or time since the last connection happened
latest feedback message metadata? Latest lsn caught up to would be useful

Might be more generic to expose the data via new metric(s).

We've been happy with the summary of different safekeepers in the seqwait timeout error. But this would be easy to add for querying via http, maybe it could even be in the timeline_detail because you'll probably want to see last_record_lsn and other lsn's at the same time...?
With "summary of different safekeepers" I mean: https://github.com/neondatabase/neon/blob/a1d6b1a4af5252e1215571257d4e79e40d961c11/pageserver/src/tenant/timeline.rs#L641-L658
Added in #4090. So the same contents, not necessarily formatted to a string but as a json would be handy as GET /v1/tenant/:tenant_id/timeline/:timeline_id/walreceiver for example, or as part of GET /v1/tenant/:tenant_id/timeline/:timeline_id.
Adding a good_first_issue because this is mostly plumbing in what we already have, perhaps adding a bit of serde::Serialize and openapi changes. However the openapi changes can be opaque, as in this is something internal to help with the on-call operator, not something to be relied for stability.


Season passing yards

Looking for help listing total passing yards for each team from the 2015 season.  I tried this but the numbers are a little off:
db = nfldb.connect()
q = nfldb.Query(db)
q.game(season_year=2015, season_type='Regular')
plays = q.as_plays()
passyds = []
aggregated = nfldb.aggregate(plays)  # Returns a list of PlayPlayer objects.
aggregated = sorted(aggregated, key=lambda p: p.passing_yds, reverse=True)
for pp in aggregated[0:32]:
passyds.append({'Team':pp.team, 'Passing Yards': pp.passing_yds})
passyds = pd.DataFrame(passyds)
print passyds
Thanks!

In my experience, the aggregate numbers are not going to perfectly match official sources but they are close enough for most purposes. If you run the example code for QB passing leaders, it is within a few yards of what is listed on other sites.
As an aside, it would be helpful to have more convenience methods to get the stats that people typically want, such as weekly and seasonal totals by team and player. The foregoing seems pretty complex just to get total passing yards for a season.

Thanks for the response. The numbers are a little off and for some reason I am getting two records for DEN with the above code. Have you found a better way to list total season stats?
Thanks.


Fix outdated comment

What does this PR do?
Removes an outdated comment.  The code is commented to state that we only use the "string" format for labels, but we actually use both (and even enforce the use of numeric labels in some cases https://github.com/DataDog/libdatadog/blob/20b50a9a8ed7f6a5427c25c88f6d9bd8270e2ccf/profiling/src/profile/mod.rs#L563 )
Motivation
Comments should reflect reality
Additional Notes
Anything else we should know when reviewing?
How to test the change?
This only removes a comment, no semantic change to code

Accidentally created on fork repo


Accept on Custom Records

DType.Accept  doesn't appear to work on custom records.
Add this test and get it to pass
https://github.com/microsoft/Power-Fx/blob/763c587a72b1c48af8fbe74cde799bae4e6141f2/src/tests/Microsoft.PowerFx.Interpreter.Tests/Scenarios/CustomRecordTypeWithUndefinedFields.cs#L28
    [Fact]
    public void TestAccept()
    {
        var t = _customRecordType._type;
        var ok = t.Accepts(t);

        Assert.True(ok);
    }

Per McCall, it should work, see:
see https://github.com/microsoft/Power-Fx/blob/763c587a72b1c48af8fbe74cde799bae4e6141f2/src/tests/Microsoft.PowerFx.Core.Tests/TypeSystemTests/LazyTypeTests.cs#L171


filmstrip ... button appears under raised filmstrip

Steps to reproduce:

Go into 3x3 mode
Select [America]
... button appears underneath the filmstrip

duplicate of #378


Network.removeTensors()

It would be nice to be able to remove all tensors from a Network file but keep the index contraction order. This way, the Network can be reused. If I just use PutTensor() several times I cannot easily check if I really reset all the tensors, so calling something like Network.removeTensors() before loading the new tensors would be nice.

This is implemented on #195 with RmUniTensor() and RmUniTensors()