[PDI-16844] The step Job Executor should overwrite the child parameter by the parent parameter.

[PDI-16844] The step Job Executor should overwrite the child parameter by the parent parameter.
Overwriting parameters and the next logic:

When the child parameter does not exist in the parent parameters, keep it.
When the child parameter does exist in the parent parameters, overwrite the child parameter by the parent parameter.
All other parent parameters need to get copied into the child parameters (when the 'Inherit all variables from the transformation?' option is checked)

Build Completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  engine/src/main/java/org/pentaho/di/job/entries/trans/JobEntryTrans.java
  engine/src/main/java/org/pentaho/di/trans/StepWithMappingMeta.java
  engine/src/main/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutor.java
  engine/src/main/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutorMeta.java
  engine/src/test/java/org/pentaho/di/job/entries/trans/JobEntryTransTest.java
  engine/src/test/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutorMetaTest.java

Newly Broken Tests:
org.pentaho.di.trans.steps.jobexecutor.JobExecutorTest.shouldAccumulateRowsByCount:
:red_circle: org.pentaho.di.core.exception.KettleException
org.pentaho.di.core.exception.KettleException:

There was an unexpected error:
 at org.apache.maven.surefire.booter.ForkedBooter.main (ForkedBooter.java:121)
 at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess (ForkedBooter.java:240)
 at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader (ForkedBooter.java:286)
 at org.apache.maven.surefire.junit4.JUnit4Provider.invoke (JUnit4Provider.java:184)
 at org.apache.maven.surefire.junit4.JUnit4Provider.execu

org.pentaho.di.trans.steps.jobexecutor.JobExecutorTest.shouldAccumulateRowsWhenGroupFieldIsSpecified:
:red_circle: org.pentaho.di.core.exception.KettleException
org.pentaho.di.core.exception.KettleException:

There was an unexpected error:
 at org.apache.maven.surefire.booter.ForkedBooter.main (ForkedBooter.java:121)
 at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess (ForkedBooter.java:240)
 at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader (ForkedBooter.java:286)
 at org.apache.maven.surefire.junit4.JUnit4Provider.invoke (JUnit4Provider.java:184)
 at org.apache.maven.surefire.junit4.JUnit4Provider.execu

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory

Branch Change: -7.2% :small_red_triangle_down:
Complexity Change: -7.7% :small_red_triangle_down:

org.pentaho.di.trans.step.StepMeta

Branch Change: -0.7% :small_red_triangle_down:
Complexity Change: -0.6% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:
Line Change: -0.3% :small_red_triangle_down:

org.pentaho.di.trans.steps.checksum.CheckSumMeta

Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.jobexecutor.JobExecutor

Branch Change: -5.2% :small_red_triangle_down:
Instruction Change: -5.6% :small_red_triangle_down:
Line Change: -9% :small_red_triangle_down:

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta

Branch Change: -0.5% :small_red_triangle_down:

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  engine/src/main/java/org/pentaho/di/job/entries/trans/JobEntryTrans.java
  engine/src/main/java/org/pentaho/di/trans/StepWithMappingMeta.java
  engine/src/main/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutor.java
  engine/src/main/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutorMeta.java
  engine/src/test/java/org/pentaho/di/job/entries/trans/JobEntryTransTest.java
  engine/src/test/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutorMetaTest.java
  engine/src/test/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutorTest.java

Unit Test Coverage
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.

:warning: Coverage Changes: <0.1% (click to expand)
org.pentaho.di.job.entries.simpleeval.JobEntrySimpleEval

Branch Change: -0.3% :small_red_triangle_down:
Complexity Change: -0.4% :small_red_triangle_down:
Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -3.1% :small_red_triangle_down:
Complexity Change: -3.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change: -2% :small_red_triangle_down:
Complexity Change: -1.9% :small_red_triangle_down:
Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta

Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.salesforceinsert.SalesforceInsertMeta

Instruction Change: -0.1% :small_red_triangle_down:

org.pentaho.di.trans.steps.salesforceupdate.SalesforceUpdateMeta

Instruction Change: -0.2% :small_red_triangle_down:

org.pentaho.di.trans.steps.salesforceupsert.SalesforceUpsertMeta

Instruction Change: -0.1% :small_red_triangle_down:

@bmorrise Could you please review and merge it?

@bmorrise Could you please review and merge it?

Build Completed
:white_check_mark: This pull request has passed all validations.
Build Commands
mvn -Dsurefire.runOrder=alphabetical -B -fn -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'engine' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  engine/src/main/java/org/pentaho/di/job/entries/trans/JobEntryTrans.java
  engine/src/main/java/org/pentaho/di/trans/StepWithMappingMeta.java
  engine/src/main/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutor.java
  engine/src/main/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutorMeta.java
  engine/src/test/java/org/pentaho/di/job/entries/trans/JobEntryTransTest.java
  engine/src/test/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutorMetaTest.java
  engine/src/test/java/org/pentaho/di/trans/steps/jobexecutor/JobExecutorTest.java


Opening menu on E?

Would this be an option in the future?
I've tried using the OpenRequestMenu in the shared (ENT:Use function) of the mediaplayer_base but couldn't get it to work.
Any ideas?

I don't have any plans to implement this feature into the addon, but it would be possible by creating a new entity that uses mediaplayer_base as its base entity. Here's some code (untested) that would use the behavior you've described:
ENT.base = "mediaplayer_base"

if SERVER then
	util.AddNetworkString("OpenReqMenu")

	function ENT:Use( ply )
		net.Start("OpenReqMenu")
			 net.WriteEntity(self)
		net.Send(ply)
	end
else
	net.Receive("OpenReqMenu", function()
		local ent = net.ReadEntity()
		MediaPlayer.OpenRequestMenu(ent)
	end)
end

Here's the full error I received (Note: The menu does pop up, however the mediaplayer doesn't turn on and the menu doesn't enable the cursor):
[ERROR] addons/gm-mediaplayer-master/lua/mediaplayer/controls/dmediaplayerreques
t.lua:84: attempt to index local 'mp' (a nil value)

GetServiceIDs - addons/gm-mediaplayer-master/lua/mediaplayer/controls/dmedi
aplayerrequest.lua:84
SendServices - addons/gm-mediaplayer-master/lua/mediaplayer/controls/dmedi
aplayerrequest.lua:92
3. SetMediaPlayer - addons/gm-mediaplayer-master/lua/mediaplayer/controls/dm
ediaplayerrequest.lua:101
4. OpenRequestMenu - addons/gm-mediaplayer-master/lua/mediaplayer/cl_init.l
ua:253
5. func - addons/gm-mediaplayer-master/lua/entities/mediaplayer_example/sh
ared.lua:22
6. unknown - lua/includes/modules/net.lua:32

Any ideas?

Sorry, I got caught up with other things. The issue has to do with the media player not existing on the client when it receives the net message. This is what the previous ENT:Use() function  managed.
When do you want clients to see the TV? If it should always be visible, try adding the following code:
if SERVER then
    function ENT:Initialize()
        self.BaseClass.Initialize( self )

        local mp = self:GetMediaPlayer()
        mp._TransmitState = TRANSMIT_ALWAYS
    end
end

ahh no problem, sorry for not clarifying. Basically I'm looking for the TV menu to (on use) open the menu for the person who pressed E on it and plays the video for anyone that also presses E on it.
Thanks :)

after adding that I seem to get this, this time the ENT spawns with the ability to walk through it, I've tried using SetModel and SetMoveType + Wake however nothing seemed to work.
This is the error I get when spawning one:
[ERROR] addons/gm-mediaplayer-master/lua/entities/mediaplayer_example/shared.lua
:16: attempt to index local 'mp' (a nil value)

unknown - addons/gm-mediaplayer-master/lua/entities/mediaplayer_example/sha
red.lua:16
Spawn - [C]:-1
3. SpawnFunction - gamemodes/base/entities/entities/base_entity/init.lua:77
4. Spawn_SENT - gamemodes/sandbox/gamemode/commands.lua:619
5. unknown - gamemodes/sandbox/gamemode/commands.lua:686
6. unknown - lua/includes/modules/concommand.lua:69

sorry for the rush, any more ideas on what I should do?


Ionic Capacitor-FCM - Error to subscribe/unsubscribe topic in iOS

I followed all the steps from tutorial and in android is working fine, but in iOS is showing only this error  when I try subscribe or unsubscribe a topic:
{"line":172816,"column":25,"sourceURL":"capacitor://localhost/vendor-es2015.js"}
My infos:
Ionic:
Ionic CLI                     : 5.4.16
Ionic Framework               : @ionic/angular 4.11.10
@angular-devkit/build-angular : 0.900.6
@angular-devkit/schematics    : 9.0.6
@angular/cli                  : 9.0.6
@ionic/angular-toolkit        : 2.0.0
Capacitor:
Capacitor CLI   : 2.0.0
@capacitor/core : 2.0.0
Cordova:
Cordova CLI       : 9.0.0<EMAIL_ADDRESS>Cordova Platforms : none
Cordova Plugins   : no whitelisted plugins (0 plugins total)
Utility:
cordova-res (update available: 0.11.0) : 0.10.0
native-run                             : not installed
System:
NodeJS : v12.16.1 (/usr/local/bin/node)
npm    : 6.13.4
OS     : macOS Catalina
Xcode  : Xcode 11.4 Build version 11E146

can you please set up a reproducible example ?

Hi, stewwan I resolved this problem.
I only had added pod 'CapacitorFcm', :path => '../../node_modules/capacitor-fcm' in podfile and works fine!

Thanks. Closing for now.. feel free to file a new issue if you find something odd.


Bump current iOS version

This PR bumps the current iOS version from 15.1 to 15.1.1.

Apple release: https://developer.apple.com/news/releases/?id=11172021

@Ein-Tim Thanks for the PR.


Update Ruby docs 2022/02/04

Closes #1029
Will bring in the changes of the issues/PRs cross-referenced by GH below in addition to the following:

https://github.com/open-telemetry/opentelemetry-ruby/pull/1081
https://github.com/open-telemetry/opentelemetry-ruby/issues/1104
https://github.com/open-telemetry/opentelemetry-ruby/pull/1120
https://github.com/open-telemetry/opentelemetry-ruby/pull/1121

Reopening for:

[ ] #1100

Actually, will keep this closed and track the Ruby-docs update via #1100.


stackdriver: update region tags

This also adds a new snippet that was previously inlined in docs.
For posterity, the Stackdriver Exporter looks in application default credentials
for the project, so we don't need to specify the project manually.

from memory, project id is only available from creds if obtained via service account JSON.


strpos(): Empty needle

some times i get this error:
/home
/admin
/domains
/mydomain
/vendor
/cviebrock
/eloquent-sluggable
/src
/Services
/SlugService.php

            //     a) the list is empty, or
            //     b) our slug isn't in the list
            // ... we are okay
            if (
                $list->count() === 0 ||
                $list->contains($slug) === false
            ) {
                return $slug;
            }

            // if our slug is in the list, but
            //     a) it's for our model, or
            //  b) it looks like a suffixed version of our slug
            // ... we are also okay (use the current slug)
            if ($list->has($this->model->getKey())) {
                $currentSlug = $list->get($this->model->getKey());

                if (
                    $currentSlug === $slug ||
                    strpos($currentSlug, $slug) === 0
                ) {
                    return $currentSlug;
                }
            }

            $method = $config['uniqueSuffix'];
            if ($method === null) {
                $suffix = $this->generateSuffix($slug, $separator, $list);
            } elseif (is_callable($method)) {
                $suffix = $method($slug, $separator, $list);
            } else {
                throw new \UnexpectedValueException('Sluggable "uniqueSuffix" for '.get_class($this->model).':'.$attribute.' is not null, or a closure.');
            }

            return $slug.$separator.$suffix;
        }

        /**
         * Generate a unique suffix for the given slug (and list of existing, "similar" slugs.
         *

Arguments

    "strpos(): Empty needle"

This issue occurs when your sluggable fields are null, you've already generated a slug, and you have 'onUpdate' set to true.
ERROR: strpos(): Empty needle
vendor/cviebrock/eloquent-sluggable/src/Services/SlugService.php(295): strpos('-14', '')
You need to check for empty slugs, can probably fix this by doing something like:
                $slug && strpos($currentSlug, $slug) === 0

Actually, it would probably be more correct to do, on line 295 of SlugService.php:
!$slug || strpos($currentSlug, $slug) === 0


WIP - When private_key is supplied as runner param, use that key for auth.

(virtualenv)/m/s/s/st2 git:issue_1889_private_key ❯❯❯ st2 run examples.remote-fib hosts=localhost start=1 end=10 username=stanley
.
id: 55dfa56732ed3504cd4be39c
status: failed
result:
{
    "localhost": {
        "failed": true,
        "traceback": "  File "/mnt/src/storm/st2/st2actions/st2actions/runners/ssh/parallel_ssh.py", line 221, in _connect
    client.connect()
  File "/mnt/src/storm/st2/st2actions/st2actions/runners/ssh/paramiko_ssh.py", line 134, in connect
    self.client.connect(**conninfo)
  File "/mnt/src/storm/st2/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 307, in connect
    look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)
  File "/mnt/src/storm/st2/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 519, in _auth
    raise saved_exception
",
        "return_code": 255,
        "succeeded": false,
        "error": "Cannot connect to host. not a valid EC private key file"
    }
}
(virtualenv)/m/s/s/st2 git:issue_1889_private_key ❯❯❯ st2 run examples.remote-fib hosts=localhost start=1 end=10 username=stanley private_key=/home/vagrant/.ssh/stanley_rsa
.
id: 55dfa57a32ed3504cd4be39f
status: succeeded
result:
{
    "localhost": {
        "succeeded": true,
        "failed": false,
        "return_code": 0,
        "stderr": "",
        "stdout": "1 1 2 3 5 8 13 21 34
"
    }
}
(virtualenv)/m/s/s/st2 git:issue_1889_private_key ❯❯❯ grep -A 3 "system_user" ~/st2.dev.conf                              ✭ ◼
[system_user]
user = lakshmi
ssh_key_file = /home/vagrant/.ssh/id_rsa

(virtualenv)/m/s/s/st2 git:issue_1889_private_key ❯❯❯

TODO
[ ] - Error message should tell user what private_key file was used.
[ ] - Tests

So if a user's private key is present on StackStorm box any user with access to StackStorm can execute an action on a remote box as that user?
Are we sure this is a good idea?

The public key still needs to be deployed to the boxes that the users are connecting to. Only then, authentication will succeed.
private_key will be in the box where the client is. I am hoping people don't share their private_keys. Otherwise, this will be a disaster anyways.

Fabric behavior is bizzare! Look at the executions with fabric with username & username + private_key provided. https://gist.github.com/lakshmi-kannan/d22e697ea032b3f7d4d0
The intended behavior is to use the private_key provided but I think there might be a bug there.
https://github.com/StackStorm/st2/blob/master/st2common/st2common/models/system/action.py#L450

private_key will be in the box where the client is.

st2 run examples.remote-fib hosts=localhost start=1 end=10 username=stanley private_key=/home/vagrant/.ssh/stanley_rsa
The key is read by the CLI? We actually have some special handling in the CLI for this file?

Fabric behavior is bizzare! Look at the executions with fabric with username & username + private_key provided. https://gist.github.com/lakshmi-kannan/d22e697ea032b3f7d4d0

https://github.com/StackStorm/st2/blob/master/st2actions/st2actions/runners/ssh/fabric_runner.py#L46
Not really it is exactly as coded.

The key is read by the CLI? We actually have some special handling in the CLI for this file?

I misunderstood this part. I am confused what's the behavior we want. So yeah, let's talk offline about this and then I'll open a PR.

I suggest you refactor it breaking current troubleshooting.rst in blocks

Lets do that in subsequent PRs please.

@lakshmi-kannan This looks like it is still WIP. I don't expect us to ship this for 0.13.1 any longer.

@manasdk Not for 0.13.1. I have to add some tests and get the docs changed.

Unfortunately, this PR also touched docs and also other things. Apologies for the giant PR.

A bit late to the game, but:

The key is read by the CLI? We actually have some special handling in the CLI for the private_key property?

It's only considered a file path and read if you prefix argument name with "@" (e.g. @private_key=/path/to/key/file). We follow similar convention as curl and also do the same for HTTP runner.
I believe this also documented in the CLI 101 section.

Hey @lakshmi-kannan
Somehow in the confusion, I missed this. This really isn't ideal. Read up on some rant.
https://stackstorm.slack.com/archives/stackstorm/p1445294185009809
Let's see how we can still provide what the user needs, but in a safer way


Sharing: take into account stage attachments

EPAM AI DIAL chat version
0.5.3
What happened?
Currently, DIAL Chat takes into account only the attachments that are attached directly to individual chat messages.
However, according to the DIAL API, each message may also have multiple stages each of which could have attachments.
Those must also be taken into account in the sharing logic, otherwise recepient of the shared link won't have an access to the stage attachments:
https://github.com/epam/ai-dial-chat/blob/f9fd30c90f98df8350dc2748a8f12b847dd10133/apps/chat/src/store/share/share.epics.ts#L52-L64
Confidential information

[X] I confirm that do not share any confidential information

Related to https://github.com/epam/ai-dial-core/issues/353


feature request: remove netrw dependency

Did you check existing requests?

[X] I have searched the existing issues

Describe the feature
It would be nice to completely disable netrw and use oil as a replacement, including for remote SSH/SCP operations.
Provide background
Netrw is buggy.
Additional details
I have written a small plugin called scamp.nvim that does this, and several others exist like netman. Happy to help if this interests you.

I have netrw disabled and it works for me

I have netrw disabled and it works for me

According to the README, it is required:

The only thing oil relies on netrw for is editing files in the format scp://servername/path/to/file.txt. If you have some other way of doing that (looks like that's what scamp.nvim does), then it should just work.
Two things that do not work with scamp from my tests:

scamp requires the username to be present in the url or it will throw an error
scamp changes the buffer name from scp://* to /tmp/*. That means that once you open a remote file, you can't browse the parent directory with oil.open().

If you can fix these then I'll add a note in the README that scamp.nvim can be used as an alternative to netrw.
I may build this functionality directly into oil.nvim in the future, or switch it up to rely on netman. Undecided as of yet.

I've added builtin support for file editing via scp. Netrw is no longer required

Doesn't work in my testing. Just get "Unknown SSH Error"

Sharing my observation: I ran :e scp://<username>@<server>/<cr> and it worked, I disabled netrw and removed scamp.nvim before running this command.

Just chiming in here, removing netrw wont break this plugin but it will break other features in vim. I would advise documenting this if you are recommending users disable netrw.
Examples of things that break when you disable netrw

tar/zip browsing
rsync browsing
webdav browsing
local filesystem browsing (with native vim, most third party plugins do not require netrw to function).

@miversen33 I think there is a misunderstanding here. This is for those who already disabled netrw. For example I always disliked netrw and actively searched to find a way to disable it in Neovim long ago and intentionally disabled it. So I was bothered that I should bring it back to activate ssh feature of oil and now I'm happy I can keep it disabled and use scp:// schema.
I mean whether to use or not use netrw is a personal decision and beyond the scope of this plugin I think, it is not a requirement of using oil as far as I understand.

@sassanh  that is incorrect, oil.nvim does indeed disable netrw. And that is fine, I am simply stating that by doing so, there are side effects that the developer (@stevearc) may not have been aware of. I dove into this a bit in my own plugin where I uncovered some consequences of disabling netrw (the ones I listed above and more). Netrw is sadly, quite engrained in vim core unfortunately, alot of things rely on it.
Disabling it is fine, but it should probably be called out somewhere in the readme (or help doc would probably be better) that there can be some side effects for disabling netrw, which this plugin does.

You are right, this is new in ssh support commit.
Yes I agree with you that it's better to be either optional (default off) or mentioned explicitly somewhere.

And if we disable netrw, shouldn't we also disable netrwSettings, netrwFileHandlers just like netrwPlugin?

Its been a while since I have dove into netrw, but I seem to recall that the later variables (netrwSettings and netrwFileHandlers) are not set (and thus null/0/whatever tf vimscript considers "falsey") by default. IE, because netrw short circuits if loaded_netrwPlugin and loaded_netrw are already set, it just returns, assuming its work is already done. Basically, setting those flags is a abusing the general practice of vimscript plugins where they check global state variables to see if they need to initialize themselves, as they may have already done so.
I don't believe setting those flags will hurt anything, though I don't believe they are necessary to "prevent" netrw from hijacking uri reads (via preventing it from setting its aucommands).

I see, but even If those two don't hijack uri, there is still benefit in disabling them: By turning off netrw and netrwPlugin we are probably making the other two obsolete and most users probably won't know about it to disable them themselves and get the startup time benefit.

my two cents:

Oil is shaping up to replace netrw for me. I wish we could either fix netrw to be less broken or somehow replace it in Neovim altogether. Oil right now is the next best thing for my use case.

I personally do not think disabling netrw should be automagic. It should be documented in the README / :help as a recommended configuration option. But Oil is not my baby, so I respect whichever way the maintainer chooses to go.

imo netrw definitely shouldn't be disabled if it's somehow avoidable
if it isn't it should be written with a disclaimer in the readme @stevearc

@miversen33 you are correct, I was under the impression that netrw only handled local and ssh file browsing, but I see it's also handling rsync, ftp, http, etc. I'm confused that you mentioned tar/zip browsing, because it looks like those are handled by separate tarPlugin.vim, gzip.vim, and zipPlugin.vim files. I did some simple tests and they seem to function even with netrw disabled.
I think the correct thing to do is to change the ssh editing scheme back to something that doesn't conflict (scp:// -> oil-ssh://). Unfortunately, that will produce some thrash since I just changed that scheme, but I can add some affordances to make it a bit easier to notice and migrate. I really hate doing anything backwards-incompatible, but I think this has to be done. Once the url schemes don't overlap with netrw, then the user can disable it or not at their leisure.
I'll remove the auto-disabling logic and leave a note in the README about how one can disable netrw.
@nat-418 with respect to the "Unknown SSH Error", I added some more details to that notification that should help you investigate a bit. Basically, the ssh process we're spawning (which we do even if you're editing a file and not a directory) is failing with a nonzero exit code. If you run :lua require("oil.adapters.ssh").open_terminal() you should be able to see the command output; hopefully it'll have some clues as to why it failed.

@miversen33 you are correct, I was under the impression that netrw only handled local and ssh file browsing, but I see it's also handling rsync, ftp, http, etc. I'm confused that you mentioned tar/zip browsing, because it looks like those are handled by separate tarPlugin.vim, gzip.vim, and zipPlugin.vim files. I did some simple tests and they seem to function even with netrw disabled.
I think the correct thing to do is to change the ssh editing scheme back to something that doesn't conflict (scp:// -> oil-ssh://). Unfortunately, that will produce some thrash since I just changed that scheme, but I can add some affordances to make it a bit easier to notice and migrate. I really hate doing anything backwards-incompatible, but I think this has to be done. Once the url schemes don't overlap with netrw, then the user can disable it or not at their leisure.
I'll remove the auto-disabling logic and leave a note in the README about how one can disable netrw.
@nat-418 with respect to the "Unknown SSH Error", I added some more details to that notification that should help you investigate a bit. Basically, the ssh process we're spawning (which we do even if you're editing a file and not a directory) is failing with a nonzero exit code. If you run :lua require("oil.adapters.ssh").open_terminal() you should be able to see the command output; hopefully it'll have some clues as to why it failed.

You could also just keep the ssh:// scheme and put it behind a feature flag like replace_netrw or something.

Sharing my observation: I ran :e scp://<username>@<server>/<cr> and it worked, I disabled netrw and removed scamp.nvim before running this command.

This does not work for me on the latest HEAD

@stevearc After looking back through netrw source it is remote compressed files it allows browsing of, not local. My mistake, its been a hot minute since I dug into the netrw source lol.
I have edited my above comment to reflect :)
Potentially another way to handle this is with some good ole "Hackity Schmakity".
Something I have been thinking about (but haven't tried yet) is auto overriding the ssh rules in the netrw augroup (or rather stripping them out altogether). You could in theory do this by setting an aucommand on the UIEnter event that checks to see if whatever your augroup is exists, and if it does, it auto rewrites the autocommands for netrw, stripping out whatever adapters you overwrite (such as ssh).
Unfortunately you can't just remove them on init of the plugin as it seems that the vimcore plugins are triggered during the load of lua plugins, thus you end up in a race condition of sometimes you can see the netrw augroup exists and other times you dont.
If however, you abuse the UIEnter event, you might be able to quickly do the above rewrite (which should in theory be "instant", especially if you pregenerate what the new command should look like during the init of oil.nvim).
Its awfully disgusting but might allow both netrw and this to coincide. Its an avenue I plan on exploring more when I get done dicking with async stuff in netman, but if you are feeling extra adventerous, it might be worth a try, if you want to get netrw and oil.nvim to live together relatively peacefully

We need to take care of running vim with an argument starting with scp:// too. I think BufEnter and FileReadPost run before UIEnter, it shouldn't be hard to handle though.
Can we use VimEnter instead of UIEnter? I'm not sure "ui" will be entered when Neovim is running embedded, not sure.

All good points and I will not atest to either idea working, it's just a theory I have in my head at this time. The main idea is, find an event that fires after vim is loaded but before buffers begin rendering. VimEnter may be it, I don't know for sure :)

@miversen33 that's a neat idea, and I definitely think it could be made to work. I'm personally hesitant to add that amount of coupling and hacky magic unless is absolutely necessary, and I'm not yet convinced that it is. Conflicts should only ever be seen by people that for some reason want both oil.nvim and netrw to be active, and even then they mostly coexist now that I changed the scheme back to oil-ssh://. If someone writes in really needing this I'd consider it, but more likely I'd come up with a snippet they could put in their own config instead of including it in oil directly.


Unable to compile my smart contract when including eosio.system contract header files

What I expected
eosio-cpp --contract sendeos eosio_docker/contracts/sendeos/sendeos.cpp
    -o build/sendeos/sendeos.wasm -abigen
    -I./eosio_docker/libraries/eosio.contracts/eosio.system/include
    -I./eosio_docker/contracts/sendeos/sendeos

Exit code 0, successful compilation
What I actually got
eosio-cpp --contract sendeos eosio_docker/contracts/sendeos/sendeos.cpp
    -o build/sendeos/sendeos.wasm -abigen
    -I./eosio_docker/libraries/eosio.contracts/eosio.system/include
    -I./eosio_docker/contracts/sendeos/sendeos

/usr/local/Cellar/eosio.cdt/1.3.2/opt/eosio.cdt/bin/wasm-ld: error: build/sendeos/sendeos.wasm: undefined symbol: eosiosystem::exchange_state::convert(eosio::asset, eosio::symbol const&)

Current setup

eosio.cdt v1.3.2
eosio.contracts 1.4.1
MacOS X 10.14.1

Relevant Code
#include <eosiolib/asset.hpp>
#include <eosiolib/action.hpp>
#include <eosiolib/public_key.hpp>
#include <eosiolib/eosio.hpp>
#include <eosiolib/system.h>
#include <eosio.system/exchange_state.hpp>
#include <eosio.system/native.hpp>

using namespace std;

class [[eosio::contract]] sendeos : public eosio::contract {

private:

// Omitted...

  eosio::asset buyrambytes(uint32_t bytes) {
    eosiosystem::rammarket market("eosio"_n, "eosio"_n.value);
    auto itr =
        market.find(eosio::symbol(eosio::symbol_code("RAMCORE"), 4).raw());

    eosio_assert(itr != market.end(), "RAMCORE market not found");

    auto tmp = *itr;
    return tmp.convert(
        eosio::asset(bytes, eosio::symbol(eosio::symbol_code("RAM"), 4)),
        eosio::symbol(eosio::symbol_code("EOS"), 4));
  }

// Omitted...

Findings

The exchange_state struct is inside a namespace eosiosystem and has a function convert with the signature

https://github.com/EOSIO/eosio.contracts/blob/v1.4.1/eosio.system/include/eosio.system/exchange_state.hpp#L33

I'm using the correct version of eosio.contracts that works with eosio.cdt

https://github.com/EOSIO/eosio.contracts/blob/v1.4.1/eosio.system/include/eosio.system/exchange_state.hpp#L33

Building eosio.contracts results in the following:

	=========== Building eosio.contracts ===========

-- The C compiler identification is AppleClang <IP_ADDRESS>01044
-- The CXX compiler identification is AppleClang <IP_ADDRESS>01044
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting up Eosio Wasm Toolchain 1.3.2 at //usr/local
-- Using eosio.cdt version 1.3.2
-- Found Git: /usr/local/bin/git (found version "2.19.1")
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/danieldyba/Code/sendeos/eosio_docker/libraries/eosio.contracts/build
Scanning dependencies of target eosio.bios.wasm
Scanning dependencies of target contracts_unit_tests
Scanning dependencies of target eosio.msig.wasm
Scanning dependencies of target eosio.wrap.wasm
[ 10%] Building CXX object eosio.bios/CMakeFiles/eosio.bios.wasm.dir/src/eosio.bios.cpp.o
[ 10%] Creating directories for 'contracts_unit_tests'
[ 21%] Building CXX object eosio.wrap/CMakeFiles/eosio.wrap.wasm.dir/src/eosio.wrap.cpp.o
[ 21%] Building CXX object eosio.msig/CMakeFiles/eosio.msig.wasm.dir/src/eosio.msig.cpp.o
[ 26%] No download step for 'contracts_unit_tests'
[ 31%] No patch step for 'contracts_unit_tests'
[ 36%] No update step for 'contracts_unit_tests'
[ 42%] Performing configure step for 'contracts_unit_tests'
-- The C compiler identification is AppleClang <IP_ADDRESS>01044
-- The CXX compiler identification is AppleClang <IP_ADDRESS>01044
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:14 (include):
  include could not find load file:

    EosioTester

CMake Error at CMakeLists.txt:28 (message):
  Found eosio version but it does not satisfy version requirements: version
  '' is invalid

  Please use eosio version 1.3.x

-- Configuring incomplete, errors occurred!
See also "/Users/danieldyba/Code/sendeos/eosio_docker/libraries/eosio.contracts/build/tests/CMakeFiles/CMakeOutput.log".
make[2]: *** [contracts_unit_tests-prefix/src/contracts_unit_tests-stamp/contracts_unit_tests-configure] Error 1
make[1]: *** [CMakeFiles/contracts_unit_tests.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 47%] Linking CXX executable eosio.wrap.wasm
[ 47%] Built target eosio.wrap.wasm
[ 52%] Linking CXX executable eosio.bios.wasm
[ 52%] Built target eosio.bios.wasm
[ 57%] Linking CXX executable eosio.msig.wasm
[ 57%] Built target eosio.msig.wasm
make: *** [all] Error 2

My suspicion is that this might be related to eosio.contracts, but the original compilation error message doesn't make that clear and points to a tool used in eosio.cdt.

You're getting undefined symbol error because compiler cannot find the definition of eosiosystem::exchange_state::convert member function.
Did you include in your project exchange_state.cpp and compile it along with your contract?

@dyba No need to.
You can either copy exchange_state.cpp to your project folder (or reference it) and include it in your project file.
Or, you can just include it in the source code where it's needed (e.g. #include "exchange_state.cpp").
This last solution is not really C++ standard way, but if you need it just in one place it should do the job.
For example, take a look how I did it:

copied exchange_state.cpp to the project
include it in the project file

@smlu Thanks for the tip! I used the #include directive and added the source path of that .cpp file using the include flag in the eosio-cpp tool. Compilation was successful 👍


creating an API resource from admin results in broken page

when there is a redirect API action defined
https://user-images.githubusercontent.com/35935196/185817355-4519293c-7dfb-4bf7-8a36-9d6772cf7ecf.mov

@Pralish
Working on another task has also fixed this bug. So, I have sent a PR addressing this issue


More efficient way to convert images from Images.jl to OpenCV Mat

As mentioned here, we can come up with a more efficient method to convert the images using c++ pointers method. I found that the wrappers are already implemented here, for manipulation using pointers in c++. But they seem to place the same int value over all the pixels. Maybe we need to come up with another wrapper to efficiently convert images ? I would like to work on this wrapper.

Sounds great.  Indeed, the reason for using the std container at method in convertToMat and generally in OpenCV.jl was initially to keep things simple.  However, as you correctly pointed out setgray and setcolor already demonstrates that using pointers is obviously feasible in this library. Indeed using GPU Mat (UMat) in addition can speed up image manipulation as well.
I suggest that you look into creating methods with templates so that it is easy to call the same function with any Type.  I believe that I tried looking into this early on, but there was no support for Templates in Cxx.jl. Have a look also at misc.jl in Cxx.jl.

@maxruby  I think we need to completely rewrite the src/OpenCV_Mat.jl to include templates.
How about I open a WIP PR ?

Sure, go ahead! If you have more specific questions, just let me know. I will do my best to answer as quickly as possible.

@kvmanohar22 .  You are welcome to have a look at WIP branch mat_templates.  It already supports conversion of JuliaImages 2d and 3d arrays to std::vector.  For 2d array images, conversion to cv::Mat_ is working fine (getters and setters), but RGB image conversion still need some small fixes. Suggestions are welcome.

I think getPixel and getPixel3d  should work for RGB images too ?

I think getPixel and getPixel3d should work for RGB images too ?

Indeed, I think both should already work.  Currently stdvec3Mat_ is not working, but it should be easy to fix.  The problem is in the inner assignment loop in stdvector3Mat_.
Other than that, there are a number of tasks to be done:

[ ] add tests for the new methods
[ ] add methods to convert cv::Mat_<T>back to Julia Image arrays

If you would like to work on these, feel free to fetch the mat_templates branch and then submit a new PR.
I will try to clean up the new methods by tonight.  There are a number of breaking changes brought as a result of changing the cv::Mat getters and setters.  I need to have a look more carefully when I have time.

If you would like to work on these

Yes, I would like to work on them.

There are a number of breaking changes

Yes this too, I tried testing the new implementation but got errors, specifically when I tried importing OpenCV with mat_templates branch checked out, this error popped up.

Yes this too, I tried testing the new implementation but got errors, specifically when I tried importing OpenCV with mat_templates branch checked out, this error popped up.

I assume this is an OS or setup-related error since I do not see it.
You mentioned in relation to this error that:

I had opencv 2.4.0 and opencv 3.0.0 installed previsouly and on top of this I installed 3.2.0.
This caused errors. Finally I removed all the versions of opencv (including the shared object files) and did a fresh install of opencv 3.2.0. And then as you mentioned I commented #include statements for flann in opencv.hpp and the problem was solved !

Which error specifically do you see in the REPL?

julia> using OpenCV
ERROR: LoadError: ReadOnlyMemoryError()
Stacktrace:
 [1] dlopen(::String, ::UInt32) at ./libdl.jl:97
 [2] macro expansion at /home/kv/.julia/v0.6/OpenCV/src/OpenCV.jl:75 [inlined]
 [3] anonymous at ./<missing>:?
 [4] include_from_node1(::String) at ./loading.jl:539
 [5] eval(::Module, ::Any) at ./boot.jl:235
 [6] require(::Symbol) at ./loading.jl:453
while loading /home/kv/.julia/v0.6/OpenCV/src/OpenCV.jl, in expression starting on line 70

julia>

Maybe related to your OpenCV installation again or something that you changed in your local repo but which is missing from the mat_templates branch?  I really do not know what could be going on since I can't reproduce it locally. . .

I have looked at the commit history and I have not made any changes to the lib loading routine in OpenCV.jl since the last commit d2851b46b6226a547706aaf256507a1a17cccd1f in the masterbranch.

I guess I have messed up again ;). I will look into this.


factorization-based preconditioner example

It would be nice to give an example how a preconditioner can be generated out of two operators - towards ILU preconditioners in the long run....

Related to #27. There's already a rough sketch there of how this would look like.


Continue ordered list on next slide

I need to do something like below. Unfortunately "Next slide" is starting new list.

---
# First slide
1. First point
2. Second point
3. Third point

---
# Next slide
4. Fourth point
5. Fifth point

For now, I suggest you use <ol start="x">
We did that for #334 and another one I think.


Tabs API isPreview and isPinned do not work

Testing #145585
I have only the welcome tab opened:
And it reports as isPreview: true even though it is not in preview.
Same actually for pinned state, the state is not updating when I change it. To clarify: I print the state from a onDidChangeTab event handler.

This is mystery to me. Could it be that pinned and preview is mixed?

Settings editor also reports as isPreview: true always.


AzurePowerShell@4/5 fails on Ubuntu agents, unable to find az module

Note
Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo
For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks
If you have an issue or request for the Azure Pipelines service, use developer community instead:
https://developercommunity.visualstudio.com/spaces/21/index.html )
Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name: AzurePowerShell
list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks
Environment

Server - Azure Pipelines

Agent - Hosted or Private:

If using Hosted agent, ubuntu

Issue Description
A simple task definition fails. Also on SO

https://stackoverflow.com/questions/55411029/azure-powershell-az-module-not-working-on-ubuntu-hosted-build-agent

  - task: AzurePowerShell@5
    displayName: 'ps test'
    inputs:
      azureSubscription: 'my-service-connection'
      scriptType: inlineScript
      inline: |
       echo 1

Task logs
2020-12-21T22:42:19.6182035Z ==============================================================================
2020-12-21T22:42:19.6183667Z Task         : Azure PowerShell
2020-12-21T22:42:19.6184830Z Description  : Run a PowerShell script within an Azure environment
2020-12-21T22:42:19.6185953Z Version      : 5.179.0
2020-12-21T22:42:19.6187075Z Author       : Microsoft Corporation
2020-12-21T22:42:19.6188185Z Help         : https://aka.ms/azurepowershelltroubleshooting
2020-12-21T22:42:19.6189335Z ==============================================================================
2020-12-21T22:42:19.8396013Z Generating script.
2020-12-21T22:42:19.8483852Z [command]/usr/bin/pwsh -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command . '/home/vsts/work/_temp/bb09c995-3e4a-4a7e-8a0c-e3dcddfb4dc7.ps1'
2020-12-21T22:42:21.4463616Z Exception: /home/vsts/work/_tasks/AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62/5.179.0/Utility.ps1:42
2020-12-21T22:42:21.4491106Z Line |
2020-12-21T22:42:21.4493479Z   42 |  â€¦             throw ("Could not find the module path with given version â€¦
2020-12-21T22:42:21.4495503Z      |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2020-12-21T22:42:21.4497111Z      | Could not find the module path with given version.
2020-12-21T22:42:21.4498555Z
2020-12-21T22:42:21.4756881Z ##[error]PowerShell exited with code '1'.

Troubleshooting
Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting
Error logs
[Insert error from the logs here for a quick overview]

@avishnyakov We are not able to repro the issue. I ran the task with latest version of az module on Hosted ubuntu agent. It is passing for me.
can you please let us know which az module version you are using ?

Closing this issue due to inactivity

Closing this issue due to inactivity


Add a method to create an empty Results

Goals
I have some table provider classes that conform to UITableViewDataSource/Delegate and are powered by Realm. These classes currently use a var objects: Results<MyObject>? variable to hold onto the list of objects. This list is provided by the VC before the table is displayed.
It's unfortunate that I need to use an optional here, but there is no way to provide a default value. If they were using a List instead, I could use var objects = List<MyObject>() and create a dummy list.
It would be nice if there were a method that made an empty Results, e.g., var objects = Results<MyObject>() (or whatever y'all think is best).
I could yolo and use an IUO, but that's Bad Swift™.
Expected Results
There's some way to generate a Results object without actually putting any data inside.
Notes
Maybe this is just a bad way to power tables off of Realm? What's the best practice here?

Using an IUO isn't bad swift, that's exactly the use case it's intended to serve.
If you really need an empty Results, you can do realm.objects(Model.self).filter("FALSEPREDICATE"). Unlike List and Object subclasses, Results can never be unmanaged.

Ah, hmm. Ok! An IUO it is then! Thanks!


Feature/sfn changes

removed empty fields from connection details
changed the matrix to express all the origins of the cluster clicked
changed the format to avoid the pdf table to be cut (this might require another change imho and go to a bigger format).

Looks okay, thanks @ddelpiano!
Suggestion: However, if we don't want the page to always be A0 size, I think one potential solution could be to check the number of columns and then assign the page size accordingly.

thanks @ddelpiano ! Looks good.


Fixed articulated hands being classified as HP Motion Controllers

Overview
Adds a check so controllers which are classified as articulated hands will not be named HP Motion Controllers
Changes

Fixes: #9403

/azp run

@CDiaz-MS can I get a quick confirm that these changes have the intended effect?


pdf渲染问题

输出的pdf打开后提示以下信息，且正文文字没有显示，使用的是vscode-pdf。但在外部pdf软件可以正常打开。
PDF.js v2.10.377 (build: 156762c48)
Message: Requesting object that isn't resolved yet pattern_p0_1.
Stack: Error: Requesting object that isn't resolved yet pattern_p0_1.
使用的是示例代码
#import "@preview/touying:0.5.3": *
#import themes.metropolis: *

#import "@preview/numbly:0.1.0": numbly

#show: metropolis-theme.with(
  aspect-ratio: "16-9",
  footer: self => self.info.institution,
  config-info(
    title: [Title],
    subtitle: [Subtitle],
    author: [Authors],
    date: datetime.today(),
    institution: [Institution],
    logo: emoji.city,
  ),
)

#set heading(numbering: numbly("{1}.", default: "1.1"))

#title-slide()

= Outline <touying:hidden>

#outline(title: none, indent: 1em, depth: 1)

= First Section

---

A slide without a title but with some *important* information.

== A long long long long long long long long long long long long long long long long long long long long long long long long Title

=== sdfsdf

A slide with equation:

$ x_(n+1) = (x_n + a/x_n) / 2 $

#lorem(200)

= Second Section

#focus-slide[
  Wake up!!
]

== Simple Animation

We can use `#pause` to #pause display something later.

#meanwhile

Meanwhile, #pause we can also use `#meanwhile` to display other content synchronously.

#speaker-note[
  + This is a speaker note.
  + You won't see it unless you use `config-common(show-notes-on-second-screen: right)`
]

#show: appendix

= Appendix

---

Please pay attention to the current slide number.

更换Pdf插件pdf-viewer后没问题了

You can use the preview feature of the tinymist extension directly and don't installing the typst-lsp extension.


Introduce ContentAgent with SeleniumBrowser for Enhanced Web Content Collection

This PR marks the first contribution towards developing the Athena agent, a pipeline engineered for aggregating, storing, and analyzing temporal events published over the web.  While the initial use case centers on monitoring the rapidly evolving field of Artificial Intelligence and its subdomains, the architecture and functionality of Athena are designed with a much broader application spectrum in mind. It aims to serve as a versatile tool for scientists, researchers, and the general community, facilitating the efficient tracking of any online-published temporal events across various domains.
The pipeline's utility will be a testament to the Autogen project as a whole and seeks to be a useful tool for anyone wanting to stay in touch with developments in their fields or to conduct comprehensive archival research on web content over time.
Key Contributions:

ContentAgent Implementation:
Introduces a dedicated agent for the web-based content collection and filesystem storage, setting the foundation for a rich archive of online temporal events.

SeleniumBrowser Function:
Adds a headless, fully-functional desktop web driver to enable dynamic interactions with web pages, crucial for accessing content reliant on JavaScript or other client-side scripts.

SeleniumBrowserWrapper Class:
Offers a seamless alternative to the SimpleTextBrowser, enhancing agent capabilities in interacting with web pages that require advanced browsing functionalities.

WebSurferAgent Enhancements:
This update enables the ability to select between SimpleTextBrowser or SeleniumBrowserWrapper based on the provided browser_config and opens the door to vision-based function calling and interactions, significantly broadening the scope of tasks and potential use cases for Autogen agents in web interaction scenarios going forward.

Unit Testing:
Extends unit tests to cover the new ContentAgent and the enhanced WebSurferAgent, ensuring both functionality and reliability in the agents' operations.

Motivation and Context
In an era where information moves at light speed, specifically in the double exponential progress of AI advancements, the challenge of staying updated with the latest developments becomes increasingly complex. The Athena agent pipeline, powered by Autogen and spearheaded the ContentAgent, seeks to automate the collection and archiving of web-based content, simplifying the process of tracking and analyzing online temporal events. The integration of Selenium for web interaction assists with the immediate needs of AI research tracking and also lays the groundwork for future enhancements, further enhancing Autogen as the scalable and adaptable solution for numerous research and informational needs.
Benefits

Broadens the scope of web-based content that can be collected
Facilitates future development of sophisticated agents capable of complex web-based vision tasks
Enhances the WebSurferAgent with configurable browser behavior

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checks

[x] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
[x] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[x] I've made sure all auto checks for ContentAgent have passed.
[ ] I've made sure all auto checks for WebSurferAgent have passed.
Everything except Bing Search was tested and passed.  Still needing to sign-up for a BING_API key

Known required updates or enhancements

An additional commit to requirements.txt is needed, perhaps with specifics to an installation type, because ContentAgent depends on beautifulsoup4, pdfminer, selenium, arxiv, and pillow
Updates for corner-cases are inevitable as ContentAgent is tested against more and more web-content

Of course, what is an Agent without their visualization? ❤

Codecov Report
Attention: 396 lines in your changes are missing coverage. Please review.

Comparison is base (2750391) 39.33% compared to head (3954412) 20.88%.

Files
Patch %
Lines

autogen/agentchat/contrib/content_agent.py
0.00%
208 Missing and 3 partials :warning:

autogen/browser_utils.py
4.76%
179 Missing and 1 partial :warning:

autogen/agentchat/contrib/web_surfer.py
44.44%
5 Missing :warning:

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1733       +/-   ##
===========================================
- Coverage   39.33%   20.88%   -18.46%
===========================================
  Files          57       58        +1
  Lines        6096     6502      +406
  Branches     1365     1564      +199
===========================================
- Hits         2398     1358     -1040
- Misses       3502     4974     +1472
+ Partials      196      170       -26

Flag
Coverage Δ

unittests
20.88% <3.17%> (-18.46%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@microsoft-github-policy-service agree
On Mon, Feb 19, 2024 at 10:24 PM Codecov Comments Bot <
@.***> wrote:

Codecov
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft
Report
Attention: 396 lines in your changes are missing coverage. Please review.
Comparison is base (2750391)
https://app.codecov.io/gh/microsoft/autogen/commit/2750391f847b7168d842dfcb815ac37bd94c9a0e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft
39.33% compared to head (3954412)
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft
21.53%.
Files
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft Patch
% Lines
autogen/agentchat/contrib/content_agent.py
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft#diff-YXV0b2dlbi9hZ2VudGNoYXQvY29udHJpYi9jb250ZW50X2FnZW50LnB5
0.00% 208 Missing and 3 partials ⚠
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft
autogen/browser_utils.py
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft#diff-YXV0b2dlbi9icm93c2VyX3V0aWxzLnB5
4.76% 179 Missing and 1 partial ⚠
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft
autogen/agentchat/contrib/web_surfer.py
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft#diff-YXV0b2dlbi9hZ2VudGNoYXQvY29udHJpYi93ZWJfc3VyZmVyLnB5
44.44% 5 Missing ⚠
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft Additional
details and impacted files
@@             Coverage Diff             @@##             main    #1733       +/-   ##
===========================================- Coverage   39.33%   21.53%   -17.81%
Files          57       58        +1
Lines        6096     6502      +406
Branches     1365     1564      +199
===========================================- Hits         2398     1400      -998     - Misses       3502     4932     +1430     + Partials      196      170       -26
Flag
https://app.codecov.io/gh/microsoft/autogen/pull/1733/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft Coverage
Δ
unittests
https://app.codecov.io/gh/microsoft/autogen/pull/1733/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft 21.53%
<3.17%> (-17.81%) ⬇
Flags with carried forward coverage won't be shown. Click here
https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft#carryforward-flags-in-the-pull-request-comment
to find out more.
☔ View full report in Codecov by Sentry
https://app.codecov.io/gh/microsoft/autogen/pull/1733?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft
.
📢 Have feedback on the report? Share it here
https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=microsoft
.
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/autogen/pull/1733#issuecomment-1953467303,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADWFCMFGKUPKH3OVLMHL573YUQQRDAVCNFSM6AAAAABDQOESUOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJTGQ3DOMZQGM
.
You are receiving this because you were mentioned.Message ID:
@.***>

This PR expands the websurfer extras requirements to include selenium.  I've also updated the workflow for python-package.yml to include the webdrivers for Edge, Firefox, and Chrome.
All pre-commit tests in the dev docker instance pass, with one exception due to no-commit-to-branch.  I realize after the fact that committing to a separate branch would have been preferred and was able to able to commit using SKIP=no-commit-to-branch git commit ....
Finally, regarding the build workflow, there are two pending issues:

It is still unclear how to inform the system that it should install the websurfer extra requirements for this PR
We're getting numerous PytestUnknownMarkWarning: Unknown pytest.mark.asyncio errors, however that markdown is used throughout and I wonder if it's an issue regarding the installed version of pytest.  As a side note in reporting, in the dev docker I had to upgrade pydantic to 2.5.3 when it couldn't find field_validator in a file unrelated to this PR

Updated workflows to include the browsers, I'll review again to ensure those are all configured
-- contrib-openai.yml
-- contrib-tests.yml
Updated setup.py so that websurfer includes selenium, requests, and arxiv packages
Added additional typing and docstrings to browser_utils.py
Updates to SeleniumWrapper to ensure 100% coverage and functionality seen in SimpleTextBrowser
Added notebook/agentchat_surfer_edge.ipynb to demonstrate cross compatibility as well as new graphical functions
Confirmed functionality with Bing API
Added notebook/agentchat_content_agent.ipynb to demonstrate core functionality
pre-commit updated unrelated notebooks notebook/agentchat_lmm_gpt-4v.ipynb and notebook/agentchat_custom_model.ipynb to my surprise, removal or addition of a empty line(s)

Thank you for the PR! In my opinion, it would be good to include some code examples in the notebook to demonstrate the advantages of the ContentAgent. From what I understand, the ContentAgent is a method for aggregating web information onto disk. However, I’m not sure whether the original assistant and userproxy agents could achieve the same goal through a meticulously designed prompt.

I’m not sure whether the original assistant and userproxy agents could achieve the same goal through a meticulously designed prompt.
Yes, I think you are correct that it could conceptually be handled strictly using existing agents and prompts.  I did it this way for two reasons.  The first being that this agent works as part of a larger pipeline that will require the archived content, and the second reason is from an energy conservation perspective.  Hopefully with recent developments it won't be a concern, but for now we ask if it's responsible to take the tractor (LLM inference) to pickup more cat food or just ride our bike (a scripted series of functions to archive the components).   That said, an LLM when done right could potentially handle the corner cases.

The WebSurfer upgrades are a byproduct from the requirements of the WebArchival (thanks @skzhang1) agent, which is a dependency of my next PR.  I could technically remove the Content or WebArchival Agent components from this PR.  I'm open to all ideas, especially what is best for Autogen as a whole.

Made some last updates based on the generous feedback from @skzhang1.  The only two other things I can think to include would be an update to the docs for the graphical WebSurferAgent and additional tests to test/test_browser_utils.py.
@sonichi thank you for running the OAI tests.  I noticed test_web_surfer.py failed with a NameError about WebSurferAgent being undefined.  That test file is unchanged in this PR, but the agent has small changes.  Looking into it we do see the agent class imported here with a catch to disable all tests in the case of failure.  Your thoughts?

I'd like to get @afourney 's opinion on the structure of the changes, e.g., whether it's ok to directly modify the current web surfer agent or should a separate agent be created. And whether a separate optional dependency is desired. My other comments may not apply if a structural change is needed.

Hi @afourney, I saw in an issue that you were thinking to pass a browser object to the agent.  Would you prefer that to the method I have configured?  I could expose an option to provide a browser object while maintaining the current behavior where the browser type is specified in the config to keep things simple for the avg end-user.  Your thoughts?

I don't know why @afourney chose not to reply here, but 2 weeks later began introducing new PRs without accepting mine first.  I'd suggested to accept my contribution and then do whatever he wants afterwards.  That appeared to clearly be the best way to proceed.

@signalprime i would like you to take a look at my PR maybe we can adopt one design as I’m working on a similar part.

This PR is against AutoGen 0.2. AutoGen 0.2 has been moved to the 0.2 branch. Please rebase your PR on the 0.2 branch or update it to work with the new AutoGen 0.4 that is now in main.

@signalprime unfortunately after rebase there are still some open conflicts. If you are still interested in bringing this one forward please see if you can get those resolved and green the latest CI.

closing as stale, please reopen if you would like to update


[EDIT: rq. impl isNumberMatch] Equivalent PhoneNumbers are not equal if the source of their country code is different

Because Eq is derived for PhoneNumber, this happens:
let full = "+1-800-555-1234".parse().unwrap();
let equivalent = `phonenumber::parse(Some(phonenumber::country::US), "800-555-1234").unwrap();

// false
full == equivalent;

phonenumber::country::Code stores the u16 as well as the phonenumber::country::Source that indicates where it came from. I can see how this may be useful for some aspects of a program's logic, but it makes straightforward equality checks of normalized PhoneNumbers awkward (requiring a helper function or a newtype).
I just wanted to draw attention to this. Maybe it's the desired behavior? Maybe this is how libphonenumber works? From my perspective, though, it violates the "Principle of Least Surprise".

As with everything else about phone numbers, matching is not trivial and has different levels of certainty and equivalence, therefore it can't be done by direct comparison. The original libphonenumber provides isNumberMatch for this purpose but we do not have a matching implementation in rust-phonenumber at the moment.

The way this phone numbers differ is in the encoding of the country code.
The fully parsed one uses the "plus"  +1.
The the other one uses the default encoding which depending on the context might or might not be +1.

I turned this into an enhancement request :wink:.
While I'm not sure we need to keep track of source in country code, which could fix your given example there are other examples which it would not fix.

Just ran into this, and it's really annoying. I have a bunch of manually written phone numbers, and my goal was to parse all of them and remove duplicates. As it turns out, some of them have the plus code, some don't... and despite being actually the same number, they are considered different.
This makes no sense; I see no reason at all why a type that describes a phone number should contain information regarding how it was parsed. If at least there were functions that would allow me to modify phone numbers this wouldn't be too much of an issue. too bad there are none.


DockerHub: Overview

Currently the overview on dockerhub is empty.  It would be nice for users if there was a helpful page on using the image.

Later.


Notification message when creating a site doesn't use Site Name field

Despite setting a site name, the notification message reads "Site 'localhost:8111:8111 [default]' created", which isn't ideal.
The port doesn't seem worth mentioning. The square brackets are a bit non-friendly. The lack of site name itself is the main thing though.

Fixed in #2144


YouTube Thumbnail Downloader

Project title
YouTube Thumbnail Downloader
Project Description
This web app allows you to download any youtube video's thumbnail.
Stack:
And in order to tick the check box just but x inside them for example - [x] like this. Please delete options that are not relevant.

[] Html
[] CSS
[] JavaScript

/assign


[Doc] [Do-not-Merge] Move schema compatibility strategy CMD from topics to topicsPolicies

Modifications
To align with #14225 , the following change has been made:

Change the pulsar-admin topics *-schema-compatibility-strategy command to the pulsar-admin topicsPolicies *-schema-compatibility-strategy command.

Doc Site Preview

Documentation

[x] doc
(If this PR contains doc changes)

Hi @nodece @Anonymitaet, can you please review this doc PR? Thanks.


[Screen Reader-Microbit Menu Controls] Control type is not defined for the controls in the Fly out of the micro bit menu items.

User Experience:
Users with Visual disabilities might not able to access the controls if control type is not defined. Like in the scenario Control type is not defined for the controls in the Fly out of the micro bit menu items.
Test Environment
OS: RS2 Version 1703(OS Build 15063.2)
Platform: Edge.
Screen reader: Narrator
Repro Steps
1-Navigate to url:- https://makecode.com/
2-Navigate to the Micro bit section element and select Code control given on it.
3-Navigate to the menu controls listed near the Simulator control for eg.Arrays
4-Navigate to the controls under micro bit menu item fly out that appears in the right and verify the issue
Actual Result
Control type is not defined for the controls in the Fly out of the micro bit menu items.
Expected Result
Control type should be defined for the controls in the Fly out of the micro bit menu items.
Note
This issue exists for all the controls lying in the fly outs of the micro bit menu items.
MAS Reference
https://microsoft.sharepoint.com/teams/msenable/Documents/LCA/MAS - July 1, 2017 Release/MAS 4.1.2 – Name, Role, Value.docx?d=w248054a65e6847719e1e242fb5025730&csf=1
Suggested fix:
1-   Using ARIA role to expose the control type of user interface component
e.g.

2-   Using ARIA attributes to expose the element’s state
e.g.

**Reference:**
https://www.w3.org/TR/WCAG20-TECHS/aria.html
https://www.w3.org/TR/WCAG20-TECHS/ARIA16.html
https://www.w3.org/TR/WCAG20-TECHS/ARIA4.html
https://www.w3.org/TR/WCAG20-TECHS/ARIA5.html
Refer Attachment

This is part of Blocky, a third-part component.

We have got exception for Blockly bugs from Julia Liuson (CVP). Closing these as Won't Fix. Following is the reasoning.

MakeCode can be used in either Text editor or Block editor.
Blocks editor is inherently visual and is based on google blockly. Usage requires drag and drop functionality and we don’t have a equivalence in MakeCode for this w.r.t accessibility.
We have evaluated couple of research projects which tries to find a equivalence to block based coding for accessibility needs:
•	Project Torino (Requires students to buy physical blocks)
•	Accessible blockly (This is text based coding without syntax errors)
Both these are drastic departure from the blockly and provides a complete new experience. These are still in research and we intend not to use them in MakeCode at this point of time.
As workaround, the text editor in MakeCode is the way we see students with the need of accessibility using MakeCode.


[Bug] 元数据Excel导出异常

Branch name (分支名)
main
Bug description (缺陷描述)
No response
Reproduction steps (复现步骤)
No response
cUrls and response (请求和响应)
No response

#1154


docs: update catalog error eventing/logging

Hey, I just made a Pull Request!
This updates the documentation to reflect the changes around catalog error logging. https://github.com/backstage/backstage/pull/23022
I wasn’t sure if adding this guide here made sense or not but I wasn’t quite sure the best place to put this content. i.e. should this section be updated and linked out to a guide.
I also feel like there is a much cleaner way (with generics) to define the shape of the catalog event. I was getting type errors when trying something like this:
    params: EventParams<{
      entity: string;
      location?: string;
      errors: Array<Error>;
    }>,

I also didn’t know if a simple module would help for users that want to turn these logs on rather than creating their own? I’m happy to contribute that!
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[x] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Thanks for this @codingdiaz, really appreciate it 🚀
Left a few comments, nothing major, I do think that maybe this should be somewhere else though. This page is great as an overview and I like that it's very focused on the specific topic only. Could we move this here: https://backstage.io/docs/features/software-catalog/configuration

Any time! I just updated the docs, I agree with you and I linked to the configuration section.

@freben / @ahhhndre. I’m getting some errors checking the API reports but locally I’m running yarn build:api-reports & then yarn build:api-reports:only --ci —docs & echo $ with an output of 0.
Am I missing something to get the api report checks passing in CI?

Funky - i think it needed a bit of a rebase and massage. Managed to generate the report properly now and pushed. Let's see how it goes.

UNLESS I PRESS THE WRONG BUTTON ACCIDENTALLY FIRST

@freben I’m all set with changes on this one (I don’t have the ability to merge).

@vinzscam looks like we need your approval here too ;)


Enhanced the font color of the Event Form

Fixes Issue🛠
Closes #1049
Description👨💻
Changed the font color to enhance the form.
Type of Change📄

[X] Style (non-breaking change which improves website style or formatting)

Checklist✅

[X] I am an Open Source contributor
[X] I have performed a self-review of my code
[X] My code follows the style guidelines of this project

Screenshots/GIF📷

You have just changed the button text color, pls make some good modifications @nidhipai78

@Rakesh9100 Changes made

I can't see any changes, make the enhancements and the new commits properly @nidhipai78


Animation object problem taking duration from easing and not from attrs

Animation object seems to fail on dur/ms expecting to come from easing.dur rather than attrs.dur

:+1:

:+1:

Merged by hand in 0.4.1


RuntimeError: The size of tensor a (3549) must match the size of tensor b (8400) at non-singleton dimension 1

when I run following commands,
model_path=model/yolox_nano.pth
model_name=yolox-nano
python tools/demo.py video -n $model_name -c $model_path --path data/cat.mp4 --conf 0.3 --nms 0.65 --tsize 640  --device gpu --save_result --trt

Matplotlib created a temporary config/cache directory at /tmp/matplotlib-7_4g5yxg because the default path (/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
2021-07-25 06:36:32.688 | INFO     | __main__:main:229 - Args: Namespace(camid=0, ckpt='model/yolox_nano.pth', conf=0.3, demo='video', device='gpu', exp_file=None, experiment_name='nano', fp16=False, fuse=False, name='yolox-nano', nms=0.65, path='data/cat.mp4', save_result=True, show_result=True, trt=True, tsize=640)
/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  ../c10/core/TensorImpl.h:1153.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
2021-07-25 06:36:32.837 | INFO     | __main__:main:239 - Model Summary: Params: 0.91M, Gflops: 2.55
2021-07-25 06:36:34.421 | INFO     | __main__:main:269 - Using TensorRT to inference
2021-07-25 06:36:34.561 | INFO     | __main__:__init__:100 - loading tensorrt file ./YOLOX_outputs/nano/model_trt.pth
2021-07-25 06:36:47.372 | INFO     | __main__:imageflow_demo:195 - video save_path is ./YOLOX_outputs/nano/vis_res/2021_07_25_06_36_47/cat.mp4
Traceback (most recent call last):
  File "tools/demo.py", line 286, in <module>
    main(exp, args)
  File "tools/demo.py", line 279, in main
    imageflow_demo(predictor, vis_folder, current_time, args)
  File "tools/demo.py", line 202, in imageflow_demo
    outputs, img_info = predictor.inference(frame)
  File "tools/demo.py", line 132, in inference
    outputs = self.decoder(outputs, dtype=outputs.type())
  File "/home/lyy/258G/code/YOLOX/yolox/models/yolo_head.py", line 231, in decode_outputs
    outputs[..., :2] = (outputs[..., :2] + grids) * strides
RuntimeError: The size of tensor a (3549) must match the size of tensor b (8400) at non-singleton dimension 1

the default sizes of yolox-nano and its trt model are 416x416. So you may change --tsize 640 to --tsize 416

@ruinmessi  thanks for your reply. your comment works.


EC:<IP_ADDRESS> is obsolete

EC:<IP_ADDRESS> associated with the following GO term is obsolete:
id: GO:0050072
name: m7G(5')pppN diphosphatase activity
namespace: molecular_function
def: "Catalysis of the reaction: 7-methylguanosine 5'-triphospho-5'-polynucleotide + H2O = 7-methylguanosine 5'-phosphate + polynucleotide." [EC:<IP_ADDRESS>, MetaCyc:M7G5PPPN-PYROPHOSPHATASE-RXN]
synonym: "7-methylguanosine-5'-triphospho-5'-polynucleotide 7-methylguanosine-5'-phosphohydrolase activity" RELATED [EC:<IP_ADDRESS>]
synonym: "decapase activity" RELATED [EC:<IP_ADDRESS>]
synonym: "M(7)G(5')pppN pyrophosphatase activity" EXACT []
synonym: "m7G(5')pppN pyrophosphatase activity" RELATED [EC:<IP_ADDRESS>]
xref: EC:<IP_ADDRESS>
xref: MetaCyc:M7G5PPPN-PYROPHOSPHATASE-RXN
is_a: GO:0016462 ! pyrophosphatase activity
(this is mentioned in #17844 but it hasn't been fixed yet)
EC (https://enzyme.expasy.org/EC/<IP_ADDRESS>) says EC:<IP_ADDRESS> has been transferred to 2 entries:
<IP_ADDRESS>:
NAME: 5'-(N(7)-methyl 5'-triphosphoguanosine)-[mRNA] diphosphatase.
REACTION: A 5'-(N(7)-methyl 5'-triphosphoguanosine)-[mRNA] + H(2)O <=> N(7)-methylguanosine 5'-phosphate + a 5'-diphospho-[mRNA]
=> This is RHEA 65384 / 65388
<IP_ADDRESS>:
NAME: 5'-(N(7)-methylguanosine 5'-triphospho)-[mRNA] hydrolase.
REACTION: 5'-(N(7)-methylguanosine 5'-triphospho)-[mRNA] + H(2)O <=> N(7)-methylguanosine 5'-diphosphate + 5'-phospho-[mRNA]
=> this is RHEA:67484
Neither of the two replacement EC numbers (or their corresponding RHEAs) appear to be in the current GO.
I'm not sure of the appropriate fix here.... @hdrabkin ??

The 2 EC correspond to the specificity of the enzymes. We should create the 2 activities, obsolete GO:0050072 and add 'consider' the two activities.
Thanks, Pascale

+[Term]
+id: GO:0140932
+name: 5'-(N(7)-methyl 5'-triphosphoguanosine)-[mRNA] diphosphatase activity
+def: "Catalysis of the reaction: a 5'-end (N2,N2,N7-trimethyl 5'-triphosphoguanosine)-ribonucleoside in mRNA + H2O = a 5'-end diphospho-ribonucleoside in mRNA + 2 H+ + N7-methyl-GMP. Can also use (N7-methyl 5'-triphosphoguanosine)-ribonucleoside in mRNA as a substrate." [PMID:24742626, PMID:32723815]
+synonym: "m(7)GpppN m(7)GMP phosphohydrolase activity" EXACT [EC:<IP_ADDRESS>]
+synonym: "m(7)GpppX diphosphatase activity" EXACT [EC:<IP_ADDRESS>]
+synonym: "m(7)GpppX pyrophosphatase activity" EXACT [EC:<IP_ADDRESS>]
+xref: EC:<IP_ADDRESS> {source="skos:exactMatch"}
+xref: RHEA:65384 {source="skos:narrowMatch"}
+xref: RHEA:65388 {source="skos:narrowMatch"}
+is_a: GO:0034353 ! mRNA 5'-diphosphatase activity
+created_by: pg
+creation_date: 2022-10-24T15:37:32Z
+
+[Term]
+id: GO:0140933
+name: 5'-(N(7)-methylguanosine 5'-triphospho)-[mRNA] hydrolase activity
+def: "Catalysis of the reaction:  a 5'-end (N7-methyl 5'-triphosphoguanosine)-ribonucleoside in mRNA + H2O = a 5'-end phospho-ribonucleoside in mRNA +  N7-methyl-GDP + H+." [PMID:18820299, PMID:29483298]
+xref: EC:<IP_ADDRESS>
+xref: RHEA:67484
+is_a: GO:0140818 ! mRNA 5'-phosphatase activity
+property_value: term_tracker_item https://github.com/geneontology/go-ontology/issues/23290 xsd:anyURI
+created_by: pg
+creation_date: 2022-10-24T15:48:42Z

[ ] Announce obsoletion
[ ] Create annotation review ticket

@hdrabkin does that seem reasonable to you?

@pgaudet these look fine.
H

what about
GO:1990521 | m7G(5')pppN diphosphatase activator activity ?

@ValWood ; can you expand comment?

you are obsoleting
GO:0050072 name: m7G(5')pppN diphosphatase activity
what about
GO:1990521 | m7G(5')pppN diphosphatase activator activity
(I would like to do both at the same time, as we need to also fix all FYPO ids that use in logical defes too)

It would make sense then to obsolete the activator activity if the activity that is being activiated is obsolete.

Dear all,
The proposal has been made to obsolete
GO:0050072 m7G(5')pppN diphosphatase activity
GO:1990521  m7G(5')pppN diphosphatase activator activity
GO:1903398 regulation of m7G(5')pppN diphosphatase activity
GO:1903399 positive regulation of m7G(5')pppN diphosphatase activity
The reason for obsoletion is that the corresponding EC term has been obsoleted. The term has been replaced by two activities:

GO:0140932 5'-(N(7)-methyl 5'-triphosphoguanosine)-[mRNA] diphosphatase activity
GO:0140933  5'-(N(7)-methylguanosine 5'-triphospho)-[mRNA] hydrolase activity

There are EXP annotations by GeneBD, PomBase, WormBase, and one mapping by InterPro. This term is not present in any subsets.
You can comment on the ticket:
https://github.com/geneontology/go-ontology/issues/23290
Thanks, Pascale

Which term replac es
GO:1990521 m7G(5')pppN diphosphatase activator activity
thanks
Val

I thought we would create a regulator activity if the regulator is specifically an activator for a specific activity.  At least we have until now?
For sure we would need a more specific term than 'phosphatase activator activity', otherwise it will look a bit weird, as though we could not annotate any more specifically, even though there are children.
mRNA phosphatase activator activity could work,  because at the moment this is the only one  I have, but what happens if there are other mRNA phosphatase activators with different specificities (i.e. not for the  mRNA guanine cap), then we can't distinguish them by MF even though we know they have different activities.


GraphQLWsProtocol incorrectly parses error messages

Summary
I'm using Hasura and sending queries/mutations/subscriptions over the WebSocketNetworkTransport using the GraphQLWsProtocol.
According to the spec, it appears that the error message's payload should be of GraphQLError[];.
It looks like the code in here
      "error" -> listener.operationError(messageMap["id"] as String, messageMap["payload"] as Map<String, Any?>)

It trying to cast the payload to a map when it should be casting it to an array of maps.
Example error payload from Hasura:
{"type":"error","id":"eb14b901-c6d2-4144-b14d-c8ef64855ea3","payload":[{"extensions":{"code":"validation-failed","path":"$.selectionSet.extensionEndpointMappingsBulkUpdate.args.extensionEndpointMappingsBulkUpdateInput"},"message":"variable 'input' is declared as 'ExtensionEndpointMappingsBulkUpdateInput', but used where 'ExtensionEndpointMappingsBulkUpdateInput!' is expected"}]}

Version
Apollo Runtime: 3.7.1
Logs
Error generated from Apollo Kotlin
com.apollographql.apollo3.exception.ApolloNetworkException: Network error while executing ExtensionEndpointMappingsBulkUpdate
	at com.apollographql.apollo3.network.ws.WebSocketNetworkTransport$execute$$inlined$map$1$2.emit(Emitters.kt:246)
	at kotlinx.coroutines.flow.internal.SafeCollectorKt$emitFun$1.invoke(SafeCollector.kt:15)
	at kotlinx.coroutines.flow.internal.SafeCollectorKt$emitFun$1.invoke(SafeCollector.kt:15)
	at kotlinx.coroutines.flow.internal.SafeCollector.emit(SafeCollector.kt:87)
	at kotlinx.coroutines.flow.internal.SafeCollector.emit(SafeCollector.kt:66)
	at com.apollographql.apollo3.network.ws.WebSocketNetworkTransport$execute$3.invokeSuspend(WebSocketNetworkTransport.kt:271)
	at com.apollographql.apollo3.network.ws.WebSocketNetworkTransport$execute$3.invoke(WebSocketNetworkTransport.kt)
	at com.apollographql.apollo3.network.ws.WebSocketNetworkTransport$execute$3.invoke(WebSocketNetworkTransport.kt)
	at com.apollographql.apollo3.internal.FlowsKt$transformWhile$1$invokeSuspend$$inlined$collectWhile$1.emit(flows.kt:53)
	at com.apollographql.apollo3.internal.FlowsKt$transformWhile$1$invokeSuspend$$inlined$collectWhile$2.emit(flows.kt:38)
	at com.apollographql.apollo3.network.ws.WebSocketNetworkTransport$execute$$inlined$filter$1$2.emit(Emitters.kt:223)
	at kotlinx.coroutines.flow.SubscribedFlowCollector.emit(Share.kt)
	at kotlinx.coroutines.flow.SharedFlowImpl.collect$suspendImpl(SharedFlow.kt:383)
	at kotlinx.coroutines.flow.SharedFlowImpl$collect$1.invokeSuspend(SharedFlow.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be cast to class java.util.Map (java.util.ArrayList and java.util.Map are in module java.base of loader 'bootstrap')
	at com.apollographql.apollo3.network.ws.GraphQLWsProtocol.handleServerMessage(GraphQLWsProtocol.kt:120)
	at com.apollographql.apollo3.network.ws.WsProtocol.run$suspendImpl(WsProtocol.kt:144)
	at com.apollographql.apollo3.network.ws.WsProtocol$run$1.invokeSuspend(WsProtocol.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

Steps to reproduce the behavior
Issue any invalid query the server wouldn't recognize over a WebSocketNetworkTransport using the GraphQLWsProtocol.

Good catch 👍 . Not sure why this hasn't been an issue so far. I guess errors at this level are not very frequent.
Interestingly, these errors are passed down to the call site as Any? and just removing the cast should make everything work. Do you want to give it a shot? If not, no worries, I'll make a patch, just let me know!

Fix in #4540. Turns out this has not been an issue so far because the error you had should actually have been caught during codegen. Trying to use  a nullable input in a non-nullable position should have triggered a compile-time error.
If you can share your schema and query, I'd be very curious to see if we're missing a validation rule there. Or maybe your client schema is a bit out of sync with your server schema. That's also another possibility.
In all cases, you're 100% right and the errors shouldn't be cast to a Map. This part should be fixed now and be part of the next release

@martinbonnin , yeah originally it was because we had an under active development/fast evolving GQL API that got out of sync but it's also because we're trying to use this library in Clojure which we're basically just abandoning all type safety/code gen features this library gives so this is bound to happen to us. Thanks for the quick fix!

This is now avilable in 3.7.2


chore: carry metrics in flight metadata from datanode to frontend

I hereby agree to the terms of the GreptimeDB CLA
What's changed and what's your intention?
This pr mainly supports carrying metrics from datanode to frontend using metrics in flight metadata.
We use RecordBatchMetrics in json format for now. A better form is expected after https://github.com/GreptimeTeam/greptimedb/issues/2374
discussion: is it possible to reduce the number of stream wrapper?
Checklist

[x]  I have written the necessary rustdoc comments.
[ ]  I have added the necessary unit tests and integration tests.
[x]  This PR does not require documentation updates.

Refer to a related PR or issue link (optional)

cc @NiwakaDev you might be interested in this

Why not put the metrics in app_metadata in message FlightData?
https://arrow.apache.org/docs/format/Flight.html#protocol-buffer-definitions

Why not put the metrics in app_metadata in message FlightData?
https://arrow.apache.org/docs/format/Flight.html#protocol-buffer-definitions

            FlightMessage::Metrics(s) => {
                let metadata = FlightMetadata {
                    affected_rows: None,
                    metrics: Some(Metrics {
                        metrics: s.as_bytes().to_vec(),
                    }),
                }
                .encode_to_vec();
                FlightData {
                    flight_descriptor: None,
                    data_header: build_none_flight_msg().into(),
                    app_metadata: metadata.into(),
                    data_body: ProstBytes::default(),
                }
            }

It's now put in FlightMetadata next to affected_rows, which is later put in app_metadata. It is sent in a separate package at the end of a record batch stream since we need to finish executing plan to collect metrics.

@shuiyisong Are the metrics mainly for the Frontend to deal with RCUs? If so, is it possible to break the whole metrics into each recordbatch's metadata, instead of put them at end of a recordbatch stream? This comes in two favor:

Obvious storage place for metrics -- we can simply add a field in Recordbatch to store the metrics.
We can accumulate the metrics on the fly. Should there be a big query (e.g. select * from a table with 1 million rows) that would exceed the RCUs, we can stop it in the middle, no need to wait for the last metric message.

Are the metrics mainly for the Frontend to deal with RCUs?

No. This feature aims to provide a way to pass execution metrics through a distributed plan tree. A simple extension over ExecutionPlan::metrics.

Obvious storage place for metrics -- we can simply add a field in Recordbatch to store the metrics.

As said above, this is derived from ExecutionPlan::metrics which gives the plan-level execution metrics. And obviously, this is not available before a plan is finished.

We can accumulate the metrics on the fly. Should there be a big query (e.g. select * from a table with 1 million rows) that would exceed the RCUs, we can stop it in the middle, no need to wait for the last metric message.

I doubt if this is viable. Do we or Can we have such a real-time CU administration system?


Simplify access to library documentation

For users, access to the documentation of a library is not simple enough. See this comment from Reddit:

I need to go
latest release
documentation
blob of text with me having to find an inline link in section 4, libraries.
unordered
automatic redirect failed
or
learn
intro? no
getting started? no
explore the content? no
literally any of these links is useless. completely.
find libraries on top
unordered, but in a hard to read list
documentation
automatic redirect failed

A suggestion to fix this issue would be to reserve the top section of the main page to the most popular use cases, like downloads and documentation, and put "why use boost", "mission" and other information that will be read just by new users in a less visible part (.e.g. lower in the page or in a different page).
The documentation link can be put nearly at the same level as the current "Download the latest release" that says "Documentation for the latest release" that would redirect to something like "https://www.boost.io/doc/libs/boost_1_84_0/libs/libraries.htm". Alternativelly, we can have a drop-down or similar menu in the front page instead of the "Documentation for the latest release" button where a user selects a library (e.g. Unordered) and the selection redirects to the documentation of the latest release of Unordered.

Feedback, which has always annoyed me on the old page as well: As a user, I don't care about the Boost mission, I want
the download

That's available on the home page.

the documentation

That's on the top already as "Learn". Maybe that's not very intuitive.

the repo

The link is available on the home page but I think incentivizing people to download the release instead of going to the super-project repo is intentional.

To get to the (randomly picked) Boost.Unordered documentation for the latest version, I need to go
latest release, documentation, blob of text with me having to find an inline link in section 4, libraries., unordered, automatic redirect failed

So, he didn't really "need to" do that. He could go "Libraries" -> "Unordered" -> "Documentation". Whether the website is making this less intuitive than the alternative is something to discuss. My guess is he was already at the download page and tried to get to unordered from there.
He mentions that later in

find libraries on top -> unordered, but in a hard to read list

I don't know if it's easy to make that list easier to read because the list is just too long. It would be nice to have a small button to the docs in each card but I think this is already proposed in another issue.

For the repo: latest release -> source code -> code tab

I didn't know where the link to the repo was. I just went ctrl+f "github" and found it in the home page. Since it's there AND also in the latest release page (where he didn't  need to go), I think that's fine.
At last, the way he ends his comment is catastrophizing too much in my opinion.

Yes, we don't need to take all comments seriously. Just that below the "Download the Latest Release" button we could put some shortcuts to the most used features like Documentation for the latest version or the "https://github.com/boostorg" link where all Boost code is hosted.

I am very much in favor of using site logs to analyze where people go the most and optimizing the interface for that


How to Container#run detached and return immediately?

Currently, the documentation provides a single example of ::Docker::Container#run with a required argument. I've figured out that you can pass nil to use the default RUN command, but #run(..) is always a blocking call. How do you use ::Docker::Container#run while also specifying that it should start the container as completely detached and return immediately to the caller?

Also note that using #exec with an empty or nil command argument array doesn't provide a workaround:
(byebug) container.exec(nil, detach: true)
*** Docker::Error::ServerError Exception: No exec command specified

(byebug) container.exec([], detach: true)
*** Docker::Error::ServerError Exception: No exec command specified

Any hints here? It's pretty frustrating that this call always blocks. :(

+1

@preston
Container#run starts the current container, commits it, then runs the command in the resulting image that was committed.
Is this what you're trying to do?
If not, try taking a look at Docker::Container.create which given an image, you can pass an empty command (if a default exists), and with the resulting container call start which does not block.
It seems like your current setup is blocking because Container#run is calling start on the current container and waiting for it: https://github.com/swipely/docker-api/blob/master/lib/docker/container.rb#L41-L43


feature: skip verifying builtin extensions cache for improved performance

fixes #192632

@SimonSiefke Thanks for the PR.
Changes required to implement the requested feature are not trivial. It would be easier, if it is implemented by the core team. Hence I am closing this PR.


<EMAIL_ADDRESS>

          >

DO-IRPV3.0--2022-06-30.pdf
trialagreement.pdf
export.csv
Comparison Sheet (2).pdf
Comparison Sheet (1).pdf
`
Comparison Sheet.pdf
documents-in-tsv.csv
pages-in-tsv.csv
rfc4844.txt.pdf
Originally posted by @OATEF4313 in https://github.com/devcontainers/cli/pull/242#pullrequestreview-2334817993

https://github.com/devcontainers/cli/issues/905#issue-2553894380


Can't disconnect Outline from IOS settings

Describe the bug
When you disconnect Outline VPN from settings it disconnects but 2 seconds later it automatically reconnects. Many users disconnect VPNs from their settings, but Outline seems to have bug with it. Maybe Outline App thinks that this disconnect was not intentional and tries to reconnect.
To reproduce

Connect to Outline server
Go to settings
Go to VPN
Press Disconnect

Expected behavior
VPN should be disconnected
Screenshots

Client system (please complete the following information):

Outline Client Version: 1.11.0
Your Operating System: IOS16.3.1

Thanks for reaching out. Closing this as a duplicate of #1444.


Update zcash_script dependency with the latest Orchard module structure

Motivation
This PR updates to the zcash_script version in https://github.com/ZcashFoundation/zcash_script/pull/27
It's part of ticket #2982.

We can re-open or re-do this PR when ticket #2982 is actually scheduled.


sjaehn plugins: use lowercase pname

As mentioned here: https://github.com/NixOS/nixpkgs/pull/123940#pullrequestreview-671700096

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 125766 at d71cc1b1 run on aarch64-linux 1

4 packages built successfully:

bjumblr
bsequencer
bshapr
bslizr