Adding support for Word 2013 and 2016

Added the new namespaces from 2013 and 2016, not exactly sure if the tags on W15 and W16se, but it solved my problem as described on #102

Hi @ManoShu, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.microsoft.com.

TTYL, MSBOT;

Hi, I am closing and re-opening this PR to bump the CLA bot. Sorry for the inconvenience!

Hi @ManoShu, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

I added the readonly attribute to the fields. Is that all?

Hi @ManoShu, sorry for the late reply. I would not have anything else. Let's see what @EricWhiteDev has to say.

@ManoShu and @EricWhiteDev, looking at this PR again, I saw that a few namespaces that you would find in Microsoft Word 2016 documents are not included in the lists of namespaces in MarkupSimplifier.cs and DocumentBuilder.cs as amended by this PR (and certainly not before).
Here's the w:document tag as produced by Word 2016 with markup added by myself to point out the differences that I saw:
<w:document
  xmlns:wpc="http://schemas.microsoft.com/office/word/2010/wordprocessingCanvas"
  xmlns:cx="http://schemas.microsoft.com/office/drawing/2014/chartex"          <== MISSING
  xmlns:cx1="http://schemas.microsoft.com/office/drawing/2015/9/8/chartex"     <== MISSING
  xmlns:cx2="http://schemas.microsoft.com/office/drawing/2015/10/21/chartex"   <== MISSING
  xmlns:cx3="http://schemas.microsoft.com/office/drawing/2016/5/9/chartex"     <== MISSING
  xmlns:cx4="http://schemas.microsoft.com/office/drawing/2016/5/10/chartex"    <== MISSING
  xmlns:cx5="http://schemas.microsoft.com/office/drawing/2016/5/11/chartex"    <== MISSING
  xmlns:cx6="http://schemas.microsoft.com/office/drawing/2016/5/12/chartex"    <== MISSING
  xmlns:cx7="http://schemas.microsoft.com/office/drawing/2016/5/13/chartex"    <== MISSING
  xmlns:cx8="http://schemas.microsoft.com/office/drawing/2016/5/14/chartex"    <== MISSING
  xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
  xmlns:aink="http://schemas.microsoft.com/office/drawing/2016/ink"
  xmlns:am3d="http://schemas.microsoft.com/office/drawing/2017/model3d"
  xmlns:o="urn:schemas-microsoft-com:office:office"
  xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"
  xmlns:m="http://schemas.openxmlformats.org/officeDocument/2006/math"
  xmlns:v="urn:schemas-microsoft-com:vml"
  xmlns:wp14="http://schemas.microsoft.com/office/word/2010/wordprocessingDrawing"
  xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing"
  xmlns:w10="urn:schemas-microsoft-com:office:word"
  xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"
  xmlns:w14="http://schemas.microsoft.com/office/word/2010/wordml"
  xmlns:w15="http://schemas.microsoft.com/office/word/2012/wordml"
  xmlns:w16cid="http://schemas.microsoft.com/office/word/2016/wordml/cid"      <== MISSING
  xmlns:w16se="http://schemas.microsoft.com/office/word/2015/wordml/symex"
  xmlns:wpg="http://schemas.microsoft.com/office/word/2010/wordprocessingGroup"
  xmlns:wpi="http://schemas.microsoft.com/office/word/2010/wordprocessingInk"
  xmlns:wne="http://schemas.microsoft.com/office/word/2006/wordml"
  xmlns:wps="http://schemas.microsoft.com/office/word/2010/wordprocessingShape"
  mc:Ignorable="w14 w15 w16se w16cid wp14">                                    <== DIFFERENT

For example, if I understand the code in MarkupSimplifier correctly, it uses the namespaces defined there  for the w:document element if it is asked to normalize the XML. This would mean that Word 2016 documents would have the missing namespaces removed from the w:document element. If this is a problem, this means that this PR should be further amended to include the missing namespaces. Further, it means that this PR is also important, because as it currently stands, none of the namespaces added with this PR are currently taken into account.

I added the namespaces which were pointed out.
Also, I just found out about this definition page for the DOCX schemas here; It is necessary to add those elements as well?

@ManoShu I don't have any information regarding those namespaces.  What are the namespace prefixes for those namespaces - can you research?

@ManoShu Thanks for updating the pull request.
I would ask @EricWhiteDev to look into this, because some changes relate to classes (e.g., WmlComparer) that he currently actively develops. However, from my point of view, there would be at least two things that we should do (one of which will have to be synchronized with @EricWhiteDev's activities.
Firstly, @ManoShu should rebase this pull request on vNext. We currently have conflicts that need to be resolved. Those are coming from the recent upgrade to v2.7.2 of the Open XML SDK. And as you have not added any files to any project, I would not expect to see any changes to the .csproj files once you've rebased. On the other hand, that creates the question of whether or not you should have added unit tests and/or test documents ;-).
Secondly, we currently have three classes (DocumentBuilder, MarkupSimplifier, and WmlComparer) in which arrays like this are declared:
        private static XAttribute[] NamespaceAttributes =
        {
            new XAttribute(XNamespace.Xmlns + "wpc", WPC.wpc),
            new XAttribute(XNamespace.Xmlns + "mc", MC.mc),
            new XAttribute(XNamespace.Xmlns + "o", O.o),
            [...]
        };

I understand that @ManoShu has just worked with what was in the code base. However, if we amend it now, it makes sense to extract this and put it in some central place (otherwise the canonical unit test would be to ensure these arrays are all equal ;-). This obviously needs to be synchronized with @EricWhiteDev's activities.

@ManoShu @ThomasBarnekow At the exact moment, I have no changes that need to be merged into vNext.  It is up to date with all development that I had been doing in branches in other repos.  Therefore this is an ideal time to update this.  It would make sense to put this array in the PtOpenXmlUtil.cs module, and reference it from other modules.

@EricWhiteDev If you've run out of changes that are ready to be merged, there would be my PR, which fixes the RevisionProcessor ;-). However, I think I'd have to rebase that as well. So, depending on how quickly @ManoShu can rebase his PR and refactor as just discussed, you could merge his PR first.

@ThomasBarnekow OK, I'm happy to do either, as you recommend.  I will probably want to do some extensive tests on the RevisionProcessor - I have a large number of tests in a private repo (must be private because contains documents that are proprietary to a customer), so will want to run those tests.  But it makes sense to:

If @ManoShu can rebase and refactor, then merge that PR
You rebase the RevisionProcessor PR
I do my extensive tests
Merge the RevisionProcessor PR

OK, I will run my tests.

@ManoShu We have already tested and merged the RevisionProcessor PR - so if you can refactor, and rebase, we can merge this PR.  Can you move that array of namespaces into PtOpenXmlUtil.cs?

@EricWhiteDev OK, rebase was done. here is what i changed:

Concentrated the XAttribute arrays (a common list and a Power Tools exclusive) on a new NamespaceAttributeUtil class on PtOpenXmlUtil.cs;
Replaced any string-based namespace declaration to XElement manipulation;
Kept the array declarations on the previous spots, now just referencing the NamespaceAttributeUtil lists;

Sorry for not adding additional tests and documents, all the 2016 DOCX files I have are from customers.

@ManoShu Not sure how exactly you rebased, but I am seeing 92 commits and 469 changed files in this pull request. We definitely can't merge it in this state.

@ThomasBarnekow Sorry, I have done something wrong about this rebase, i used this list of commands, but instead of using git merge, I used git rebase.
I will rollback those commits and make it the proper way, and again sorry, my usual set of git commands used is very basic.

@ManoShu This is always easier if you have made your changes on a feature branch. You used vNext directly, which makes it a bit harder to do right.
Normally, if you use a feature branch, say support-word-2016, you would simply make sure your origin vNext is in sync with upstream vNext and then rebase your feature branch onto vNext. For example:
git checkout vNext
git pull upstream vNext
git rebase vNext support-word-2016
git push --force origin

If you are working on vNext directly, which is not the recommended practice, the following might work. But note that I have not tried this and you should confirm that it will do the right thing:
git pull --rebase upstream vNext
git push --force origin

@ThomasBarnekow Yes i was using the vNext branch to do the changes.
Once again I am sorry, but while trying to resolve the conflicts to rebase from upstream, my branch is now empty. It is worth it trying to recover it or just re-fork and do the right way? I am assuming that re-forking will make this PR null if I do that?

@ManoShu I think it does make sense to start from scratch, at least more or less.
I don't know what exactly your local history looks like, but to restart you could either:

create a completely new fork (as you suggested) or
do a hard reset of your local vNext branch to a well-defined commit (that you see in this repo's history), do a git pull upstream vNext (where upstream is this repository), and lastly do a git push origin vNext (where origin is your fork).

You should obviously store away your relevant source code files before doing any of the above.
With a new fork, you would likely have to create a new pull request (on a feature branch, please). That's not a problem, though. You would just close this one and we would continue our discussion on the other one.

Obligatory XKCD reference:
https://xkcd.com/1597/

@EricWhiteDev Yeah, I sure need to git gud, as some say.
@ThomasBarnekow I reforked and redid my changes, however a single test (OxPt.RpTests.RP001(name: "RP/RP052-Deleted-Para-Mark.docx")) failed.

@ManoShu I struggled (and still struggle) with git, as @ThomasBarnekow can attest to.  I am still ashamed of the commit history that is there in the early parts of the Open-Xml-Sdk.  I made a mess out of it.  Oh well.  Open-Xml-PowerTools is somewhat cleaner, but not perfect.
I can't count the number of times that I have messed up my repo and ended up re-forking / re-cloning, to clean it up.  I am still clueless about squashing commits.
I have just created a PR for fixing the single test that fails.  It will not conflict with your PR - you can create it.

@ManoShu Yes, as @EricWhiteDev pointed out, he already created a PR to fix the failing unit test, which is totally unrelated to your PR.
I would just want to make sure everything is nice and clean the next time. Thus, I assume you have created a feature branch off of the latest upstream (this repo's) vNext. Right? If so, when you push, your commits to origin (i.e., your fork) you can close this PR and then open a new one.
Now, my next question is about whether or not we need unit tests for this PR. You will obviously want to make sure that some changed output or behavior can be observed, now and in the future. That should be covered by unit tests.

OK, the push was done. Closing this PR.


Add enqueue library

Message Queue packages
Features:

Feature rich.
JMS like transport abstraction.
Supports  transports AMQP (RabbitMQ, ActiveMQ), STOMP, Amazon SQS, Redis, Doctrine DBAL, Filesystem, Null.
Symfony bundle
Magento1 extension
Message bus support.
RPC over MQ support.
Temporary queues support.
Well designed components decoupled and reusable.
Carefully tested including unit and functional tests.
For more visit quick tour.

Hi Everyone.
If there anything else I can do to participate in your great list?


Building and Run Unikernels with OPS

Topic Suggestion
Building and Run Unikernels with OPS
Proposal Submission
Proposed title of article
Building and Run Unikernels with OPS
Proposed article introduction
Because of their size, security, and performance, unikernels have long been hailed as the next generation of cloud architecture, allowing megatrends like microservices and serverless to thrive while combatting the never-ending barrage of data breaches, cryptojacking, and other issues. However, there's a major issue. Developers have been slow to respond. Some outside the ecosystem have blamed bogus issues such as a lack of debugging, but this isn't the case. For example, gdb is frequently used by everyone I know that uses unikernels.
Key takeaways
1 Introduction to OPS
2 Code, build and run Unkernel
References
N/A
Templates to use as guides

How To Guide Template
Software Review Template
Tutorial Template

Good topic @Gladysoby, please make sure your topic is unique and adds value to what is available online.
This is to help avoid article closure in the PR section 👍.


Hide FormatMail implementation

Why not, but it seems to me (now that we've picked up a few more sticks off the top) that MailAction wants to own this implementation. Observe the pattern - we're always instantiating MailAction with the result of calling FormatMail (via various format_mail methods which do not take consistent parameters).
From the perspective of client code we might as well cut the middleman and pass to MailAction's constructor whatever we're passing to format_mail - a mail template filename, a recipient list, and a templating context. This will make the calls more consistent. Killing two birds with one stone, that will also get rid of the smell in MailAction where it has to reach into the innards of a hash structure. Instead, MailAction can instantiate FormatMailand ask it for subject and recipients.
Let me know if you'd like to tackle this or if you'd rather I did.

Sounds good, go ahead 🙂.

Sorry for my laconic previous message. Long version ahead:
I'm really happy about the work we've done the last few weeks. We've together let emerge, in a pragmatic way,  some patterns by refactoring rigorously. I'm convinced that's a very good way to build software (I can't say the best, as I can't either we've applied "best practices", for there's always better: there's always space for improvement, and what's best can only come from within and not imported from the outside).
That being said, I've confidence in the fact we're arriving to a state where we can feel responsibilities are better distributed, architecture becomes more modular and patterns recognisable. The payoff of our efforts is an application that's easier to extend, to better fit its purpose.
Kudos!


[Feature request] Code Profiling for Ruby/Rails

We're looking for alternatives for our current APM service and noticed that AppOptics doesn't yet support for Code Profiling for Ruby/Rails.
Any plans to add this feature to the AppOptics Ruby Agent?

:) working on it, see branch https://github.com/appoptics/appoptics-apm-ruby/tree/AO-12790_profiling
The solution is written in c++ and uses the same approach as stackprof.
Still a lot of details to work out.

FYI:
Prerelease version is available from packagecloud:
Add this to the end of the Gemfile
source "https://packagecloud.io/solarwinds/appoptics-apm-ruby" do
  gem 'appoptics_apm' , '4.13.0.pre1'
end

more prereleases coming...
Please report any bugs, anomalies, ideas, feature requests...

FYI:
Prerelease version is available from packagecloud:
Add this to the end of the Gemfile
source "https://packagecloud.io/solarwinds/appoptics-apm-ruby" do
  gem 'appoptics_apm' , '4.13.0.pre1'
end

more prereleases coming...
Please report any bugs, anomalies, ideas, feature requests...

make sure to add
AppOpticsAPM::Config[:profiling] = :disabled in the config file
The sampling interval can be changed to as little as 1ms
AppOpticsAPM::Config[:profiling_interval] = 1
The profiler is modeled after stackprof
The profile shows up when looking at a trace. If it is deemed the most interesting info about the trace it will show in the summary view and otherwise you have to dig into a span in the details view

make sure to add
AppOpticsAPM::Config[:profiling] = :disabled in the config file
The sampling interval can be changed to as little as 1ms
AppOpticsAPM::Config[:profiling_interval] = 1
The profiler is modeled after stackprof
The profile shows up when looking at a trace. If it is deemed the most interesting info about the trace it will show in the summary view and otherwise you have to dig into a span in the details view

Finally released today!
https://rubygems.org/gems/appoptics_apm


Add mypy Typehints to pyperclip.copy

Mypy complains about the lack of typehints in the 3rd party library (which is amazing, thank you).
I believe the only method exposed is pyperclip.copy(input: str) -> None
It would be great if this could be added.

As in https://inventwithpython.com/beyond/chapter11.html from the author of pyperclip

after 3 years, this is still opened, still missing, still needed, the static analyzers are still complaining but the "consider donating to its creator on Patreon" (for not maintaining the lib) is still in README.
@asweigart, any comments on how to resolve this contradiction? 😄

I have pull request that would resolve this: #255
I hadn't seen this issue at the time I created the PR.

Can anyone merge that PR, please? :)


:seedling: adding contextual logging in placement component

Summary
Adding contextual logging
Related issue(s)
Fixes https://github.com/open-cluster-management-io/ocm/issues/191
For More Info:
https://www.kubernetes.dev/blog/2022/05/25/contextual-logging/
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/migration-to-structured-logging.md

/assign @haoqing0110

/lgtm

/approve


After HA event, no guest network nor private gw config is pushed to the router

Situation:

working single VPC router on XenServer 6.5
hypervisor crashed
HA kicks in and restarts VM elsewhere

Observation:

router comes back online
none of the VMs in the tier can ping the gateway served by the router
the private gw is not configured
private gw used to be eth6, but the device is unknown
other devices eth0-5 are present
no ip_assoc cmd sent for private gw (grepped all the json files, it wasn't there)
somehow the order of the nics seemed shuffled, because with the old config from /etc/cloudstack/ip.json it didn't work
restart with cleanup gave an error. had to remove private gw in order to do that and replug it.

I collected logs and json files so we can investigate further.

The cause seems to be that the order of the nics in the database, doesn't match the device_id's that are listen. In other words, the list isn't sorted by device_id?
select ip4_address, device_id, broadcast_uri from nics where instance_id=2214

<IP_ADDRESS>	0	NULL
<IP_ADDRESS>	1	vlan://untagged
<IP_ADDRESS>	2	NULL
<IP_ADDRESS>	3	lswitch:39b25b20-fabe-4336-8532-1d3b2db13e0a
<IP_ADDRESS>	4	lswitch:ee7d9436-4d5c-42de-9c13-ca08946414cd
<IP_ADDRESS>	2	lswitch:02007be9-1a9b-406a-8162-631a698634b8

As you can see, device_id 2 got removed and re-added, and as a result Cosmic plugs the interface in the wrong lswitches. Combined with the fact that no json with config is sent, this results in a non-working router after HA event.

These are the commands being sent to the router:
-rw-r--r-- 1 root root   557 May  3 13:17 cmd_line.json.91d79d69-b3ef-4938-a68b-7a55be2ef16b.gz
-rw-r--r-- 1 root root   240 May  3 13:18 ip_associations.json.82be2c20-f896-4e6a-8e5b-abbcff08ac92.gz
-rw-r--r-- 1 root root  1483 May  3 13:18 network_acl.json.a2c6a7da-efd1-4a34-8aa9-5a0479ff358c.gz
-rw-r--r-- 1 root root  1175 May  3 13:18 network_acl.json.c0f5d6ea-e7c9-4cf2-81f6-f0bde87de270.gz
-rw-r--r-- 1 root root   269 May  3 13:18 static_routes.json.dfdc627e-5cee-4378-930d-5cc5ee72fa0c.gz
-rw-r--r-- 1 root root   208 May  3 13:18 vm_dhcp_entry.json.841be47f-b8f2-4586-9817-09fcf36332d9.gz
-rw-r--r-- 1 root root   208 May  3 13:18 vm_dhcp_entry.json.9d9da7ea-f4a7-4f6a-92bd-9302f6c31974.gz
-rw-r--r-- 1 root root   387 May  3 13:18 vm_metadata.json.5fc5af0d-c04c-4bb0-9602-0a5fdf1c179a.gz
-rw-r--r-- 1 root root   388 May  3 13:18 vm_metadata.json.61ef35bc-c667-4bb3-82ee-823f60f9e5b1.gz
-rw------- 1 root root 35066 May  3 13:18 VR-19dc2a40-88cb-455d-b20b-f34ec8ba5231.cfg
-rw------- 1 root root    71 May  3 13:18 VR-2af47e93-649d-4666-be9b-17e8b8872993.cfg
-rw------- 1 root root 16223 May  3 13:18 VR-d56dbc19-6f34-4fc2-878d-7af177df5472.cfg

However, none of them contains info for guest networks or private gw configs. So, the config already in /etc/cloudstack/ips.json is used and since that doesn't match anymore, it won't work.

Assumption: when removing a tier (guest network or private gw) and readding it.
Scenarios to write an integration test for:
Basic (should already work):

vpc single
tier 1 and tier 2
HA event

Scenario 1:

vpc single
tier 1 and tier2
priv gw
cleanup
( priv gw remove )
( priv gw add )
HA event / crash hypervisor

Scenario 2:

single vpc
tier 1, tier 2
delete tier 1
HA event

Scenario 3:

single vpc
tier 1, tier 2
delete tier 1
add tier 1
HA event

Could reproduce it without NSX:
Xenserver 3 tiers:
[root@xen2 ~]# xe vif-list vm-name-label=r-84-VM params=all | grep -E "device|VLAN"
                      device ( RO): 2
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-183
                      device ( RO): 4
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-163
                      device ( RO): 3
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-142

select id,ip4_address,broadcast_uri,device_id from nics where instance_id=84;

129	<IP_ADDRESS>	NULL	0
130	<IP_ADDRESS>	vlan://untagged	1
131	<IP_ADDRESS>	vlan://183	2
132	<IP_ADDRESS>	NULL	3
133	<IP_ADDRESS>	vlan://163	4

Add/remove tier3 vlan 142
[root@xen2 ~]# xe vif-list vm-name-label=r-84-VM params=all | grep -E "device|VLAN"
                      device ( RO): 2
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-183
                      device ( RO): 4
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-163

readding a tier (vlan 152):
[root@xen2 ~]# xe vif-list vm-name-label=r-84-VM params=all | grep -E "device|VLAN"
                      device ( RO): 2
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-183
                      device ( RO): 3
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-152
                      device ( RO): 4
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-163

root@r-84-VM:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 0e:00:a9:fe:02:7c brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 06:f1:e2:00:00:19 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth1
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:25:59:00:04 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/27 brd <IP_ADDRESS> scope global eth2
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:6b:96:00:03 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/27 brd <IP_ADDRESS> scope global eth4
7: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:2c:7e:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/27 brd <IP_ADDRESS> scope global eth3

select id,ip4_address,broadcast_uri,device_id from nics where instance_id=84;

129	<IP_ADDRESS>	NULL	0
130	<IP_ADDRESS>	vlan://untagged	1
131	<IP_ADDRESS>	vlan://183	2
132	<IP_ADDRESS>	NULL	3
133	<IP_ADDRESS>	vlan://163	4
136	<IP_ADDRESS>	vlan://152	3

HA EVENT
[root@xen2 ~]# xe vif-list vm-name-label=r-84-VM params=all | grep -E "device|VLAN"
                      device ( RO): 2
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-183
                      device ( RO): 4
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-152
                      device ( RO): 3
          network-name-label ( RO): VLAN-3d76f172-0ea1-06a0-8095-f6cc0cb5ecf0-163

As you can see, dev3 was vlan152, now it's vlan163.

KVM 3 tiers:
root@r-87-VM:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 0e:00:a9:fe:02:ac brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 06:0d:86:00:00:1a brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth1
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:0b:8c:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth2
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:06:57:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth3
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:09:9e:00:04 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth4

Nics are plugged like this:
[root@kvm1 ~]# virsh dumpxml r-87-VM | grep source | grep bridge
      <source bridge='cloud0'/>
      <source bridge='cloudbr0'/>
      <source bridge='breth0-155'/>
      <source bridge='breth0-172'/>
      <source bridge='breth0-187'/>

Remove tier2
[root@kvm1 ~]# virsh dumpxml r-87-VM | grep source | grep bridge
      <source bridge='cloud0'/>
      <source bridge='cloudbr0'/>
      <source bridge='breth0-155'/>
      <source bridge='breth0-187'/>

root@r-87-VM:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 0e:00:a9:fe:02:ac brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 06:0d:86:00:00:1a brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth1
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:0b:8c:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth2
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:09:9e:00:04 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth4

readd tier2
[root@kvm1 ~]# virsh dumpxml r-87-VM | grep source | grep bridge
      <source bridge='cloud0'/>
      <source bridge='cloudbr0'/>
      <source bridge='breth0-155'/>
      <source bridge='breth0-187'/>
      <source bridge='breth0-184'/>

root@r-87-VM:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 0e:00:a9:fe:02:ac brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 06:0d:86:00:00:1a brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth1
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:0b:8c:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth2
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:09:9e:00:04 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth4
7: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:39:7d:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth3

HA EVENT
root@r-87-VM:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 0e:00:a9:fe:02:ac brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 06:0d:86:00:00:1a brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth1
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:0b:8c:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth2
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:09:9e:00:04 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth3
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:39:7d:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth4

[root@kvm1 ~]# virsh dumpxml r-87-VM | grep source | grep bridge
      <source bridge='cloud0'/>
      <source bridge='cloudbr0'/>
      <source bridge='breth0-155'/>
      <source bridge='breth0-187'/>
      <source bridge='breth0-184'/>

json files:
drwxr-xr-x 2 root root 4096 May  3 16:12 .
drwxr-xr-x 3 root root 4096 May  3 16:12 ..
-rw-r--r-- 1 root root  471 May  3 15:59 cmd_line.json.ae98e37e-8c34-4791-ba8a-9ec23faa4e6b.gz
-rw-r--r-- 1 root root  471 May  3 16:12 cmd_line.json.d96c9dda-b29a-4301-8e7b-ab0e82f04b20.gz
-rw-r--r-- 1 root root  471 May  3 15:59 cmd_line.json.fde0b3a6-6e78-4a44-a825-c25007d8d774.gz
-rw-r--r-- 1 root root  223 May  3 16:10 guest_network.json.0f508f50-8ad1-4b23-b0c1-e3bccfc5ffee.gz
-rw-r--r-- 1 root root  219 May  3 16:02 guest_network.json.1812b500-82a1-4e77-a568-3bdf409a4368.gz
-rw-r--r-- 1 root root  218 May  3 16:09 guest_network.json.588c3cf4-6fd1-46a0-979a-94117e267958.gz
-rw-r--r-- 1 root root  222 May  3 16:03 guest_network.json.60a7a6bf-bd31-4140-b468-1a33ef2aca50.gz
-rw-r--r-- 1 root root  221 May  3 16:06 guest_network.json.818bd25d-43bd-4ed2-91a0-034685c76a8a.gz
-rw-r--r-- 1 root root  238 May  3 16:00 ip_associations.json.590ccd14-41e7-4af5-98e6-a55fb984d60b.gz
-rw-r--r-- 1 root root  231 May  3 16:10 network_acl.json.022a3eb3-36a2-4837-938f-a6c606b3c6f0.gz
-rw-r--r-- 1 root root  202 May  3 16:09 network_acl.json.236fad49-cd93-43f4-92d0-3a5088cad0f3.gz
-rw-r--r-- 1 root root  231 May  3 16:03 network_acl.json.72fc3dac-efc3-469f-927e-252c44ca5eaf.gz
-rw-r--r-- 1 root root  230 May  3 16:06 network_acl.json.ddbb1137-f2a9-4572-a057-603af864061a.gz
-rw-r--r-- 1 root root  227 May  3 16:10 vm_dhcp_entry.json.4da92fb8-cefe-465e-9030-60e564b35b9c.gz
-rw-r--r-- 1 root root  227 May  3 16:02 vm_dhcp_entry.json.743faa9a-1a84-422a-962c-1b5cd6d1b73d.gz
-rw-r--r-- 1 root root  227 May  3 16:03 vm_dhcp_entry.json.9a2def7c-4900-4906-8e4c-60ed15b3aac6.gz
-rw-r--r-- 1 root root  227 May  3 16:06 vm_dhcp_entry.json.f8d9e0ed-caa1-4d4d-96e0-3f738359f1f8.gz
-rw-r--r-- 1 root root  366 May  3 16:02 vm_metadata.json.12039525-1406-4d85-aa07-0014bbfc0bfc.gz
-rw-r--r-- 1 root root  366 May  3 16:03 vm_metadata.json.95c2e903-e65a-4dc0-b23d-28db76f7a1f3.gz
-rw-r--r-- 1 root root  367 May  3 16:06 vm_metadata.json.b1b3c95b-9513-40c5-a92f-c0468893ed3f.gz
-rw-r--r-- 1 root root  366 May  3 16:11 vm_metadata.json.fa3adb83-9e2d-4f89-9134-35dadfdde3ee.gz
-rw-r--r-- 1 root root  130 May  3 16:03 vm_password.json.514c9c8b-c374-42de-abed-c32e4a9eb826.gz
-rw-r--r-- 1 root root  132 May  3 16:06 vm_password.json.c4c4efd6-958b-4e94-8d43-8f868c609879.gz
-rw-r--r-- 1 root root  132 May  3 16:11 vm_password.json.d5f76158-8b5f-4437-9d37-fde5568cb32a.gz
-rw-r--r-- 1 root root  130 May  3 16:02 vm_password.json.f40ef209-edba-4439-ae18-bc46d66b5c6a.gz
-rw------- 1 root root   71 May  3 16:02 VR-5aa1a917-cf6f-4f62-bf0c-1b4802352076.cfg
-rw------- 1 root root  534 May  3 16:10 VR-b011a28a-d367-4d97-a480-e833292ae73e.cfg
-rw------- 1 root root  534 May  3 16:06 VR-e80b0e50-25fc-4f47-8fce-d0791084fdee.cfg
-rw------- 1 root root  534 May  3 16:03 VR-eb61c952-47d7-43f3-8ae1-c34d70510a8f.cfg

This seems to work fine on KVM/vlans. Need to test with OVS/NSX.

 They only issue is that the last nic won't get an ip or route. This is due to the fact the guestnetwork.json contains an empty spot for the removed nic.
{
    "eth2": [
        {
            "add": true,
            "cidr": "24",
            "device": "eth2",
            "dns": "<IP_ADDRESS>,<IP_ADDRESS>",
            "domain_name": "csa1cloud.local",
            "mac_address": "02:00:7c:c5:00:01",
            "router_guest_gateway": "<IP_ADDRESS>",
            "router_guest_ip": "<IP_ADDRESS>",
            "router_guest_netmask": "<IP_ADDRESS>",
            "type": "guestnetwork"
        }
    ],
    "eth3": [],
    "eth4": [
        {
            "add": true,
            "cidr": "24",
            "device": "eth4",
            "dns": "<IP_ADDRESS>,<IP_ADDRESS>",
            "domain_name": "csa1cloud.local",
            "mac_address": "02:00:24:d8:00:01",
            "router_guest_gateway": "<IP_ADDRESS>",
            "router_guest_ip": "<IP_ADDRESS>",
            "router_guest_netmask": "<IP_ADDRESS>",
            "type": "guestnetwork"
        }
    ],
    "id": "guestnetwork"
}

@remibergsma  @borisroman perhaps stating the obvious, but I would say the root cause is the fact that configuration of the nics on the VR relies on the interface names, while the interface names cannot be relied upon (over reboots):
(Building on examples above, KVM, eth2, 3, 4);

When just removing 3;
after reboot nics have been dynamically (re-)numbered,
numbered in the order of creation; new2=old2, new3=old4
but guestnetwork.json specifies an empty config for 3.
When removing, re-adding;
after reboot nics have been dynamically (re-)numbered,
numbered in the order of creation; new2=old2, new3=old4, new4=old3

(see example Remi, KVM 3 tiers, bridge ordering before/after removal/adding).
Solution is to configure interfaces, from guestnetwork.json, based on their MAC instead of interface number?
Or use udev rules? But that would be based on MAC probably again...

@bheuvel I don't the best way to solve this is add more hacks in the Python code. I do think checking for mac addresses would work but it's messy.
The config was supposed to be persistent, and to me the root cause is the fact that the management server decides to reorder, without sending corresponding guest_network.json config file. Would that happen, then the router would merge the received file and reconfigure properly.
The other option I see is to send the nics always in the same order by sorting on the device_id field in the db. That way, you will match whatever had been sent before to the router.

The order, and thereby names, is determined by the plugging order in the config of the VM.
By you example:
[root@kvm1 ~]# virsh dumpxml r-87-VM | grep source | grep bridge
      <source bridge='cloud0'/>
      <source bridge='cloudbr0'/>
      <source bridge='breth0-155'/>
      <source bridge='breth0-187'/>
      <source bridge='breth0-184'/>

Is this order controlled by the management server?

@bheuvel Yes, I think it is. It seems to use the order that they are in the db and not sorted by dev-id. Not sure where it happens, we might need to investigate with the debugger.

@remibergsma when using the sorting based on device_id  (which table is that?), then you will most likely have the same pluging order as in the VM config,
unfortunately you will have basically the same problem, only now that it won't work before the reboot:
So:

create 3 tiers; eth2, eth3, eth4
remove "tier 2":eth3
(re-) add "tier 2" --> will get eth3 from OS

Only after reboot, the OS will have then numbered in the order of creation. So in that case it would get the wrong config (if based on interface name/"number") BEFORE reboot

So where could we find "hard evidence" that (e.g.) eth3 belongs to which network, other then relying on ordering of interface name, number, list-entry?

@bheuvel The only hard link I found is in the nics table, where there is a link between network_id and device_id, like in: select id,ip4_address,broadcast_uri,device_id,network_id from nics;
I don't mean to order on creation, I want to order on device_id because I think that isn't changing. So, in your example, before reboot eth3 is plugged, then device_id equals 3. After reboot, when stuff is ordered on device_id it's still eth3 and will work. I monitored the table and I'm quite sure that doesn't change.
The router doesn't decide on the device, it gets it in the json file from the mgt server based on device_id.
Maybe I miss something but this is what I saw last time I looked into it. Maybe the debugger can shine some light on this otherwise.

@remibergsma
From what I see the router (Linux in general) DOES decide on what name it will assign to a device (at boot or creation), and does this sequentially at discovery order. (unless you have configured otherwise with udev rules)
With the eth3 replug example; everything is configured correctly before reboot
But after reboot Linux will give names in (discovered) order, it will give "new" names; old4->eth3, old3->eth4
ip a actually lists the nics in the correct order. So note how the mac stays on the same order entry, but the ip "stays" with the interface name, as ip IS configured based on interface name.
ip a before reboot:
15: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:22:20:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth4
16: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:26:8f:00:04 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth3

ip a after reboot:
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:22:20:00:02 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth3
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:00:26:8f:00:04 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth4

So main thing is that unless you are using udev rules, a devicename (interface name) CAN, and does, change between reboots.

@bheuvel I totally agree with you. My only point is that the "order" in which they are "discovered", is the same order as the management server "plugs" the nics. So, if we do that in the "device_id" order we should be fine. At start, only the control nic and the public nic are plugged. The rest is plugged later. If the order is different, then we end up with a mismatch.

@bheuvel If you feel we should solve it in the Python code instead then go for it. Although that's a beast ;-)

@remibergsma @borisroman

@remibergsma @borisroman

@remibergsma @borisroman
Long story short, and for reference; I think we agree that the interface name should not be the index in the guest_network.json, but should be the mac address

@bheuvel Yes. Although you make a fair point in suggesting dev should handle that. I'll have a look to see why dev is not working as one would expect.

Udev is disabled in the systemvm template, this was the first occurrence in the code, later it changed from a symlink to a rm of the file. On the other hand, in cloud-early-config there seems to be code to do some magic with udev. Not sure yet what the exact reason is for it to be disabled.


fix: use Osmosis chain type

Description
Since CAIP support was added for Osmosis, and Chain Adapters 1.35.1 was released, a bug surfaced in web, where useWalletSupportsChain.ts will throw, caused by the addition of the Osmosis Chain Type.
This PR makes a small change that fixes the bug whilst bumping the Chain Adapters package.
Notice
Before submitting a pull request, please make sure you have answered the following:

[x] Have you followed the guidelines in our Contributing guide?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Pull Request Type

[x] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[ ] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)
N/A
Risk
N/A
Testing
yarn
yarn dev
Navigate to http://localhost:3000/assets/eip155:1/slip44:60, confirm that you do not get the error in the screenshot below.
Screenshots (if applicable)

Closing in favour of https://github.com/shapeshift/web/pull/1075

Closing in favour of https://github.com/shapeshift/web/pull/1075


Adding compile2 alternative compile function

This PR adds a new compile2() function that allows the compilation of SLP OP_RETURN statements. Because this is new code, there is no danger of a regression bug being introduced into BITBOX.

This is a PR to replace PR #4.


Fully reland https://github.com/flutter/engine/pull/45300

I'm working on fully relanding https://github.com/flutter/engine/pull/45300, which was reverted in https://github.com/flutter/engine/pull/46919 due to performance regression.
Due to how little and trivial production code the original PR touches, I really couldn't figure out the exact line that caused it except through experimentation: I'll reland the changes bit by bit, and perform performance test after each land, since the performance test requires changes to be officially landed on the main branch.

This issue is obsolete since we're no longer seeking to reland the said PR.


Need ability to query multiple compartments at a time

We follow standard practice of keeping network and compute/storage/DB etc under separate compartments. using OKIT we can only query one compartment at a time, either NET or compute/storage/DB.
The OKIT show compute/storage/DB etc without VCNs as they are not under same  compartment. Need an ability to pick multiple compartments to draw the complete picture.

@revnav If you pick a higher level compartment it will query sub compartments to bring back that information.

@toxophilist Yeah, I tried that already. We cannot pick the root compartment from the query option. It only give first level compartments underneath the root.
We have NET under root and then other compartments. The only parent for all is the root level compartment.

@revnav dded Tenancy Root to the list of selectable compartment. This will be in the 0.6.0 release.


Bug: using Struct or BaseModel as dependency callable raises exception

Description
When using a Struct or BaseModel (pydantic) type as the callable for dependencies (@provinzkraut thanks for the idea!), it results in an ImproperlyConfiguredException claiming that it's not type hinted even though it is.
It works only with dataclasses.
URL to code causing the issue
No response
MCVE
from typing import Optional

from msgspec import Struct

from litestar import Litestar, get

class UserFilterQuery(Struct):
    first_or_last_name_contains: Optional[str] = None
    is_active: bool = True

@get("/users", dependencies={"filter_": UserFilterQuery})
async def users_retrieve(filter_: UserFilterQuery) -> None:
    pass

app = Litestar([users_retrieve])

Steps to reproduce
1. Run the MCVE

Screenshots
No response
Logs
Traceback (most recent call last):
  File "/home/guacs/open-source/litestar/litestar/typing.py", line 597, in from_parameter
    annotation = fn_type_hints[parameter.name]
                 ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'first_or_last_name_contains'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/guacs/open-source/litestar/_app.py", line 18, in <module>
    app = Litestar([users_retrieve])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/guacs/open-source/litestar/litestar/app.py", line 453, in __init__
    self.register(route_handler)
  File "/home/guacs/open-source/litestar/litestar/app.py", line 618, in register
    route_handler.on_registration(self)
  File "/home/guacs/open-source/litestar/litestar/handlers/http_handlers/base.py", line 532, in on_registration
    super().on_registration(app)
  File "/home/guacs/open-source/litestar/litestar/handlers/base.py", line 524, in on_registration
    self.resolve_dependencies()
  File "/home/guacs/open-source/litestar/litestar/handlers/base.py", line 357, in resolve_dependencies
    provider.parsed_fn_signature = ParsedSignature.from_fn(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/guacs/open-source/litestar/litestar/utils/signature.py", line 219, in from_fn
    return cls.from_signature(signature, fn_type_hints)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/guacs/open-source/litestar/litestar/utils/signature.py", line 233, in from_signature
    parameters = tuple(
                 ^^^^^^
  File "/home/guacs/open-source/litestar/litestar/utils/signature.py", line 234, in <genexpr>
    FieldDefinition.from_parameter(parameter=parameter, fn_type_hints=fn_type_hints)
  File "/home/guacs/open-source/litestar/litestar/typing.py", line 599, in from_parameter
    raise ImproperlyConfiguredException(
litestar.exceptions.http_exceptions.ImproperlyConfiguredException: 500: 'first_or_last_name_contains' does not have a type annotation. If it should receive any value, use 'Any'.

Litestar Version
HEAD (main)
Platform

[X] Linux
[ ] Mac
[ ] Windows
[ ] Other (Please specify in the description above)

[!NOTE]
While we are open for sponsoring on GitHub Sponsors and
OpenCollective, we also utilize Polar.sh to engage in pledge-based sponsorship.
Check out all issues funded or available for funding on our Polar.sh dashboard

If you would like to see an issue prioritized, make a pledge towards it!
We receive the pledge once the issue is completed & verified
This, along with engagement in the community, helps us know which features are a priority to our users.

The issue is in this line. Due to it setting the function as the __init__ of the class, we don't get any type hints. This is because the msgspec generated __init__ has no type hints. The same is the case for pydantic models though in that case the type hints aren't empty but is instead {"data": Any, "return": NoneType} which is obviously incorrect.
Using the class directly instead of __init__ method of the class solves the issue for these types, but then it'll fail for the classes that are not of this type since in those cases getting the type hints of __init__ is the right behavior.

Pydantic types their __init__ methods like:
__init__(__pydantic_self__, **data: 'Any') -> 'None'

Msgspec types their __init__ methods like:
__init__(self, /, *args, **kwargs)

We could always merge the annotations from the class and the __init__ method where the dependency is a class, however there is an issue with priority.
Say I have a pydantic model like:
class DepClass(BaseModel):
    data: int | None = None

B/c pydantic annotates it's __init__ signature, inspecting its types will return {"data": Any, "return": None}, where data isn't the annotation of our model attribute, it is the annotation of pydantic's __init__(..., **data: Any) argument.
So, for this case we would have to prioritize the class annotations over the __init__ annotations.
However, for a regular python class, something like this is possible:
class DepClass:
    data: int

    def __init__(self, data: Literal["abc", "def"]) -> None:
        self.data = 1 if data == "abc" else 2

In this case, we'd need to prioritize the __init__ arg type over the class type.
Therefore, I think the correct approach would be to offload parsing of annotations to plugins where the type is a class and a plugin supported type.

That makes sense to me. Only default to __init__ if none of the plugins know how to get the type hints for the given class. I have a doubt regarding the implementation: would we have to create a new plugin for this?

would we have to create a new plugin for this?

I think we can leverage the existing ones as they're already capable of extracting the type information from the type.

Is this another instance of needing to add a new non-protocol plugin base type if we have to add a new method for this?

Is this another instance of needing to add a new non-protocol plugin base type if we have to add a new method for this?

Yeah. We should really work on migration all plugins to ABCs. It’s just way more flexible.
Or are you suggesting we don’t do that and find another way to keep working with protocols?

Or are you suggesting we don’t do that and find another way to keep working with protocols?

Not at all.

Yeah. We should really work on migration all plugins to ABCs. It’s just way more flexible.

Yeah, this has been a learning experience for me with protocols. Seems they are only good for simple things like typing a function parameter that expects a thing to have a particular method. The lack of default implementations makes ABCs far superior to an API that is expected to evolve.


Local Host .env variable JISHAKU_NO_UNDERSCORE = true doesn't work

Summary
I am trying to remove the _ before ctx, author... and if I do it with a host it works, but if I do it in local host it doesn't work.
Reproduction steps
Create a .env file, add JISHAKU_NO_UNDERSCORE = true to it, run a thing like jsk py author.name
Expected results
It should work without the underscore
Actual results
It works only with the underscore
Checklist

[ x] I have updated discord.py and jishaku to the latest available versions and have confirmed that this issue is still present
[x ] I have searched the open issues for duplicates
[ ] I have shown the entire traceback, if possible
[ x] I have removed my token from display, if visible

System information

Jishaku: Version: <IP_ADDRESS>
discord.py: Version: 1.3.2
Ruuning on Windows-10-10.0.18362-SP0

Did you even load the environment

Jishaku uses os.getenv which is the Python standard library way of getting the values of environment variables.
The flag not being read means it's not in the environment, which is probably to do with you being on a Windows host that has no compatibility with .env files, perhaps unlike your remote host. This is thus not an issue with Jishaku (as setting the flags in a Windows-compatible way, e.g. $env:JISHAKU_NO_UNDERSCORE=1; in PowerShell works fine).


Separated strings for 'archived'/'archived_menu'

Contribution Type: bug fix
Primary Area: view layout
Description of change:
Separated strings for 'archived'/'archived_menu'

Changes llok fine, can i ask why you need to have a separate string for archived menu?

Because in russian there are two different words. Word 'archive' (archived_menu in lang file) is for the archived projects and 'archived' is for archived project (one project).

So, the main issue is related to one/may element in translation?

Yes. And to be precise it is the different words, but in english they have the equal forms. I think in many non-english languages, these words are not equal too.

@mavimo I answered your question?


crypto/tls: server should send empty server_name extension when using SNI

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.6.2 darwin/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/frschroeder/gopath"
GORACE=""
GOROOT="/Users/frschroeder/go1.6.2"
GOTOOLDIR="/Users/frschroeder/go1.6.2/pkg/tool/darwin_amd64"
GO15VENDOREXPERIMENT="1"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"

What did you do?

Sample app: https://play.golang.org/p/Tk9CR4BUyU
a) Save certPEM from the sample app into a file, e.g. cert.pem
b) Start WireShark and capture traffic on lo or lo0
c) Set a filter tcp.port == 4433
d) Goto Preferences -> Protocols -> SSL and add the cert.pem to the RSA key list for <IP_ADDRESS>:4433 protocol tcp
Run the sample program from  as follows:
1st terminal: go run main.go -server
2nd terminal: go run main.go
This will start a server in the first terminal and a client in the second terminal which executes a SNI handshake.
WireShark will capture and decrypt the traffic

What did you expect to see?

A Server Hello with an empty server_name extension, e.g.
Secure Sockets Layer
    TLSv1.2 Record Layer: Handshake Protocol: Server Hello
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 53
        Handshake Protocol: Server Hello
            Handshake Type: Server Hello (2)
            Length: 49
            Version: TLS 1.2 (0x0303)
            Random
            Session ID Length: 0
            Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
            Compression Method: null (0)
            Extensions Length: 9
            Extension: server_name
                Type: server_name (0x0000)
                Length: 0
            Extension: renegotiation_info
                Type: renegotiation_info (0xff01)
                Length: 1
                Renegotiation Info extension
                    Renegotiation info extension length: 0

What did you see instead?

Secure Sockets Layer
    TLSv1.2 Record Layer: Handshake Protocol: Server Hello
        Content Type: Handshake (22)
        Version: TLS 1.2 (0x0303)
        Length: 49
        Handshake Protocol: Server Hello
            Handshake Type: Server Hello (2)
            Length: 45
            Version: TLS 1.2 (0x0303)
            Random
            Session ID Length: 0
            Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
            Compression Method: null (0)
            Extensions Length: 5
            Extension: renegotiation_info
                Type: renegotiation_info (0xff01)
                Length: 1
                Renegotiation Info extension
                    Renegotiation info extension length: 0

The extension section of the ServerHello does not contain a server_name extension.
When you make a request with an SNI header the client sends a client hello with a server name extension. RFC 6606 Section 3 states:
   A server that receives a client hello containing the "server_name"
   extension MAY use the information contained in the extension to guide
   its selection of an appropriate certificate to return to the client,
   and/or other aspects of security policy.  In this event, the server
   SHALL include an extension of type "server_name" in the (extended)
   server hello.  The "extension_data" field of this extension SHALL be
   empty.

We found this when debugging a TLS connection from the AWS API GW to a go server instance.
I am not sure whether this is a real issue or something that can be ignored. In any case it seems to be mentioned in the RFC and I couldn't find any mention in the issues. So even if it doesn't get addressed people can still find out why.
I realize that you prefer discussion first over code but the following patch added the server_name extension:
diff --git a/src/crypto/tls/handshake_messages.go b/src/crypto/tls/handshake_messages.go
index 111ce53..34f8eaa 100644
--- a/src/crypto/tls/handshake_messages.go
+++ b/src/crypto/tls/handshake_messages.go
@@ -490,6 +490,7 @@ type serverHelloMsg struct {
 	cipherSuite         uint16
 	compressionMethod   uint8
 	nextProtoNeg        bool
+	serverName          bool
 	nextProtos          []string
 	ocspStapling        bool
 	scts                [][]byte
@@ -545,6 +546,9 @@ func (m *serverHelloMsg) marshal() []byte {
 		nextProtoLen += len(m.nextProtos)
 		extensionsLength += nextProtoLen
 	}
+	if m.serverName {
+		numExtensions++
+	}
 	if m.ocspStapling {
 		numExtensions++
 	}
@@ -614,6 +618,11 @@ func (m *serverHelloMsg) marshal() []byte {
 			z = z[1+l:]
 		}
 	}
+	if m.serverName {
+		z[0] = byte(extensionServerName >> 8)
+		z[1] = byte(extensionServerName & 0xff)
+		z = z[4:]
+	}
 	if m.ocspStapling {
 		z[0] = byte(extensionStatusRequest >> 8)
 		z[1] = byte(extensionStatusRequest)
diff --git a/src/crypto/tls/handshake_server.go b/src/crypto/tls/handshake_server.go
index e16cddc..66a9c02 100644
--- a/src/crypto/tls/handshake_server.go
+++ b/src/crypto/tls/handshake_server.go
@@ -122,6 +122,7 @@ func (hs *serverHandshakeState) readClientHello() (isResume bool, err error) {
 	c.haveVers = true

 	hs.hello = new(serverHelloMsg)
+	hs.hello.serverName = len(hs.clientHello.serverName) > 0

 	supportedCurve := false
 	preferredCurves := config.curvePreferences()

CC @agl

I've never understood why the RFC says that and crypto/tls may not always know whether a callback used the SNI name or not. So the only reasonable answers are to never echo the extension or to always echo it. We currently do the former. Your change would lead to the latter. I welcome any suggestions about advantages!

Do other implementations depend on that behavior because it is in the RFC? I realize that the RFC says SHALL and not MUST.

There's no reported problems arising from this I believe. OpenSSL, for what it's worth, allows a callback to decide whether to echo this. Since we don't have that and there's no concrete motivation, I don't see impetus to change our behaviour here.


Ah OK, I thought you could do them with component+binding too

decreased proteasome core complex binding FYPO:0005142
abolished proteasome core complex binding FYPO:0005143
decreased proteasome regulatory particle binding FYPO:0005144
abolished proteasome regulatory particle binding FYPO:0005145
TG'd the required GO MF terms; reasoner will fill in sensible FYPO ancestry when the new GO terms are available.

edit file: a411f52957726ae8601bcdc51ab138446e7a0399
release: a0f809c8ef7bf13df45f7e875d5dd36751ca02f9


Would you please describe the zopfli block-splitting heuristic?

I have been amazed at how good a job it can do, eg finding the boundaries of an inlined (data URL) image very closely in an HTML5 document.
I'd like to understand better how it works its magic.
Rgds
Damon

Would still like to understand this!  B^>
Rgds
Damon


Milestone 1

Client:

[x] Basic syntax highlighting
[x] Diagnostics
[x] Commands
[x] Shutdown - Send shutdown to compiler on de-activate.
[x] Hover
[x] Goto
[x] Find references
[x] CodeLenses (specifically RunMain, RunAllTests)
[x] Show version tooltip when started.
[x] Support multiple instances with multiple ports ...
[x] Check Java version
[x] Download hardcoded version (if not already downloaded and not flix.jar in project folder).
[x] Add command to re-download jar. (add current.jar to nightly?)
[x] Recover/restart from hard crash.
[x] Make robust in face of general errors.
[x] Debounce when typing.
[x] Build a vsix package that actually works.
[x] Remove SelectionRangeRequest
[x] Remove Document Symbol Request
[x] Remove FoldingRangeRequest
[x] Remove Completion

Compiler:

[x] Remove SelectionRangeRequest
[x] Remove Document Symbol Request
[x] Remove FoldingRangeRequest
[x] Remove Completion

Other:

[x] MS marketplace (MM)
[x] Readme (MM)

I moved rename request to milestone 2. I want to narrow the focus and get something releasable.

I moved Document Highlights Request to milestone 2.

Recover/restart from hard crash.

Currently we try to restart the plugin when Flix writes to stderr (vscode has a built-in max of 5 restarts, so it's "safe")

Yay, we did it! Now let the money roll in: https://github.com/flix/flix/issues/1374


Having issue suppressing credential scanner results with .gdnsuppress file

I am setting the task as follows:
- task: MicrosoftSecurityDevOps@1
   displayName: Run MS Security DevOps
   inputs:
     command: 'run'
     policy: 'microsoft'
     break: true
     artifactName: 'CodeAnalysisLogs'
   ${{ if or(ne(parameters.SuppressionPaths, ''), ne(parameters.SuppressionPlaceholders, '')) }}:
     env:
       GDN_RUN_SUPPRESSIONFILE: MSDOSuppressionFile

I am creating the .gdn/MSDOSuppressionFile.gdnsuppression file with the following data:
{
    "tool": "Credential Scanner",
    "suppressions": [
        {
            "file": [
				"$(Build.SourcesDirectory)/src/main.ts",
				"$(Build.SourcesDirectory)/src/test.ts"
            ],
            "_justification": "False positive files"
        },
        {
            "placeholder": [
				"'client_secret': x.mrdhApicCSec,",
				"'Some-Secret': x.mrdhApicXIBMCSec,",
				"'Some-Client-Secret': x.deregApicXIBMCSec,",
				"'Some-Client-Secret': x.mozartApicXIBMCSec"
            ],
            "_justification": "False positive lines"
        },
        {
            "hash": [
				"9F0Kuc6b47nAOIqDqhEFct82R5oR+bP1+CcIwZlrRuI=",
				"test"
            ],
            "_justification": "False positive hashes"
        }
    ]
}

I have slowly expanded from its original which only had the "file" object. Since it was not working, I kept expanding to figure out what worked so I then added placeholders and then hashes.
Currently I am getting:
Suppression Files: MSDOSuppressionFile
...
##[error]3. Credential Scanner Error CSCAN-GENERAL0130 - File: src/test.ts:src/test.ts. Line: 70. Column 29.
    Signature: 94538d51b97d725caf27deaa18b50e2b5fa257d8008b1dcc49be508c2af846db
    Tool: Credential Scanner: Rule: CSCAN-GENERAL0130 (Client Secret / Api Key). https://aka.ms/CredScanDocs
    A potential secret was detected. Validate file contains secrets, remove, rotate credential, and use approved store. For additional information on secret remediation see the remediation section at https://aka.ms/CredScanDocs

I have tried multiple ways to set the path for the files to be suppressed, in case I was setting the files wrong, but those two files keep getting picked up; test.ts and main.ts.
Not sure where the suppression is failing and how.

Hi @DevOpsAzurance, thank you for reaching out! I'm happy to help you get this resolved. First off, I can see that you are manually constructing the suppressions file which can be error-prone and confusing. Have you tried generating a suppression file using the GDN_RUN_OUTPUTSUPPRESSIONFILE environment variable and comparing the contents? If not, please give that a try and see if it helps. You can read more about it in the table of supported environment variables here: https://github.com/microsoft/security-devops-azdevops/wiki

Hi @DevOpsAzurance, thank you for reaching out! I'm happy to help you get this resolved. First off, I can see that you are manually constructing the suppressions file which can be error-prone and confusing. Have you tried generating a suppression file using the GDN_RUN_OUTPUTSUPPRESSIONFILE environment variable and comparing the contents? If not, please give that a try and see if it helps. You can read more about it in the table of supported environment variables here: https://github.com/microsoft/security-devops-azdevops/wiki

Part of our Pipeline-As-Code allows DevOps team to add hashes, files paths and placeholders to be suppressed.  So it get's built in an automated manual process.
this is the template json we use:
      $credSuppressionTemplate = @'
      {{
          "tool": "Credential Scanner",
          "suppressions": [
              {{
                  "file": [
      {0}
                  ],
                  "_justification": "False positive files"
              }},
              {{
                  "placeholder": [
      {1}
                  ],
                  "_justification": "False positive lines"
              }},
              {{
                  "hash": [
      {2}
                  ],
                  "_justification": "False positive hashes"
              }}
          ]
      }}
      '@

And we allow automation to fill in the fields using:
$credSuppressionTemplate -f "$strSuppressionFiles","$strSuppressionPlaceholders","$strSuppressionHashes" | Out-File -FilePath .config/CredScanSuppressions.json -Encoding utf8 -Force
the output suppression file I got is returning the hashes, which is problematic, because we are running the tool at the root of the repo, but the pipeline is only concerned about one project out of two.  I want to exclude the other paths, but suppressions does not accept wildcards, so trying to add the file paths so any future changes, will be suppressed, instead of using hashes which will cause a new false positive.
{
    "hydrated": false,
    "properties": {
        "helpUri": "https://eng.ms/docs/microsoft-security/security/azure-security/cloudai-security-fundamentals-engineering/security-integration/guardian-wiki/microsoft-guardian/general/suppressions",
        "hydrationStatus": "This file does not contain identifying data. It is safe to check into your repo. To hydrate this file with identifying data, run `guardian hydrate --help` and follow the guidance."
    },
    "version": "1.0.0",
    "suppressionSets": {
        "default": {
            "name": "default",
            "createdDate": "2024-04-23 20:13:14Z",
            "lastUpdatedDate": "2024-04-23 20:13:14Z"
        }
    },
    "results": {
        "7897d9d52460c52d4bdadac5f541a2f4f897440a845a77bae4597b6640d64cb4": {
            "signature": "7897d9d52460c52d4bdadac5f541a2f4f897440a845a77bae4597b6640d64cb4",
            "alternativeSignatures": [],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "092b6ef5ac77143789f747f7b2139bd1260990b3aec72ff018a74cc7b01cabac": {
            "signature": "092b6ef5ac77143789f747f7b2139bd1260990b3aec72ff018a74cc7b01cabac",
            "alternativeSignatures": [
                "f20f43937ce582e53641938340a9c8a58616e1a0a1c1468d3fb75b2ceb5e901e"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "5e66fbf5b767ffc89212e43d17a76394d2c3128f11d6e9a5c6e3da1a6249a9c3": {
            "signature": "5e66fbf5b767ffc89212e43d17a76394d2c3128f11d6e9a5c6e3da1a6249a9c3",
            "alternativeSignatures": [
                "5a2a092ffe417b9094f90884e068080afaae75f0ee3b58c98935517d59332f40"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "e03032d4a9ad066785c4b20e559c671b2a0829b78069ed70ae0c24c0bab7e82b": {
            "signature": "e03032d4a9ad066785c4b20e559c671b2a0829b78069ed70ae0c24c0bab7e82b",
            "alternativeSignatures": [
                "861712aaf8a6de62452e31a9a8fe53644c491972e8b6a86e91a2a9362be5807c"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "515f09c25304103cb819e7014e3b8b7ad4b5e429423e816e97c23da1b0865144": {
            "signature": "515f09c25304103cb819e7014e3b8b7ad4b5e429423e816e97c23da1b0865144",
            "alternativeSignatures": [
                "aa665d8a162a70ee3ac145eee84747e0f10cfb0a4adbd0315e55d31808eb714c"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "7889bd0856caa9ae9bb6afe87de9290e2e612cd75f7818f865a514c488cf154a": {
            "signature": "7889bd0856caa9ae9bb6afe87de9290e2e612cd75f7818f865a514c488cf154a",
            "alternativeSignatures": [
                "40e8a53fc8672cf51853c4483ff1a6caa321e2671037fe6d367bb878821546c0"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "8059d408810d9c5c5c10a96c8c9ae20aedcf26a7b9979179ed0ffbc8ad2fff06": {
            "signature": "8059d408810d9c5c5c10a96c8c9ae20aedcf26a7b9979179ed0ffbc8ad2fff06",
            "alternativeSignatures": [
                "5f872581a4ef33068acb7beeea26bdd7cdd3d868d769814a5d422d9332933a21"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        },
        "2efb759f5f84ea9d5d330d08ff02bced716da76799adbd42ba8f2b55b417dd27": {
            "signature": "2efb759f5f84ea9d5d330d08ff02bced716da76799adbd42ba8f2b55b417dd27",
            "alternativeSignatures": [
                "7c1e930e322ce4eb1d69e4d1272ad9ab68fea0d9e5546cd70a4432aece9bbe66"
            ],
            "memberOf": [
                "default"
            ],
            "createdDate": "2024-04-23 20:13:14Z"
        }
    }
}

How can I use paths instead...

Thank you for explaining your scenario @DevOpsAzurance
Based on your description, I think a simpler path may be to pass through the argument to CredScan that makes it scan a particular folder. You can override the default with the env variable GDN_CREDSCAN_TARGETDIRECTORY. Here is an example.

task: MicrosoftSecurityDevOps@1
displayName: 'Run MSDO'
env:
GDN_CREDSCAN_TARGETDIRECTORY: 'Relative/Path/To/Folder'

Thank you for explaining your scenario @DevOpsAzurance
Based on your description, I think a simpler path may be to pass through the argument to CredScan that makes it scan a particular folder. You can override the default with the env variable GDN_CREDSCAN_TARGETDIRECTORY. Here is an example.

task: MicrosoftSecurityDevOps@1
displayName: 'Run MSDO'
env:
GDN_CREDSCAN_TARGETDIRECTORY: 'Relative/Path/To/Folder'

(Please disregard incorrect indentation above, it keeps reformatting on me)

That particular environment variable is not in the list at:
https://github.com/microsoft/security-devops-azdevops/wiki
Is there a more complete list that is not currently shared.

Unfortunately the exclusion of CredScan from those docs is intentional as the tool was officially deprecated from MSDO last September but it has not been removed entirely yet. For a better experience with secret scanning in Azure DevOps, you may want to look at GitHub Advanced Security for Azure DevOps.

How would I create a the suppression file, but using paths?  I ran pipeline using the command to output the suppression file, but it is using the hashes.  I don't want to run into constant issues, if they do edit the file, so would rather have the suppression done at the file level? I cannot find something similar to it. Since the GDN_RUN_CONFIG can take multiple configs, I tried the following:
 ```

$terrascanSuppressionTemplate = @'
{{
"tools": [
{{
"tool": {{
"name": "Terrascan",
"version": "Latest"
}},
"suppressions": [
{{
"file": [
{0}
],
"_justification": "False positive files"
}}
]
      }}
    ]
  }}
  '@

I have seen similar for the Credential Scanner tool, and instead of arguments, has suppressions.  It fails though as it seems to be making the call to terrascan in a mannger, that instead I get the "how to properly use" terrascan output:

Analyze:
Running Terrascan <IP_ADDRESS>
------------------------------------------------------------------------------
/usr/EveDORunnersLinux1/_work/_msdo/packages/nuget/Microsoft.Guardian.TerrascanRedist_linux_amd<IP_ADDRESS>.1/tools/terrascan
Terrascan
Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure.
For more information, please visit https://runterrascan.io/

Usage:

rascan [command]
Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  init        Initializes Terrascan and clones policies from the Terrascan GitHub repository.
  scan        Detect compliance and security violations across Infrastructure as Code.
  server      Run Terrascan as an API server
  version     Terrascan version

Flags:
  -c, --config-path string      config file path
  -h, --help                    help for terrascan
  -l, --log-level string        log level (debug, info, warn, error, panic, fatal) (default "info")
      --log-output-dir string   directory path to write the log and output files
  -x, --log-type string         log output type (console, json) (default "console")
  -o, --output string           output type (human, json, yaml, xml, junit-xml, sarif, github-sarif) (default "human")
      --temp-dir string         temporary directory path to download remote repository,module and templates

terrascan [command] --help" for more information about a command.
Tool run time: 0.025032 seconds
------------------------------------------------------------------------------
Terrascan completed with exit code 0
------------------------------------------------------------------------------
Process:
Convert:
Converting any raw tool logs to Sarif format ...
Completed converting raw tool logs to Sarif format.
Import:
##[error]JsonReaderException: Unexpected character encountered while parsing value: T. Path '', line 0, position 0.
##[error]MSDO CLI exited with an error exit code: 1

I am assuming the failure is that it expected output in json, and instead got a readme.


Some console commands are not available in zombie Z mode.

bot_add
buyammo1
You cannot use the Bot add command and bullet purchase command.

The bot_add console is used and Buyammo1 is not needed because it is available in the in game store.


DASH trick mode tracks

Shaka Player has no knowledge of DASH trick mode tracks.  According to DASH IOP v3.3, trick mode tracks should not be selected for normal playback.
Steps for us:

find sample assets for trick mode so that we can properly test this feature
ignore trick mode tracks, since we do not yet support them  (we should get at least to this stage as soon as possible, for release in v2.1 or v2.0.1)
use trick mode tracks when player.trickPlay() is used, limited to maxPlayoutRate from the manifest  (this may take much longer to achieve, and may or may not make v2.1)

Here's the relevant text from IOP v3.3:

3.2.9. Trick Mode Support
Trick Modes are used by DASH clients in order to support fast forward, seek, rewind and other operations in which typically the media, especially video, is displayed in a speed other than the normal playout speed. In order to support such operations, it is recommended that the content author adds Representations at lower frame rates in order to support faster playout with the same decoding and rendering capabilities.
However, Representations targeted for trick modes are typically not be suitable for regular playout. If the content author wants to explicitly signal that a Representation is only suitable for trick mode cases, but not for regular playout, it the following is recommended:

add an Adaptation Set that that only contains trick modes Representations
annotate the Adaptation Set with an EssentialProperty descriptor or SupplementalProperty descriptor with URL "http://dashif.org/guidelines/trickmode" and the @value the value of @id attribute of the Adaptation Set to which these trick mode Representations belong. The trick mode Representations must be time-aligned with the Representations in the main Adaptation Set. The value may also be a white-space separated list of @id values. In this case the trick mode Adaptation Set is associated to all Adaptation Sets with the values of the @id.
signal the playout capabilities with the attribute @maxPlayoutRate for each Representation in order to indicate the accelerated playout that is enabled by the signaled codec profile and level.

If an Adaptation Set is annotated with the EssentialProperty descriptor with URI "http://dashif.org/guidelines/trickmode" then the DASH client shall not select any of the contained Representations for regular playout.

@maxPlayoutRate is not the only limiting factor for trick play rate.  We should also make sure that the trick play track's bandwidth, multiplied by the trick play rate, does not exceed the user's estimated bandwidth.
For example, in this test clip from the DASH IF, the lowest bitrate normal stream is about 3x the bitrate of the trick play stream.  For clients unable to play any but the lowest bitrate stream, the max trick play rate would be about 3.
Another way to look at it, which would still work for content without a trick play track, is to consider the current playback rate in track selection.  Choose a track based on estimated bandwidth divided by current playback rate.

Two working test clips from the DASH IF:

http://dash.akamaized.net/dash264/TestCases/9b/qualcomm/1/MultiRate.mpd
http://dash.akamaized.net/dash264/TestCases/9b/qualcomm/2/MultiRate.mpd

Finally, we should consider how reverse trick play works.  Today, browsers do not support negative playback rates.  It is unclear if they ever will.  We currently use repeated seeking to simulate negative playback rates, but this is not a great experience.
Our forward trick play experience will be greatly improved by the use of special trick mode tracks, so we should look for a way to improve our reverse trick play experience as well.  Perhaps having a special trick mode track will open up another possibility or allow us to refine the technique we use today.

The above-referenced trick-play assets fail to decode in Chrome (except the trick play tracks themselves, ironically).  So we'll need to find other trick play assets.  Media logs (formatted via this script):

Timestamp
Property
Value

00:00:00 00
pipeline_state
kCreated

00:00:00 00
event
WEBMEDIAPLAYER_CREATED

00:00:00 00
url
blob:http://localhost/6dd4de73-abdd-4c59-ae4f-1d4ec253bb26

00:00:00 00
pipeline_state
kInitDemuxer

00:00:00 229
duration
654

00:00:00 425
info
Audio codec: mp4a.40.2. Sampling frequency: 24000Hz. Sampling frequency(Extension): 48000Hz. Channel layout: 3.

00:00:00 425
found_audio_stream
true

00:00:00 425
audio_codec_name
aac

00:00:00 480
info
Video codec: avc1.4D401F

00:00:00 480
found_video_stream
true

00:00:00 480
video_codec_name
h264

00:00:00 480
pipeline_state
kInitRenderer

00:00:00 482
audio_dds
false

00:00:00 482
audio_decoder
FFmpegAudioDecoder

00:00:00 483
video_dds
false

00:00:00 483
video_decoder
FFmpegVideoDecoder

00:00:00 483
pipeline_state
kPlaying

00:00:00 694
error
video frame with PTS 0us has negative DTS -83333us after applying timestampOffset, handling any discontinuity, and filtering against append window

00:00:00 694
error
Append: stream parsing failed. Data size=131072 append_window_start=0 append_window_end=654.04

00:00:00 698
pipeline_error
chunk demuxer: append failed

00:00:00 698
pipeline_state
kStopping

00:00:00 699
pipeline_state
kStopped

Since the DASH-IF's test assets for trick play do not decode in Chrome, I will need to make a new one.  I analyzed the trick play track from the manifest linked above, and I see only I-frames at 1 fps.  The segment size is 2s, just as in the normal tracks.  This will not be difficult to replicate.

@waqarz added new trick mode vectors here: http://testassets.dashif.org/#testvector/list
Thanks!

This is deferred until after some HLS-related refactoring is done.  HLS work is higher priority and may conflict with this, since both affect the internal manifest representation.  HLS work is tracked in #279.

Refactoring for HLS is done, so we can now continue work on trick mode.


Add better errors to well-formedness check

Closes #164

Changed check_wf return type to Result<(), TerminationInfo> such that we can append error information.
Added a String to TerminationInfo::IllFormed for the error message.
Adapted the throw_ill_formed macro such that it adds the position of the error.
Made ensure a macro as well such that it interacts nicely with throw_ill_formed.

The error messages are quite brief in the moment, but I am also happy to change that to more descriptive ones.

@RalfJung I updated the PR :)


[BUG] Gitsplit

Observed behavior
Gitsplit does not push tags to target repository
Expected Behavior
Relevant tags in this repository will exist in the downstream repository after splitting
Steps to reproduce
Merge a release, and check the downstream repository for the created release tag. It will not exist.

It may make sense to forego gitsplit as it doesn't seem to support tags.
In this case, we could instead use git filter-branch since all directories are self contained.
git filter-branch --prune-empty --subdirectory-filter providers/Flagd -- --tags

You can rename tags as well with --tag-name-filter. More info on that here.

Honestly, the simple example for git-filter-repo may be sufficient, but will need to development into a GitHub Action

--subdirectory-filter would suffice to both filter to a subdirectory and rename it with one flag.


Protect page error

When I try to protect a page with DiscordNotifications installed I get the following error:
[W4N5pjbmAHgAjUUkWSvWIAAAABA] /index.php?title=Main_Page&action=protect ArgumentCountError from line 212 of /home/retror/public_html/extensions/DiscordNotifications/DiscordNotificationsCore.php: Too few arguments to function DiscordNotifications::discord_article_protected(), 4 passed in /home/retror/public_html/includes/Hooks.php on line 177 and exactly 5 expected
Backtrace:
#0 /home/retror/public_html/includes/Hooks.php(177): DiscordNotifications::discord_article_protected(WikiPage, User, array, string)
#1 /home/retror/public_html/includes/Hooks.php(205): Hooks::callHook(string, array, array, NULL)
#2 /home/retror/public_html/includes/page/WikiPage.php(2521): Hooks::run(string, array)
#3 /home/retror/public_html/includes/page/Article.php(2593): WikiPage->doUpdateRestrictions(array, array, boolean, string, User)
#4 /home/retror/public_html/includes/ProtectionForm.php(312): Article->doUpdateRestrictions(array, array, boolean, string, User)
#5 /home/retror/public_html/includes/ProtectionForm.php(187): ProtectionForm->save()
#6 /home/retror/public_html/includes/page/Article.php(1544): ProtectionForm->execute()
#7 /home/retror/public_html/includes/actions/ProtectAction.php(52): Article->protect()
#8 /home/retror/public_html/includes/MediaWiki.php(500): ProtectAction->show()
#9 /home/retror/public_html/includes/MediaWiki.php(294): MediaWiki->performAction(Article, Title)
#10 /home/retror/public_html/includes/MediaWiki.php(861): MediaWiki->performRequest()
#11 /home/retror/public_html/includes/MediaWiki.php(524): MediaWiki->main()
#12 /home/retror/public_html/index.php(42): MediaWiki->run()
#13 {main}

Can you try removing the parameter $moveonly = false from the function call on file DiscordNotificationsCore and let me know if it works after that?
So change this:
static function discord_article_protected($article = null, $user = null, $protect = false, $reason = "", $moveonly = false)
To this:
static function discord_article_protected($article = null, $user = null, $protect = false, $reason = "")
I have a feeling that the last parameter is not actually being passed by MediaWiki Core and that's causing the issue.

Closing as no reply.


Make npm bundle smaller

The npm bundle comes with everything and tests. But also comes with the docs, which is >90% of the package total size:

I'm adding the files entry in package.json to tell npm to only publish that file. Please let me know if some other file should be packed.

Pull Request Test Coverage Report for Build 23

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 22:
0.0%

Covered Lines:
625

Relevant Lines:
625

💛  - Coveralls


Add an endpoint that returns a list of bids made by a specific account

To make this screen: https://app.zeplin.io/project/5ce5002ad64fdd1e1600ee8f/screen/5d719807039b774ecd49d012

@davidyuk the screen is not accessible without login. would be nice if you can provide more detailed feature request.

sample endpoint
sample return data
list of extra query params that would be nice to have

The endpoint that I'm asking for:

should accept an account address
should return an array of objects (bids made by that account)
each object should contain at least

some identifier of auction (probably name)
amount of tokens at that bid
the flag that should display is it the last bid in corresponding auction or not
bid time may be included as well

http://aeternal.io:8000/middleware/names/auctions/bids/account/ak_VYYoJkwLARSzeJWUsATEwz9hk7e8g4SXuGjrqzVDfPeCt8mg5
There is no simple way to implement the 'is it winning or not' field, is this totally necessary?

It is need for this screen:

otherwise, I will have to fetch the details of each auction by myself

yes, I do see your point.
I spent some time at the weekend trying to find a clean way of doing this. There are two options which work the way the middleware currently works:

you could use the http://aeternal.io:8000/middleware/names/auctions/active endpoint and use the winning_bidder info in there to figure out whether the user you care about is winning, or
I can do this in the code, and give you the results back as a tuple of (winning: {true|false}, transaction: tx}

otherwise I must make a new struct, which I hate. Let me know if either of these works, and we'll figure it out

The second option looks better for me. Does it have any performance drawbacks or why it is not a simple way?

It ought to be quick. It's just not beautiful. But it does keep the transaction JSON format consistent, which is nice, and we can add more metadata in later if required. I will crack on with this.


Graphql

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[x ] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Test A
[ ] Test B

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules
[ ] Coverage is the same or higher than it was before my changes. If it's lower, it's because:

To-Dos
List anything that needs to be looked at or changed in future commits. Please include file path and line number if relevant.

[ ] to do

Closed with team approval, only partially done


fix

The paper "Speeding Up End-to-end Query Execution via Learning-based Progressive Cardinality Estimation" is accepted by SIGMOD 2023, not 2022.

thanks for your correction


Support for IDEA version 2022.2

Thanks for creating this PR. I will check on Friday if everything still works correctly on this new version.


'name' incosistency in seg_track_20 JSONs for MOTS labels

Hello, I found some name inconsistency in JSON files which make errors while converting format using to_coco.py script.
Most of the 'name' values in the JSON files consist of 'videoName-framenubmer.jpg'.
However, the name of the following files is like videoName/videoName-framenubmer.jpg.
A list of files:
0062e803-38c0a33a.json
000f157f-dab3a407.json
006fdb67-f4820206.json
0062f18d-f8cd3a65.json

Should be fixed, you can try redownloading the annotations and trying


Why does the notebook image need to have a custom kernel and virtualenv?

Just a quick question. If it's in a container, why does it need to have a virtual environment set up for it?

@automa7 Ya, I guess it is not necessary. You can install the libraries globally.


update packages due to two vulnerable dependencies

When installing the package there a two major and some moderate vulnerable packages.

I tried to do it myself but I am not sure if I really have a view for the whole logic and can fix it without breaking the package.


Pre merge l2geth

Applies the l2geth changes which are in the pending merge from upstream, reducing merge conflicts. There will still be a conflict for the oETH address.

Codecov Report

Merging #54 (b44812b) into develop (0fe7ad9) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head b44812b differs from pull request most recent head bc09e85. Consider uploading reports for the commit bc09e85 to get more accurate results

@@           Coverage Diff            @@
##           develop      #54   +/-   ##
========================================
  Coverage    73.06%   73.06%
========================================
  Files           65       65
  Lines         2198     2198
  Branches       311      311
========================================
  Hits          1606     1606
  Misses         592      592

Flag
Coverage Δ

batch-submitter
59.59% <ø> (ø)

contracts
90.36% <ø> (ø)

core-utils
56.91% <ø> (ø)

data-transport-layer
37.68% <ø> (ø)

message-relayer
71.69% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0fe7ad9...bc09e85. Read the comment docs.


com.android.dex.DexException: Multiple dex files define Lcom/google/android/gms/internal/zzqf;

Hi, @evollu !
i want help to guys to fix this issue with multiple dex ,etc.
really working solution is forking needed repos, and change dependency versions in build.gradle at /node_modules/{NAME_LIB}/android/build.gradle, to bring to one appearence
e.g, in my project i need<EMAIL_ADDRESS>and<EMAIL_ADDRESS>in maps lib build.gradle was set:

compile "com.google.android.gms:play-services-base:9.8.0"
compile "com.google.android.gms:play-services-maps:9.8.0"

and in fcm:

compile 'com.google.firebase:firebase-core:+'
compile 'com.google.firebase:firebase-messaging:+'

so that we need change build.gradle file our fcm lib so that it looks as follows, and error will disappear=)

compile 'com.google.firebase:firebase-core:9.8.0'
compile 'com.google.firebase:firebase-messaging:9.8.0'

add this ad README.md file, for someone it may be useful)
thank you

@andryuwka please search the issues
the solution is to force a specific version of firebase-core

@sibelius i searched solution in this two repos, and i tryied force change version and excluding dependencies, but nothing helped.
can you show me, what i need to do?
now my app build.gradle dependencies looks like this:

compile project(':react-native-fcm')
compile project(':react-native-maps')
compile project(':react-native-spinkit')
compile ('com.android.support:appcompat-v7:25.0.1') {
exclude group: 'com.google.android', module: 'support-v4'
}
compile ('com.google.firebase:firebase-core:9.8.0')
compile fileTree(dir: "libs", include: ["*.jar"])
compile "com.facebook.react:react-native:+"  // From node_modules
compile('com.crashlytics.sdk.android:crashlytics:2.6.5@aar') {
transitive = true;
}

and at the bottom, there is line with apply plugin: "com.google.gms.google-services"

u should use the latest version of firebase-core

yeah, i can read instructions=) but it's not helped to me)

https://github.com/evollu/react-native-fcm#im-getting-comandroiddexdexexception-multiple-dex-files-define-lcomgoogleandroidgmsinternalzzqf

i have already read this

I write whole article about this problem:
<EMAIL_ADDRESS>Hope I helped you :)

@Nodonisko you are awesome!!! it works!
@evollu i think that you can close this issue and post this solution at README.md

it is in readme FAQ section.
I will include @Nodonisko 's article

https://github.com/airbnb/react-native-maps/issues/831
worked for me!

I had this problem, and this was my solution:
Add compile 'com.google.android.gms:play-services-gcm:+' to the dependeencies in
/node_modules/react-native-fcm/android/build.gradle
Im not experinced in Android, but seems to me that if we configure the version of that library in the build.gradle of the app, it will create conflict with other existent modules that has already that library as a dependence, perhaps with different version.
To avoid that, I added the dependence in the build.gradle of the module itself, and now every module works with the version it needs.
(This is my own interpretation, may be wrong but the fix worked for me.. I hope it helps you.. Cheers!!)

sounds like the right way. let me verify


Dependencies update

Fixes #205
The most important is asm dep update. As jarjar 1.6.x depends on asm 5.0.4, which contains bug https://github.com/sbt/sbt-assembly/issues/205#issuecomment-279964607 (thx @karol-brejna-i  for pointing to that).

Thanks for the fix!

:+1: thanks, @pomadchin, for following up on this and providing a workaround!

Any plan to release with this bug fix ? We (Gearpump) prefer to upgrade sbt-assembly rather than a workaround.

I can get this out today.

https://github.com/sbt/sbt-assembly/releases/tag/v0.14.4


Vermont

Hello!
I saw you post this on Twitter and thought I could be of assistance. I work for the Vermont Agency of Transportation (VTrans). We have an Esri Open Data Portal which might be helpful for this project.
VTrans ODP
The Projects data might be of most use.
Projects data
If there is specific data that you would like to see from us, please let me know and I'll try to get in touch with the people in charge of that info.
~Stephen

This is great! Thanks for the info Stephen.


Can't get out of the starting blocks

Hi,
I'm doing a really simple proof of concept, and it's not working.
In my DbContext class I have:
      protected override void OnConfiguring(DbContextOptionsBuilder options)
        {
            options
                .UseInternalServiceProvider(_serviceProvider)
                .UseSqlServer(_connectionString)
                .UseSqlServerDateDiff();
        }

Then, in my repo code I have:
var q = dbContext.Things.Where(t => DateDiff.Seconds(t.Start, t.End) > 3)
    .ToList();

And this immediately throws an exception:

System.InvalidOperationException : Call UseSqlServerDateDiff to use DateDiff

What am I doing wrong?

I note tangentially with interest that this is issue no 1 logged here.  Is this project perfect, or just unused?

Hi, @shaulbehr
Surely, it's perfect :)
Speaking seriously, as README states:

As far as I know, there is no "official" way to extend EF.Core to use DATEDIFF.
The way used here is a pure hack. It may break with the next EF.Core release. Use with caution.

This library was implemented for EF.Core 1.0.0 and has never be tested/updated for following EF.Core releases. What version you are using? If it's something > 1.0.0, I'm afraid I can not help you. But you are welcome to contribute if you like.


Overload error

Getting overload errors when compiling
Value.cpp:227:51: error: call of overloaded 'String(double&, uint8_t&)' is ambiguous

The current version of the Arduino WappstoIoT library supports only ESP32 library version 1.0.x. The ESP32 library version we are testing with is 1.0.6.


Build error with clean checkout from master

Clean checkout of master branch, running make binary
[INFO]	--> Setting version for gopkg.in/mgo.v2 to 29cc868a5ca65f401ff318143f9408d02f4799cc. [INFO]	--> Setting version for gopkg.in/fsnotify.v1 to a8a77c9133d2d6fd8334f3260d06f60e8d80a5fb. [ERROR]	Failed to set references: Unable to update checked out version (Skip to cleanup) The command '/bin/sh -c glide install' returned a non-zero code: 1 Makefile:51: recipe for target 'build' failed make: *** [build] Error 1
I can start searching about inside the container, but wanted to check if I'd missed something silly first.  I have read contributing.md.

No, you're not the first one to notice this. I've left a note on Slack for the developers. The error itself is a bit unusual since the commit with that hash is still accessible from the repository.

Hi @skeptomai @nathan-osman, it should be fixed by #773

Merci.  That does seem to fix it for me.  I'm new to the codebase, so I don't see the equivalent of a 'clean' target in the makefile though, to ensure I'm really safe.


Allow a zero timeout for g_obj_wait()

At the moment, timeouts of -1 or 0 for g_obj_wait() both mean 'no timeout', i.e. wait forever. This commit changes a timeout of 0 to mean 'return immediately'. This is more consistent with poll() and WaitForMultipleObjects(), and can be used by the new encoder logic.
There are a very few places on xrdp where a timeout of 0 is currently specified to mean 'wait forever'. These are replaced with values of -1.

While doing this I've discovered the timer queue processing in chansrv can be significantly improved. I'll fix that in devel at some stage

Issue #2906 added about chansrv timer queue mentioned above.
BTW the FreeBSD CI failures above can be fixed by using a later FreeBSD (either by rebasing gfx_mainline_merge_work or cherry-picking 05c78bb284764d06bf7750e37cea19c773f439ff)

Yes, FreeBSD 12 is EoL'd

@matt335672 Could you cherry-pick 05c78bb284764d06bf7750e37cea19c773f439ff to gfx_mainline_merge_work?

I've updated gfx_mainline_merge_work in the main repository to address the FreeSDB issue.
@Nexarian, @jsorg71 - can you update your local repositories?

Will update

This PR looks good to me, but for some reason I don't understand why the diff has changes in it that it shouldn't. This should just apply to the g_obj_wait changes? Right?

This is what you intended to change, correct? https://github.com/neutrinolabs/xrdp/pull/2891/commits/353d76f7543f2992aec326d036fcbbaff89cb1d1

I'll delete this now - @Nexarian has already merged it.


[VT-flash] Refactor closure generation

Function __vt_flash_optim_closures in variable-related code seems duplicative for gradient's and helmholtz energy's closures.
We may generate them for an AbstractVTFlashState from user-provided gradient! and nmolvol. So there won't be need to define __vt_flash_optim_closures.
See, the code below is general
https://github.com/vvpisarev/CubicEoS.jl/blob/dd2b6a14fe0a73d235c499527639d6269407a18e/src/vt_flash/state_ratio.jl#L141-L161
Only constrain step closure seems to be user-provided.

Closed by #48.