net: validate DNSSEC in Go's DNS resolver

DNSSEC is being deployed.
We should support it eventually.

I am working on a proposal for adding DNSSEC to Go. DNSSEC validation requires multiple queries up to the DNSKEY RRSet of root.
There are a few solutions I have considered for the proposal. All solutions will have x/net/dns/dnsmessage adding necessary RRs and a new package, x/net/dns/dnssec, to perform validation. Any solution will also have to support EDNS0 in case of DNSSEC queries. The three possible solutions include:

Vendor dnssec and add exported functions to net such as LookupIPAddrSecure, LookupMXSecure, etc.

Add a LookupBytes function returns the byte slice of the DNS response for the caller. The net package still checks query's Header and Question. The call will have to re-parse the byte slice but can skip over the question. From exchange() up the call stack, a []byte return would need to be added.

LookupRRSet function which uses duplicate structures defined in the net package. The dnssec package can take net package's structures and transform them back to dnsmessage package. Using the dnsmessage structures is not an option since the caller's dnsmessage package is different than the vendored package.

#1 is the cleanest if the extra dependency is acceptable. #2 is clean for unix, but ugly for Windows. Windows 7 and up can support DNSSEC lookups, but the bytes of the DNS response would have to be reconstructed. #3 has duplication of code and two copies made of the unpacked response.
I would lean towards #1, but do not know how serious the dependency issue is. If the dependency issue is a deal-breaker, then I can make a proposal for #3. The proposal would target Go 1.13's release cycle.

Vendoring dnssec is OK; we've already got dnsmessage in vendor/golang_org/x/net/dns/dnsmessage.
Why LookupIPAddrSecure?  Why not just make LookupIPAddr use DNSSEC if available?

The most immediate reason is systemd-resolved's stub resolver (the <IP_ADDRESS> /etc/resolv.conf server) always strips out DNSSEC records. On my own computer (Ubuntu 18.04), "dig . DNSKEY" gave an empty result while "dig @<IP_ADDRESS> . DNSKEY" gave the expected result. I had to disable systemd-resolved to get a DNSKEY lookup. Ubnutu Bug Report
To soft-fail the lookup routine, one way is to have always require valid DNSSEC if DNSKEY/RRSIG for root is successful. It is unlikely to have records stripped out for other queries if the root DNSKEY lookup is successful. To hard-fail, what about adding a parameter such as "DNSSECStrict" to Resolver? Hard-fail means either getting validation of either NSEC of a DS record or full validation.
I would also like a LookupTXTSecure function in particular, so the caller could know the TXT record was verified. Verifying TXT records in particular has a use for the [MTA-STS proposal] (https://tools.ietf.org/html/draft-ietf-uta-mta-sts-21) and for DMARC/DKIM records. DNS-01 challenges with ACME could also use knowledge of whether TXT records are secure, though CAs would also need a CAA record lookup. LookupTXTSecure could be added later though, in a different issue and CL.

@mwwaters,
I'm not sure what's the goal of this issue but enabling security-aware DNS stub resolver as defined in RFC  4033 sounds reasonable, I mean, not entering the territory of DNSSEC-based technologies such as DANE, DSO or other fancy features from IETF dnsop-wg.
As you mentioned, a small start is always preferable, for example, 1) add a few RRs to dns/dnsmessage package of x/net, 2) create dns/dnssec package of x/net for implementing a validator for stub resolver, 3) stop, take a breath and re-consider adaptation to the standard library; API surface (e.g., why not extending net.Resolver?), dependency management (especially for cipher suite packages), etc. I feel like having dns/dnslookup package of x/net is not so bad.
@iangudger, thoughts?

I am all for adding DNSSEC and other features to both x/net/dns and the standard library. The approach listed by @mikioh sounds reasonable to me.
That said, do @mikioh or @bradfitz (or someone else?) have the bandwidth to review changes like this?

@mikioh
I should have been more clear that any changes within net would come after
dns/dnsmessage and dns/dnssec are built out. Those changes are relatively
straightforward compared with changes touching the API.
In RFC 4033, a validating security-aware stub resolver needs to get the
DNSSEC records. There are difficult issues with changing the behavior of
Go's resolver, given how DNSSEC will fail for many installations.
@iangudger
I assumed there wouldn't be time for code review until the next release
cycle. The process could start with new RRs for dnsmessage this cycle
though.
On Wed, Oct 3, 2018 at 3:36 AM Ian Gudger<EMAIL_ADDRESS>wrote:

I am all for adding DNSSEC and other features to both x/net/dns and the
standard library. The approach listed by @mikioh
https://github.com/mikioh sounds reasonable to me.
That said, do @mikioh https://github.com/mikioh or @bradfitz
https://github.com/bradfitz (or someone else?) have the bandwidth to
review changes like this?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/13279#issuecomment-426539614, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AKOIc680fmOrNJRFWMKbEVbQu8PnRCKiks5uhGkNgaJpZM4GjNFX
.

is it still a thing? any chance to have it as a part of std lib?

Probably this assumes that we will have to put the KSK keys in the binary, but I don't think that we will be able to implement RFC 5011 in the net package here (for KSK rollovers).

@mateusz834 how do external resolvers solve the KSK key issue? (I would assume the key might be configurable somewhere on the system, similar to TLS root certificates.)
I think the main reason to implement the validation in the Go DNS resolver is to prevent issues with MITM attacks between external resolvers and the application. See https://www.rfc-editor.org/rfc/rfc4033#section-7

@PSanetra
They implement RFC 5011.

I would oppose doing DNSSEC validation in the stub library. DNSSEC requires more records with larger responses. To have decent performance, it needs localhost cache. Which should be doing the validating and create resulting AD bit in reply.
What is more important is proper EDNS0 support, to be able to handle larger responses than legacy 512 bytes. That is what stub library has to support. Related to https://github.com/openshift/cluster-dns-operator/pull/276.

@pemensik Side note:

What is more important is proper EDNS0 support, to be able to handle larger responses than legacy 512 bytes. That is what stub library has to support.

The go resolver by default uses EDNS0 now (since go 1.19, #301fd8ac8b)

Nice! There are few networks where this causes issues. It would be nice to have that configurable and make possible to send queries without EDNS0. Good resolvers would handle it better, but bad ones are still present. Could it read  options edns0 from /etc/resolv.conf for example? Recent glibc also added trust-ad option, that does not have equivalent?

We've been advertising EDNS0 in our DNS packets since the Go 1.19 release (https://go.dev/doc/go1.19#minor_library_changes, in the entry for the net package).  We have not seen any reported problems.  We can add a knob if necessary, but we would prefer to only add a knob if it is truly needed.

@pemensik

Recent glibc also added trust-ad option, that does not have equivalent?

We add the AD bit to queries when trust-ad is present in the resolv.conf file (this is what glibc does), ad bit is not used in responses in any way.

@mateusz834

We add the AD bit to queries when trust-ad is present in the resolv.conf file (this is what glibc does), ad bit is not used in responses in any way.

Oh, this is not true by far. If you have running dnssec-trigger+unbound or any other DNSSEC validating resolver on localhost, each client does not need to double-verify already verified signatures. AD bit is set only when that name were verified by cache.
That makes dig example.com report AD bit, but dig google.com do not. AD bit is quite useful if you have trusted channel to validating resolver. Which running it on localhost is, alternative is using DoT or DoH. Which is why trust-ad is needed. Plain UDP forwarding to <IP_ADDRESS> should not consider AD bit valid, because it crosses unprotected network.
I think ssh uses just AD bit to consider SSHFP records verified. That is okay.

We've been advertising EDNS0 in our DNS packets since the Go 1.19 release (https://go.dev/doc/go1.19#minor_library_changes, in the entry for the net package). We have not seen any reported problems. We can add a knob if necessary, but we would prefer to only add a knob if it is truly needed.

There were strange behaviour reported on systemd:

https://github.com/dns-violations/dns-violations/blob/master/2018/DVE-2018-0001.md
https://github.com/systemd/systemd/pull/27078

It should never be required for proper implementations. But there might appear some broken software, which might require turning it off.

Is there any Go-based DNSSEC validating implementation already? coredns does not seem to implement that. Is there any cache written in Go, which implements that? Reinventing a wheel with DNSSEC is a bad idea. That protocol is not simple to implement well.

@pemensik https://github.com/golang/go/issues/13279#issuecomment-1615887376
I don't get your point here. We are not validating DNSSEC in any way in the go resolver. We set the AD bit in the query when trust-ad is present, mostly because there are DNS resolvers that don't perform DNSSEC validation when they don't get the AD bit in the query (https://doc.powerdns.com/recursor/dnssec.html#process).
We are not exposing the AD bit in responses in any way, just setting the AD bit in the query (this is even what the dig command does by default).

Exposing AD bit present in response via generic API would be more useful than implementing whole validation chain IMO. Especially in low level framework. Information whether the result is signed (and verified) should be provided by special service like Powerdns Recursor. Simple boolean check that the response were validated should be enough. If that is not yet possible, please add that.

Exposing AD bit present in response via generic API would be more useful than implementing whole validation chain IMO.

True, but the net package should work well cross-platform, but there is no way to get the AD bit through the windows DNS APIs, secondly the net package does not use the DNS all the time, for example LookupHost,  LookupAddr responses might come from /etc/hosts file, or even through glibc getaddrinfo/getnameinfo, this might or might not use the DNS (depending on the nss modules), i am not aware of any way to get the AD bit from getaddrinfo/getnameinfo.

I think the only way to obtain AD in response is using res_nquery or res_nsearch from libresolv of glibc. Or similar DNS-only library. It kind of make sense, since AD makes sense only for DNS specific results. Unfortunately getaddrinfo and getnameinfo do not pass that information to userspace. I have looked inside glibc, there is no flag for it prepared.
If other platforms do not export that or it does not make sense like /etc/hosts file source, then false is good value. Since getaddrinfo with AF_UNSPEC makes two queries, it would have to combine two flags somehow.

It should never be required for proper implementations. But there might appear some broken software, which might require turning it off.

Let's see if such broken software appears.  Let's not fix it proactively.  As the release notes say, there is already a workaround.


Question on C interoperability

Hi,
I am looking for a small LISP-like language to embed into a C++ project and am interested
whether femtolisp fits the bill. My requirements specifically are these

Define opaque objects
Call C function on these objects

For objects, void * would be enough, but if I can make these proper citizens, it would
be much appreciated.
Is there documentation/examples/advice on how to do this? I know of the use in Julia, but
there no new datatypes are defined.
I also found the related cvalue interface, and can define some values with that, but how to
use struct (define struct, access members) and functions?
Cheers,
Thorsten

Please take a look at define_opaque_type  in types.c. For example, the iostream type is defined using it.
The representation of Scheme values is laid out at the top of flisp.h. Look at value_t and TAG_* and the rest.

Sorry, Lisp values :p Well, FemtoLisp almost passes for a Scheme implementation anyway.

Thank you very much, that's what I needed.


error when starting up mastermind in simple mode

Steps to reproduce the expected behaviour
Starting mastermind up with this command...
sudo mastermind --response-body "$(pwd)/13.json"  http://content.ott.sky.com/v3/brands/go/devices/kids/navigation/nodes/1e3c534e0b9ff410VgnVCM1000000b43150a____?represent=(child/node)

causes this error in mastermind...
Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "mastermind/__init__.py", line 127, in main
KeyError: 'url'
mastermind returned -1

This happens  whether is the URL is escaped or not
Mastermind Version: v1
Operating System: OS X 10.11.4

@concertman the simple mode expects two flags: --response-body and --url so in your case, you need to:
sudo mastermind --response-body "$(pwd)/13.json"  --url http://content.ott.sky.com/v3/brands/go/devices/kids/navigation/nodes/1e3c534e0b9ff410VgnVCM1000000b43150a____?represent=(child/node)

That said, the error should be clear and never a python exception so I'll keep it as an open bug.

@arnau looks like we need to fix the documentation for this as well. It's missing the --url flag.
https://github.com/ustwo/mastermind/blob/master/docs/simple-mode.md

I'm working on it on https://github.com/ustwo/mastermind/tree/bugs/34-simple-mode

Closed by 552475a


d3d11 bindings for vs and ps creation

Adds the nessesary bindings to create pixel and vertex shaders on d3d11.
Also adds naughty bindings for d3dcompiler

Thanks!
Would be awesome to also have bindings for dxc (https://github.com/microsoft/DirectXShaderCompiler/blob/master/include/dxc/dxcapi.h). I plan to do it in the future (I will need it for hot shader recompilation) - but PR is very welcome :)

Cool! Yeah dxc would better.
I will give you a heads up if I get to dxc bindings, so we don't duplicate effort there :)


Clarify if and, or, xor, and not,  are bitwise or logical

At the end of section 3 (Bytes), there is the following quote:

There are also attributes for bitwise operations: ★and, ★or, ★xor, ★not, ★left, and ★right.

Which implies that not is the bitwise complement operation, and that and/or/xor also operate bitwise.
For example, not 00-00 would be FF-FF.
However, on the same page, towards the end of subsection 4.2 (Bool), there is the following

The not attribute is a Boolean inversion. Attributes and, or, xor are logical conjunction, disjunction, and exclusive disjunction of the object itself with an a array of other Boolean values.

Which implies that these operations are logical (meaning they operate only on true/false values).
For example, not of 00-00 would be 00-01.
Which of these variants is the correct one?
And as an additional question, what is meant by "an a array of other Boolean values"? Do these operations accept an array as an argument or just one other value?

@maxonfjvipon can you please help

@aabounegm

we don't have bool anymore, there are true and false objects
bytes.and and true.and are different objects. bytes.and is bitwise "and", true.and is logical "and"

Yes I remember that blog post. So it's safe to assume that all of those operators (with their respective atoms listed here) are bitwise, correct?

@aabounegm and, or, xor, not - yes, bitwise

Understood, thank you. Feel free to close the issue or keep it open until the paper is updated


While Dragging Component it ContextMenu got Triggered itself

Version
1.9.1
i am using react-dnd for managing drag and drop in my project and i am using context menu also but when dragging starts  context menu got triggered dont know why ...
here what happening.

have your tried holdToDisplay value as -1?

Thanks It worked


Underfloor Heating Thermostat "no response"

I have just had electric underfloor heating installed with a Wifi thermostat. The thermostat is controlled using the Tuya Smart app but for some reason it's showing "no response" in Homekit/Homebridge.
I also have some electric radiators connected via a different wifi thermostat, also controlled using the Tuya Smart app which is working fine in Homekit/Homebridge.
Any ideas what might be causing this new wifi thermostat to be unresponsive? I'm just surprised that it works in the Tuya Smart app but not then via Homebridge to HomeKit.
Thanks,
Anthony

Hi, any update on this please?


Markdown rendering - Horizontal Rule

When rendering a horizontal rule, the rule is not displayed and all normal text prior to the rule is rendered as H2.
The rendering is the using webchat, portal.azure.com, the emulator and the twilio channel.
Below is the code to created the message with simple markdown and I've attached a screen shot of the emulator window.
Also, is this the correct project to bring up rendering issues?  I have some others that are twilio specific.
Thanks in advance.
Mike
public static IMessageActivity BuildLineBreakBug(ITurnContext turnContext, bool addLine)
{
    var text = new StringBuilder();

    text.AppendLine("Line 1 **Bold text**");
    text.AppendLine("Line 2 Normal text here.");
    if (addLine)
    {
        text.AppendLine("---");
    }
    text.AppendLine($"Line 3 - Line break between 2 and 3 = {addLine}");
    text.AppendLine("Line 4 - Normal text here.");

    var response = turnContext.Activity.CreateReply(text.ToString());

    return response;
}

You're correct. This does appear to be a bug and needs to be fixed. In the meantime, you can accomplish what you're looking for using *** (asterisks) or ___ (underscores).

As it turns out, this is not a bug.
You might be more familiar with using --- for horizontal rule because many utilities (GitHub, included?) follows Don Gruber's original specs for Markdown, which came out way back in 2004. Bot Framework's WebChat, which is used to render WebChat and emulator, uses markdown-it, which uses the continually-updated CommonMark spec. It treats --- and === as a Settext Heading and makes the text on the previous line a heading.


[IRGenSIL] Force early materialization of liverange-extended values

At -Onone, we may opt to extend the liverange of a value by inserting
fake uses of the value in blocks which postdominate its def. This keeps
values available throughout the entirety of a function when stepping
through it in a debugger.
This patch makes it so that we also insert a fake use at the start of a
value's lifetime. This forces early materialization of the value, i.e it
becomes available for inspection in a debugger as soon as it's defined.
This is useful when a value has no uses except fake uses. In such cases,
ISel may (correctly!) decide to materialize the value at the end of its
scope. This is really unintuitive and unhelpful for users, because you
need to be done stepping through the value's scope before you can
inspect it.
For the same reasons, forcing early materialization is also useful when
a value does have uses. Consider:
  let d = ...
  print("Here is the value of d: \(d)")

Without early materialization, if you set a breakpoint on the line
containing the print, you would not be able to inspect "d". That's
because "d" would be materialized right before string interpolation
happens, which is after the breakpoint PC.
rdar://38465084

@swift-ci smoke test

@swift-ci smoke test

LGTM, sorry for the late review.


Load Reporting back to using metadata

No significant performance differences

This needs a mechanism to include the load reporting filter or not.

@ctiller at compile time? There's https://github.com/grpc/grpc/blob/master/src/core/ext/filters/load_reporting/load_reporting.c#L77 for runtime.

No significant performance differences

Issues: #10576 #9542
@ctiller can you PTAL?

Issues: #9542 #10576


7 Tests fixed. Issue #349

https://github.com/steemit/steem-js/issues/349

How long it takes reviewers with write access to approve sth?
I am slowly thinking about forking library...


[Composer] Filter out platform packages

Hi! Currently confused outputs "ext-" as unsafe packages. You may probably want to filter them out. You may want to check "lib-" prefix or "hhvm" as well: https://getcomposer.org/doc/01-basic-usage.md#platform-packages
Example output:
Issues found, the following packages are not available in public package repositories:
 [!] ext-readline
 [!] ext-zip
 [!] ext-redis
 [!] ext-mongodb
 [!] ext-zlib
 [!] ext-mysqli
 [!] ext-json
 [!] ext-curl
 [!] ext-libxml
 [!] ext-zend-opcache
 [!] ext-amqp
 [!] ext-iconv
 [!] ext-simplexml
 [!] ext-ast
 [!] ext-openssl
 [!] ext-apcu
 [!] ext-pdo
 [!] ext-calendar

Thanks for opening the issue! I'll dig into this a bit more and figure out the proper action. I'll have to make sure that these are not claimable in the Composer repositories (which I assume is the case) etc.


Suggestion: Calculator - decimal point preferences

Hello everyone,
being from Europe means I use a comma as decimal point instead of the American/Asian dot.
A cool feature would be to allow the user to choose their prefered decimal point in the preferences or even allow for both directly during input.
Cheers,
Stefan

@stefanpawlik implemented in https://github.com/hainproject/hain/pull/268 , currently pending code review for inclusion in the next release

Now in develop,


Reproduce conflicting stream issue

WIP

After adding a delay the tests now pass I don't think this is a bug.

Appears to just be an iOS bug closing again.


Feature Request: Memcache Support

So it may already have this and I just haven't figured out how to set it up. At the moment, it looks like BSA is using file caching written to bootstrap/cache using options in the .env file. I looked at Laravel's memcache setups but they didn't seem to feature the .env file itself in the reading material I perused.
It'd be nice to have it support Memcache though, in order to allow for HA and session saving. PHP provides this via this sort of system. Using memcache in combination with unison ( just sync file stores every 5 mins with cron; gets around needing something heavy duty like ceph / gluster or offsite like Amazon ) is how I manage HA on all my other PHP apps.

Memcache support is baked into laravel so it'll just be a case of exposing the config to be editable via the .env file. I'll get this in the next release.
If you're urgent for this feature you can currently set the following variables in the .env file:
CACHE_DRIVER=memcached
SESSION_DRIVER=memcached

Then edit the config/cache.php file, search for memcached and you can edit the server details. Just beware this will be overwritten when you next update.


Phase 2 features fixes

when no nfts in a collection use an empty array [] instead of null
use CollectionEntity as type for a new field collection in NftEntitycollectionId
add nbNfts field into CollectionEntity

PR #98 implement this enhancements


Flexible toolbar and card toolbar

I haven't found a way to replicate this behaviour:

Has anyone tried to achieve this via properties or custom CSS?
Also, big card headers with images would be great. It's pretty easy to do with CSS, but it'd be cool to have an actual "media" property would be more standard and reusable. Something like this:

Then, we could combine a flexible card with an image with a flexible toolbar, which would be very interesting on desktop apps.
Cheers all!

Oh! So there's CardMedia... that will work!
http://react-toolbox.com/#/components/card


ARROW-1434: [Python] Support datetime64[D] numpy arrays

MSVC still unhappy it seems. I can have a look on my windows machine later if you like

@wesm yes, that would be much appreciated. I don't have much experience with silencing MSVC errors.

OK, warning squelched. Will merge when build passes


increase single yarn application tag to 250

To avoid truncated application_tag with insufficient flow/job description.
Local build pass with test successful.

hey ! Any reason why the test has 350 and the config is set to 250 ? Is it just to test if its going over the set limit of 250 ?

Yes! Just the test purpose, it can be any value over 250


python310Packages.tldextract: 3.2.1 -> 3.3.0

Description of changes
https://github.com/john-kurkowski/tldextract/blob/master/CHANGELOG.md#330-2022-05-04

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

looks like the knockpy changes got rolled into another commit

This change is removed. Thanks.

python310Packages.scapy: disable failing tests

It's called scrapy.


Lots of changes, more tests etc.

Coverage increased (+14.9%) to 86.838% when pulling 99e696463d023008a4255f962fb20a8e4b046a91 on dev into 3c4adac10113ef144424e91523330bf11a00bb1d on master.


job search error

Error while processing route: jobs.show this._initProperties is not a function TypeError: this._initProperties is not a function

I'm closing this issue for now, the entire app has been overhauled (behind the scenes, it doesn't look any different) so it should be a bit more stable now. Please try out the new version and let me know if you have the same issue 👍


Strategy to match on first or on all matches

Thanks to @strategy the matchers will check all, or try the first one and stop finding next matches.
That way You can put more specific matches before these less specific
# Possible values :one, :many. Defaults to nil
@strategy :one | :many | nil

Hey @wende, thanks for the PR! :)
Could you include some tests for the new functionality?


Flash of unstyled content on page refresh

Bug Description
When a page is refreshed, the text on the page flashes between two different stylings

This was already happening with query viewers, but the text flashing is a new issue:

We might be able to fix this by including the font locally rather than downloading them from a CDN?
Last time I checked it took about 400ms to get the fonts, and it was a "blocking" operation in the loading waterfall

No longer present in our deploy previews, closing for now!


Missing info on how to move an On Prem backup from 1 vault to another.

This page doesn't cover the process or procedure when needing to move the backed up data of an on-prem VM from 1 Azure Backup vault to another.
The use case is I had 1 Azure vault for both Backup and Site Recovery.
I then upgraded the Site Recovery to the modern experience which created a new vault.
Now I have 2 vaults and cannot get my onPrem VM backup data moved over to the new vault.
It seems the only way is to completely delete all my data and start over on the new vault which is not acceptable.
Please add that process to this page.
Thanks.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 7c6fd79a-0eac-8f42-22f9-94060d012b93
Version Independent ID: 150470f2-f17b-90bc-2884-c29358357ba2
Content: Manage and monitor MARS Agent backups - Azure Backup
Content Source: articles/backup/backup-azure-manage-mars.md
Service: backup
GitHub Login: @v-amallick
Microsoft Alias: v-amallick

@eric-PBI
Thanks for your feedback! We will investigate and update as appropriate.

@eric-PBI, thank you for the suggestion.
@v-amallick, could you please review this feedback and share your thoughts. Thanks.

@v-amallick Is this a short term freeze? Or long term? I'm needing to get this solved fairly soon, and am relying on this documentation to proceed.
Thanks,

@eric-PBI - Yes, this is a short term freeze, will be lifted very soon. Thanks!

@eric-PBI - Sorry for delay in responde. We are tracking this internally. will update soon.

Hi @eric-PBI -
Moving existing MARS, DPM, MABS backups to new vault is currently not supported. You can choose the Stop-backup with retain data option to retain old backups in the old vault and re-register MARS, DPM, MABS to new vault.
For information about moving recovery points to a new vault for MARS, see this section. We will soon update the document for MABS and DPM.
We are now closing this issue ( #please-close ); if you've other queries, please reach out to us.
Thanks!


Behind Nginx Reverse Proxy

How would you configure the Nginx sever block for reverse proxying to Webhook Tester?

Hmm, as usual... Do you have a problem with that?

I thought I had it working but the sending application is saying a SSL connection could not be established. My other reverse proxy server configs work so I know my certs are fine.
This is the location block I have:
	location / {
		proxy_pass http://<IP_ADDRESS>:8080/;
		proxy_http_version 1.1;
		proxy_set_header Upgrade $http_upgrade;
		proxy_set_header Connection "Upgrade";
		proxy_set_header Host $host;
	}

Do I need to have a generic location / block and then a websocket specific one like location /api/?

I managed to get everything working by doing this:
location / {
	proxy_pass 			http://<IP_ADDRESS>:8080;
	proxy_redirect 		http://<IP_ADDRESS>:8080/ $scheme://$host:$server_port/;
	proxy_set_header Host $host;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-Host $server_name;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header X-Forwarded-Proto $scheme;
	proxy_set_header X-Forwarded-Ssl on;
}

location /socket.io {
	proxy_pass http://<IP_ADDRESS>:8080/socket.io;
	proxy_http_version 1.1;
	proxy_set_header Upgrade $http_upgrade;
	proxy_set_header Connection "Upgrade";
	proxy_set_header Host $host;
}
location /ws {
	proxy_pass http://<IP_ADDRESS>:8080/ws;
	proxy_set_header   X-Forwarded-For $remote_addr;
	proxy_set_header   Host $http_host;
	proxy_set_header Upgrade websocket;
	proxy_set_header Connection Upgrade;
}


Bitcoin: Add transactionHash to BitcoinBlock format

As a recent request there was the need mentioned to add the transaction hash also to the org.zuinnote.spark.bitcoin.block format (it is already added in the org.zuinnote.spark.bitcoin.transaction format).
However, this should be made configurable via the parameter "enrich" (similar to the format org.zuinnote.spark.ethereum.block) due to its performance implications. If true then it is added, if false then it is not. Default is false.
This needs to include integrationTests.

Proposed by 	artyomboyko

published with release 1.1.1


the experiment (1) output files is empty

Hello, thanks for sharing your wonderful codes.
My host is based on Windows system. I try to reproduce the experiment by using the README file in your code experiment : (1) Experiences to verify the fairness of the q-FFL objective, and compare with uniform sampling schemes. However, after I execute the run.sh file, the output is empty. I checked my execution steps and found no problem at present. I hope you can confirm the cause of the problem for me.
Thanks again!

The names of the output files are: fedavg_uniform_run1、ffedavg_run1_q0、ffedavg_run1_q5
See the figure below:

The problem has been solved through screening, thank you and your excellent article!


💡[Feature]: Add Epidemics Predictor

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
The goal is to determine and analyze the spread of an epidemic in specific areas. This can be particularly useful to villages and other areas where healthcare facilities are not readily available. Machine learning models can be built that predict the nature of the spread of an epidemic in an area and also determine where the next outbreak of an epidemic is most likely to occur.
Factors such as geography, climate, demographics, and distribution of an affected area’s population have to be taken into account while training the machine learning model so that you can identify other areas prone to outbreaks.
Use Case
Helps to  predict and manage epidemic outbreaks.
Benefits
No response
Add ScreenShots
No response
Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@pavitraag assigned!


BLE: Simplify device name string creation.

Saw this when looking at https://github.com/lancaster-university/codal-microbit-v2/issues/370, it's a small change that results in the same string, but it saves a bit of flash with gcc v10.3.
The bloatly CI workflow should post here the size diff.
The modelVersion instance is not used anywhere else, disName results in the same string and is not used anywhere else either.

Looks good to me, lets get this merged in.


Allow specifying ignore_failure :quiet to disable the error spew

Directly inspired by https://github.com/sous-chefs/sysctl/pull/127 but more generally people have been asking for this forever so why not?

@coderanger thank you! I literally just ran into an odd situation where I needed a ton of resources to have ignore_failure and many chef runs were noisier than my brain could handle.


⚠ Bofrost.be has degraded performance

In e18c22a, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1492 ms

Resolved: Bofrost.be performance has improved in 0683170.


Startup chat messages

Please would you consider adding an option to disable the startup messages that the mod puts into chat when the game starts/world loads. As a modpack creator, I'd like nothing more than to prevent all the mods in my packs from doing this, as typically, the messages are related to version updates, which players of modpacks have no control over. It can be quite overwhelming for players to be subjected to many many lines of messages when the game loads.

The startup chat message you  are getting from RS is stating you're playing on a beta version that can cause serious damage if you don't take backups. Most people don't realise this, and frankly, you shouldn't be including beta versions in packs.

Understood, thanks.


Possible solution for delay 5 minutes

Hello
https://dev.binance.vision/t/delay-binance/6996/2 I created a discussion on the binance developers forum and I received such a response for delay 5 minutes

that means the token is listed and traded on binance. the ideea of the announcements "Binance will list..." was to learn about a listing before this occurs. Otherwise, the announcement should say "Binance listed ...."


[ROIs] Use FOV ROIs for labeling (at arbitrary pyramid level)

Starting from discussion in

https://github.com/fractal-analytics-platform/fractal/issues/27
https://github.com/fractal-analytics-platform/fractal/issues/46
https://github.com/fractal-analytics-platform/fractal/issues/95

we are now splitting the work on ROIs into several steps/issues:

Parsing pixel sizes and FOV ROIs.
Transform physical-unit ROI into pixels, at arbitrary pyramid level.
Use FOV ROIs for illumination correction (at highest-resolution level).
Use FOV ROIs for labeling (at arbitrary pyramid level). (THIS ISSUE)

Useful resources:

https://forum.image.sc/t/proposal-for-ome-ngff-table-specification/68908/3
https://github.com/kevinyamauchi/ome-ngff-tables-prototype
https://anndata-tutorials.readthedocs.io/en/latest/index.html

This is the slightly more complex that the illumination-correction case, because:

We should read/write at an arbitrary resolution level.
We may have to force a sequential scan of ROIs (this is to be verified).

Other than that, the tentative plan is similar to the illumination-correction one:

Read ROIs from some path, and transform them to indices at an arbitrary level.
Switch from map_blocks to an explicit loop over ROIs which constructs an array by delayed correction functions. In v0, this should be constrained to never act on more than a single ROI at a time.
Write to disk, triggering execution of the correction function for all FOVs. Keep in mind that when working at level>0 there are some additional scale prefactors to introduce - see https://github.com/fractal-analytics-platform/fractal/blob/tasks/fractal/tasks/image_labeling_whole_well.py#L159-L175.

Current status of the per-FOV image-labeling task:

It takes an argument labeling_level (in the same way as image_labeling_whole_well), which can be larger than 0.
It loops over FOV ROIs at this arbitrary resolution level and performs segmentation.
The loop is currently not in parallel - this is something to explore later.
After segmentation, the task writes the mask with the correct scale factors. Since the rescaling operation is shared among the two labeling tasks, it is now obtained with the function rescale_datasets from lib_zattrs_utils.py:

def rescale_datasets(datasets : List[Dict] = None,
                     coarsening_xy : int = None,
                     reference_level : int = None,) -> List[Dict]:
    if datasets is None or coarsening_xy is None or reference_level is None:
        raise TypeError("Missing argument in rescale_datasets")

    # Construct rescaled datasets
    new_datasets = []
    for ds in datasets:
        new_ds = {}

        # Copy all keys that are not coordinateTransformations (e.g. path)
        for key in ds.keys():
            if key != "coordinateTransformations":
                new_ds[key] = ds[key]

        # Update coordinateTransformations
        old_transformations = ds["coordinateTransformations"]
        new_transformations = []
        for t in old_transformations:
            if t["type"] == "scale":
                new_t = {"type": "scale"}
                new_t["scale"] = [
                    t["scale"][0],
                    t["scale"][1] * coarsening_xy**reference_level,
                    t["scale"][2] * coarsening_xy**reference_level,
                ]
                new_transformations.append(new_t)
            else:
                new_transformations.append(t)
        new_ds["coordinateTransformations"] = new_transformations
        new_datasets.append(new_ds)

Note that we explicitly forbid the existence of a global coordinateTransformation, at the moment. We can also decide to allow this feature, and then we need to combine the two scale's (here and also somewhere else - e.g. when parsing pixel sizes).
There a few of relatively simple open points:

Make sure that output is chunked correctly;
Perhaps include the global scale transformation;
Explore parallel writing.
Make sure that image_labeling still works also for 2D (concerning this point: do we actually need to support the per-FOV 2D segmentation? Or should we drop it?)

Sounds great! 👏🏻

Make sure that image_labeling still works also for 2D (concerning this point: do we actually need to support the per-FOV 2D segmentation? Or should we drop it?)

Per FOV 2D segmentation can still be useful, because it allows for 2D segmentation at full resolution, while whole well 2D segmentation needs to be done at a lower resolution for memory reasons.

Perhaps include the global scale transformation;

If it's simple to do, why not. Otherwise, let's wait how transformations change in the v0.5 spec and eventually refactor for that

Explore parallel writing.

There are some cases we know would be save. e.g. at level 0. Is there a way to detect if parallel writing will be save? Also interesting to explore whether it can be safely done in cases where FOV are in overlapping chunks.

Make sure that image_labeling still works also for 2D (concerning this point: do we actually need to support the per-FOV 2D segmentation? Or should we drop it?)

Per FOV 2D segmentation can still be useful, because it allows for 2D segmentation at full resolution, while whole well 2D segmentation needs to be done at a lower resolution for memory reasons.

Ok.
This feature is in-place, and I just tested it on a MIP array. It cannot be tested within Fractal, at the moment, because of our "poor" handling of task arguments (the per-FOV segmentation always acts on the "main" zarr file, and in the current version it cannot be overridden), but it works smoothly from the command-line interface.

Partial update on status:

Make sure that output is chunked correctly.
Done. Chunk sizes correspond to images (e.g. 1x2160x2560), and the image size is obtained by combining FOV ROIs (from the AnnData table) and pixel sizes (from the zattrs file) - both for the highest-resolution level.
Make sure that image_labeling still works also for 2D.
Done (although not as part of Fractal).
Perhaps include the global scale transformation.
This is indeed something simple, but perhaps let's postpone since more changes will come with v0.5. As of https://github.com/fractal-analytics-platform/fractal/commit/273ab078c25c631e209e4b5c1537165caaf212d8, we raise a NotImplementedError with a decent error message when a global transformation is found.
Explore parallel writing.
Ongoing. The first bunch of tests with num_threads>1 just ran through (with reasonable output), even if they included possibly-non-safe parallel writing. Maybe to_zarr already takes care of it? More tests/understanding needed. We may also provisionally change the chunking choices (making chunks very large), to create more edge cases where many processes should write to the same chunk.
Re: Is there a way to detect if parallel writing will be save?
There is a way, of course, but it requires an explicit check that each chunk only hosts a single FOV/ROI. If expected safe operations are only the ones acting on single images, then I wouldn't take the road towards a general automatic check of safety - since this can be achieved directly by requiring that chunks are of the size of images. If @jluethi has in mind something else (like checking that no pair of organoids belong to the same chunk), then the automatic check would be needed.
I'd say let's re-assess after we explore point 4. In the best case, parallel writing is always safe and we can skip this discussion.
Profile memory usage. This is just to make sure that we've not introduced any unexpected large memory usage.

Concerning memory usage, here is the profiling for three single-well cases. TL;DR Everything looks OK.
Note that multi-well examples may lead to a known issue - see https://github.com/fractal-analytics-platform/fractal/issues/109#issuecomment-1188958357.
In all three cases below, segmentation takes place per-FOV, at the highest-resolution level, and with relabeling.
Case 1: 2x2 well with 10 Z planes (num_threads=2):

Case 2: 2x2 well with 42 Z planes (num_threads=1):

Case 3: 9x8 well with 19 Z planes  (num_threads=2):

Awesome progress, very nice to see that memory is stable as well!
A NotImplementedError is harsh, but I guess we don't have good ways to parse warnings so far, right? We probably wouldn't want the whole pipeline to crash if someone specifies extra metadata. So as soon as we think the library may gain broader usage, this would have to go. But fine for now :)

Explore parallel writing.

Sound good. Let's make this the focus. I agree, only if parallel writing is possible, but causes issues in some cases will we need to implement some check as in 5. Eventually, we may also want to check that there is no ROI overlap (e.g. for the organoid case). But let's have a look first at how save the to_zarr is. I like the idea of provoking simultaneous writes, could also do this for some synthetic examples. If to_zarr handles this well, than we only need to worry about actual pixel overlap, not parallel chunk access :)
👏🏻👏🏻👏🏻

Here is some first anecdotal evidence of successful parallel writing.
For a single 9x8 well, I performed per-FOV segmentation at resolution level 4, where a FOV has YX shape 135x160 and a single 2160x2560 chunk may host 256 FOVs (that is, more than the 72 FOVs of the well).
I ran this test with num_threads=10 parallel executions of Cellpose (which is possible, memory-wise, because each FOV has shape 10x135x160), which is a highly "dangerous" operation because all segmentation calls write on the same (single) chunk.
Observations:

The code ran through, with no obvious errors/warnings.
The labels appear reasonable in napari (apart from the fact that they are extremely coarse and irregular, but that is an expected consequence of working at level 4).

This points in the good direction!
We should now find some more stringent tests to be sure that everything is happening safely (maybe we can quickly come up with an artificial test of to_zarr.. let's see).

Oh, this sounds great! Very promising indeed! 🎉

Based on the discussion in https://github.com/fractal-analytics-platform/fractal/issues/122 (and under the assumptions described there), we consider parallel writing as safe - until further notice. This ends the current work on per-ROI labeling, and I'm closing this issue.
Things that are not addressed here:

There is still a broader memory issue which comes up during labeling, but it is not related to the per-ROI scheme -- see https://github.com/fractal-analytics-platform/fractal/issues/109.
Another minor missing point (support for a global coordinate transformation) is not specific for labeling but rather shared among tasks. It can be addressed in the future, the first time that we actually need it. See https://github.com/fractal-analytics-platform/fractal/issues/123.


[v1.0.0-beta.2] @license comments aren't removed

I was trying out the latest beta (v1.0.0-beta.2), but it is not working for me. I also reverted back to v.0.4.6 but that version works.
vue git:(develop) ✗ npm run build

><EMAIL_ADDRESS>build /../vue
> cross-env NODE_ENV=production webpack --config webpack.prod.config.js --progress --hide-modules

/../vue/node_modules/schema-utils/dist/validateOptions.js:40
    throw new _ValidationError2.default(ajv.errors, name);
    ^

false
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>build: `cross-env NODE_ENV=production webpack --config webpack.prod.config.js --progress --hide-modules`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

webpack.config.js
const UglifyJSPlugin = require('uglifyjs-webpack-plugin');

if (process.env.NODE_ENV === 'production') {

  config.plugins.push(

      //Define production environment
      new webpack.DefinePlugin({
        'process.env': {
          NODE_ENV: '"production"'
        }
      }),
      //https://www.npmjs.com/package/assets-webpack-plugin
      new AssetsPlugin({
        filename: 'webpack.assets.json',
        fullPath: false,
        prettyPrint: true,
        path: path.join(__dirname)
      }),
      //Extract CSS from JS files
      new ExtractTextPlugin({
        filename: '[name].[hash].min.css',
        allChunks: true
      }),
      new OptimizeCssAssetsPlugin({
        assetNameRegExp: /\.css$/g,
          cssProcessor: require('cssnano'),
          cssProcessorOptions: {
          discardComments: {
            removeAll: true
          },
          reduceIdents: false,
        },
        canPrint: true
      }),
      new webpack.LoaderOptionsPlugin({
        minimize: true
      }),
      new UglifyJSPlugin({
        compress: {
          warnings: false,
        },
        output: {
          comments: false
        },
        sourceMap: true
      })
  );

  module.exports = config;
}

Versions:
➜  vueV git:(develop) ✗ npm -v
5.3.0
➜  vueV git:(develop) ✗ node -v
v8.1.2
➜  vueV git:(develop) ✗ yarn version
yarn version v0.27.5

@sybrendotinga can your provide minimum reproducible test repo?

I had the same issue, the compress and output options passed to UglifyJSPlugin need to be moved to uglifyOptions, however the plugin should provide a better error message.
new UglifyJSPlugin({
    uglifyOptions: {
        compress: {
          warnings: false,
        },
        output: {
          comments: false
        },
    }
    sourceMap: true,
})

Moving my options to 'uglifyOptions' worked for me.

It seems that @lincense comments are not removed anymore by passing the options trough uglifyOptions. For example:
{return S}),n.d(e,"mapState",function(){return A}),n.d(e,"mapMutations",function(){return G}),n.d(e,"mapGetters",function(){return T}),n.d(e,"mapActions",function(){return V});/**
 * vuex v2.3.0
 * (c) 2017 Evan You
 * @license MIT
 */

@sybrendotinga seems your are right

@sybrendotinga Could you please share your current config?

I bundled my files to one for a better overview. But I hope that I didn't miss anything. This should be it.
const webpack = require('webpack');
const ExtractTextPlugin = require('extract-text-webpack-plugin');
const AssetsPlugin = require('assets-webpack-plugin');
const OptimizeCssAssetsPlugin = require('optimize-css-assets-webpack-plugin');
const UglifyJSPlugin = require('uglifyjs-webpack-plugin');
const LodashModuleReplacementPlugin = require('lodash-webpack-plugin');
const commonPaths = require('./webpack.common-paths');

module.exports = {
  devServer: {
    contentBase: commonPaths(env).publicPath,
    hot: true
  },
  entry: {
    app: commonPaths(env).jsPath + '/main.js',
    vendors: [
      commonPaths(env).jsPath + '/vendor.js'
    ]
  },
  output: {
    path: commonPaths(env).distPath,
    publicPath: commonPaths(env).distPath + '/',
    filename: '[name].[chunkHash].min.js',
    chunkFilename: '[name].[chunkHash].chunk.js',
  },
  devtool: 'source-map',
  performance: {
    hints: false
  },
  module: {
    loaders: [
      {
        test: /\.js$/,
        exclude: /node_modules/,
        loader: "babel-loader"
      }
    ]
  },
  resolve: {
    alias: {
      'vue': 'vue/dist/vue.js',
      '@': commonPaths(env).publicPath,
    }
  },
  plugins: [
    // Extract vendor chunks for better caching
    new webpack.optimize.CommonsChunkPlugin({
      async: true,
      minChunks: Infinity
    }),
    new webpack.optimize.CommonsChunkPlugin({
      async: true,
      children: true,
      minChunks: 2
    }),
    new webpack.optimize.CommonsChunkPlugin({
      name: 'common',
      minChunks: (module, count) => {
        return module.resource && /element-ui|axios|vue-sticky|vue-axios|vuex|vuelidate|kute|domtastic/.test(module.resource) && count >= 0
      }
    }),

    new webpack.optimize.ModuleConcatenationPlugin(),

    //Tree shake unnecessary lodash bundles
    new LodashModuleReplacementPlugin(),

    //Define production environment
    new webpack.DefinePlugin({
      'process.env': {
        NODE_ENV: '"production"'
      }
    }),

    //https://www.npmjs.com/package/assets-webpack-plugin
    new AssetsPlugin({
      filename: 'webpack.assets.json',
      fullPath: false,
      prettyPrint: true,
      path: commonPaths(env).root
    }),

    //Extract CSS from JS files
    new ExtractTextPlugin({
      filename: '[name].[chunkHash].min.css',
      allChunks: true
    }),

    new OptimizeCssAssetsPlugin({
      assetNameRegExp: /\.css$/g,
      cssProcessor: require('cssnano'),
      cssProcessorOptions: {
        discardComments: {
          removeAll: true
        },
        reduceIdents: false,
      },
      canPrint: true
    }),

    new webpack.LoaderOptionsPlugin({
      minimize: true
    }),

    // minify JS
    new UglifyJSPlugin({
      uglifyOptions: {
        compress: {
          warnings: false,
        },
        output: {
          comments: false
        }
      },
      sourceMap: true
    })
  ]
};

Basically this is my UglifyJSPlugin configuration:
    // minify JS
    new UglifyJSPlugin({
      uglifyOptions: {
        compress: {
          warnings: false,
        },
        output: {
          comments: false
        }
      },
      sourceMap: true
    })

And my current version is v1.0.0-beta.2.

Is this issue solved?

@7chandan not yet

@7chandan @evilebottnawi I think it is resolved now? I cannot reproduce it with the latest version. My current UglifyJS configuration is:
new UglifyJSPlugin({
  uglifyOptions: {
    compress: {
      warnings: false
    },
    output: {
      comments: false
    },
    sourceMap: true,
    parallel: true
  }
})

@sybrendotinga thank for answer, feel free to open if problem will happen again


feat: migrate lottie-types project from lottiefiles

The PR contains the migration of the project from lottiefiles organization that has been used internally.
Please read README for more information.
Notes:

changesets - if it is not suitable you can suggest another versioning and changelog tool
we should sync namings with the lottie-web/docs/json because current type names are based on lottie-docs
perhaps we should set prettier standards
should we remove /examples?

Once the PR is approved we can publish and release the v1.0.0 version.

looks great!


After workshop: write blog post about the training

Thanks to all who participated
About the schedule and instructors
X participants from X countries
Some noticeable happenings and feedback
Materials available
Let us know if interested in next iteration

Drafting here: https://hackmd.io/@coderefinery/CR_TTT_blog

From day 2 feedback: collect and summarize also all the resources and tips from participants


NAT handling

Up till now, I have done lots of testing with drachtio-server running in a Docker network along with my other services. External clients could come in and access the server, but it was all enclosed by a private network.
Now I am trying to deploy the Docker setup to AWS. However, when I run drachtio-server in a Docker network on an AWS EC2 instance, I can't getting NATing working correctly. The drachtio-server can't reach my clients. I even tries running drachtio-server directly on an AWS EC2 instance, but it has the same results. The log below is this case without Docker.
drachtio.txt
In this example, my client REGISTERs, sends a SUBSCRIBE, and sends a PUBLISH. We get the 200 OK responses back for both of those. However, the NOTIFY we send from the Node.js/srf side doesn't reach the client. This is after the fix from before where we now send the NOTIFY after the 200 OK is successfully sent (it works well in the private network scenario).
In the log, <IP_ADDRESS> is the public IP from my office and 10.11.33.* is the private IP range. <IP_ADDRESS> is the public IP of the AWS EC2 instance and 10.0.3.* is the private IP range there.
When I send the NOTIFY with the drachtio-srf object, I am using the received and rport parameters from the via[0] line (<IP_ADDRESS>:5070). But from the logs it looks like drachtio-server is trying to send to the private IP of my office network.

Hi there, yes till now I have not dealt with any NAT scenarios.  Let me look into this.  I'll probably create a branch to work on, and hope to have something for you to test this week

Could you retry your scenario with the docker image davehorton/drachtio-server:edge ?  This is not the complete fix that I want to do, but I want to verify if possible whether the change I made sorts out your scenario.

alternatively, build the latest on the 'nat-handling' branch and test with that.  If you are able to test both the nat'ed and non-nat'ed scenarios, that would be helpful

I have done the tests. Please see the log files:
bad_edge_nat.txt
good_edge_local.txt
good_local.txt
good_local.txt is the local network testing from before for reference.
good_edge_local.txt is the local network testing with the edge docker instance.
bad_edge_nat.txt is the nat testing with drachtio running in a docker network on an AWS EC2 instance.
I confirmed that I am pulling the edge branch docker (v0.7.3-rc1-1-gb384a4b). However, that is not your latest commit, since they are failing automatic github builds.

Its strange, in that there are log messages that I put in the code which should be appearing in your log (the bad log) but are not.  I just made one minor logging change, and the docker image (edge label) should finish building in a bit -- could you re-run the nat test with that, so I can see those log messages?
(You can ignore the build failures for now -- there are some post-build tests which for some reason are failing on travis which pass locally that I am looking into)

Ok, I have mixed results but seems in a good direction... For some reason, I guess the Docker drachtio-server:edge tag won't pull from the latest commit, just the latest successful one. So I manually built the images in Docker from the latest commit (v0.7.3-rc1-4-gf942515).
bad_edge_local.txt
good_edge_nat.txt
"bad_edge_local.txt" -> When I tested locally, my client didn't get a 200 OK for the SUBSCRIBE. So couldn't move on from there.
"good_edge_nat.txt" -> When I tried from the AWS side (again, in an AWS EC2 instance running in a Docker container), the NOTIFY was successful! It works there.

yes, it looked like it crashed in the bad-local scenario when attempting to send the 200 OK to the SUBSCRIBE.  I'm sure this is related to my changes.  Any changes you can get a stack trace?

I think I figured out the issue - it looks like it was crashing on a new log statement I put in, in the non-nat case.  I've checked in a fix, when it finishes building can you retest both scenarios?

Sure! It was taking a while to get gdb in to the Docker container. Will leave it in for now...

Alright, I have retested and attached the current logs. Both local and remote/nat work, for the NOTIFY case. I will also start to test session establishment (INVITE) between remote clients (proxied via srf). Do you think your fix will help them, or should I await further adjustments? They weren't working before, with similar behavior to the NOTIFY.
good_edge_local_fix.txt
good_edge_nat_fix.txt

It would probably help the INVITE scenario if you app is a B2B but possibly not if you are a proxy.  Please do retest and attach your logs if it doesn't work so I can have a look

Here is the INVITE case. User 8 registers, then user 9. User 9 attempts to send an INVITE to user 8. It never leaves the server.
rcs_invite_nat.txt
It is a proxy using the srf object. Thanks.

Interesting, the problem there is different.  We try to send the invite via udp, but the sip stack complains the message is too large, and then tries tcp, which won't work to that client:
nta.c:8463 outgoing_try_tcp_instead() nta: INVITE (1) too large for UDP, trying TCP
I will probably have to make a change to fix this, but could you retest that same scenario with this as the destination:
sip:<IP_ADDRESS>:5070;transport=UDP
i.e., lets just see what happens if you explicitly request UDP.

When we set transport=UDP, we don't see any change in behavior. The logs are:
invite_transport_udp.txt
I also tried forcing tcp from the client side by setting mtu size to something very low. This actually works, but the drachtio server can't send ACK because it tries to do so over UDP. For this I removed the "transport=UDP" setting. The logs are:
invite_force_tcp.txt
I tired running drachtio-server with only tcp support (sip:*:15060;transport=tcp), but it crashed when trying to find a candidate for for the protocol (you can see at line 1039 where it starts again). Sorry, I don't have a stack trace. The logs are:
invite_tcp_only.txt
I notice in our client that we aren't setting "transport=tcp" in our contact header or via. We are looking to make sure this is done. Is drachtio-server looking for it?

OK, I appreciate you running these tests.  At this point, with your logs I think I have enough data to recreate something in my environment.  I'm going to first make a build that removes (or increases) the limit on UDP message size that is being triggered in your initial case, as I have run into this as well and I think it is unnecessarily low for most cases.

I just made a commit to increase the mtu over udp to 4096.  This is not the fix but I would like to see what your results are with this build if you have a chance ... not forcing it to udp or tcp, just the original scenario.  I'll probably end up making this configurable, but I would like to push through this initial issue to see what lies on the other side.  Its building now on docker...

Quick question: When I remove my Docker drachtio/drachtio-server:edge image and pull it again, it is still on version b384a4b (from 4 days ago), not the latest commit. Is there some trick to get it to pull from the latest commit? Otherwise, I need to rebuild the image in AWS, which takes time.

Sorry I think that was my problem...I forgot to update the Dockerfile on the branch to refer to the branch, so I think it was kicking off the build for the 'develop' branch.  Just committed that change, building now.

Ok great! I will give it try.

You should see some extra logging at startup along the lines of this:
reset mtu size to: 4096 transport: udp/<IP_ADDRESS>:5060

In these logs, I tried a few sessions to see how things work. The first session had some issue, but it may be an MO issue. At least, you can see the ACK handling for a subsequent session around line 4280. The transport it finds should be <IP_ADDRESS>:30967 I think...
rcs_test_udp.txt

So here is what I see in that log:

we get the INVITE from the client
your app proxies it on, without adding a record-route header
we get the 200 OK from the far end, and pass that back to the caller
now, since you did not add a record route header, it is up to the calling device to send the ACK directly to the far end
since we see retransmits of the 200 OK from the far end, I assume your calling device could not send the ACK to the address of the Contact header in the 200 OK, which is
sip:+14085550007@<IP_ADDRESS>:15000;transport=UDP

so if you want or expect the ACK to go through the drachtio server then you would add a property to the proxyRequest call opts.remainInDialog (alias opts.recordRoute)

I am already setting that option:
srf.proxyRequest(req, dest, {
    forking: 'sequential',
    remainInDialog: true,
    provisionalTimeout: '15s',
    finalTimeout: '30s',
    followRedirects: false
}, (err, results) => {
...
});

Ah, right I missed that, sorry.
So then the question is, why is the sending client not sending the ACK to sip:<IP_ADDRESS>:15060, per the record-route header?  Where, in fact, is it sending it?
Also, why is there a Route header in the initial INVITE?

Hi, we fixed a few issues on the client side such as transport was always set to UDP in the contact even though it was actually TCP and our TCP port was incorrect (which led to some other issues). Also, I have run a local modification of the drachtio-server to set the UDP mtu to 100 so that it favors TCP only. This would be better for us. I will send some fresh logs shortly. We still see the ACK problem, it just doesn't go out.
Do you think the Route header is a problem? I believe it was set as an outbound proxy, but we can disable it.

The Route header is not necessarily a problem for me...I am just proxying it on.  I'm not used to seeing it on a new INVITE, and not sure whether it is valid there.  I guess the main point is that if you are expecting me to strip it, I am not.  Perhaps the presence of that Route header is a problem for the client, and the reason it is not sending the ACK out?
I'm not sure whether the results of dropping the mtu down to 100 are going to be great for you....it will cause tcp top be used for every outbound message.  If you really just want to use tcp only, why dont you just start the drachtio server with only a tcp endpoint (/contact)?

Good question, I think the issues we had from our side were forcing some UDP, so it never worked. But now I will test the scenario again with TCP only and post the log here. Thanks.

tcp_log_noroute.txt
tcp_log_route.txt
These two logs show the same scenario with route header in and not in the Invite. So I think you can focus on the noroute version. Either way, the behavior is the same.
Here, I have set the server to use the edge of the nat-handling branch (no changes from me), but I have set the contact for tcp only. It doesn't get very far. The INVITE comes, 100 Trying is sent, but it is not forwarded/proxied. I can see the log entry on line 796 where it is searching for "udp/sip:<IP_ADDRESS>:15000". I am not sure why it is looking for UDP.

tcp_hardcoded_proto.txt
In this log, I hardcoded proto to "TCP" in findAppropriateTransport() in sip-transports.cpp and it is able to proxy the INVITE to the other client. There is still the ACK problem though.
I see that drachtio-server supports keep-alive packets (/r/n/r/n) which is great. This will help later.

ok, I will fix the issue where it did not select the tcp transport on its own.
What I see in the tcp_hardcoded... file is that the B party puts an address of sip:+14085550007@<IP_ADDRESS>:17777;transport=TCP in the Contact header of the 200 OK.
So the 200 OK goes back to A with a Record-Route of the proxy and that Contact header for B.
A properly sends an ACK with the Route header of the proxy, and request-uri of the contact above.
So now the proxy properly removes the Route header, and then tries to connect to <IP_ADDRESS>:17777 via TCP.  It looks to me like it can not connect to that address, and that is why the INVITE does not go out.
So I presume that <IP_ADDRESS> is nat'ed and the public address is <IP_ADDRESS> ?
Perhaps you are expecting the proxy send the ACK to <IP_ADDRESS>, but I'm not sure that would be right.  I dont think the proxy can make an assumption that <IP_ADDRESS> is reachable via <IP_ADDRESS> simply because that is what came back in the Contact header of the 200 OK.  In other words, it is a legitimate use case that a proxy should route an ACK down a different route than the original INVITE.  Your scenario would work if I broke that rule and just routed the ACK the same as the INVITE, but I don't want to do that (globally) since it would break other scenarios
Hmm.  the knowledge that B is nat'ed behind <IP_ADDRESS> is really something discovered at the time B registers.  But right now I am not keeping an internal mapping of all nat'ed endpoint addresses that have registered.  So when the ACK arrives to get routed the proxy does its normal thing.
I'm sure I can solve it, but I need to give it some more thought I guess on what is the right way to do so.
By the way, do you have an opinion on whether you want your app to be a proxy versus a back-to-back user agent?  Most nat scenarios that I see are combatted primarily using a B2B, which gives the app more direct control of the routing.

Hi, we are using the srf app to act as simple registrar and router between clients. It is mainly for testing, so we thought it could just proxy the requests from client to client. Are you suggesting maybe we could implement it as a b2bua (as in drachtio-b2b-media-proxy)?
To answer your other question, yes, <IP_ADDRESS> is nat'ed to the <IP_ADDRESS> public address. In our app, when we receive the REGISTER, we maintain the rport and received IP from the via which are the public IPs. We provide this as the destination when we proxy the INVITE. That is a little of the confusion, why doesn't the INVITE get forwarded to that that IP?
I will look at the b2bua support in srf.

Thank you for your help. I would like to close this issue. A B2BUA seems like a desirable solution.


Add theme: PywalZen

Add theme: PywalZen
This PR adds a new theme to the theme library.
Theme Details

Name: PywalZen
Description: Enable correct support for Pywalfox with this theme!
Homepage: https://github.com/Axenide/PywalZen
Author: @Axenide

Thanks!


Conduit

Issue Description:
Conduit Facade are clear and show nothing no mater what you paint them.
What happens:
Place block and it goes away but is still there.
What you expected to happen:
For it to hide the conduit and show the block.
Steps to reproduce:

Place Conduit
Place Facade
That's all.
...

Affected Versions (Do not use "latest"):

EnderIO: 3.1.192
EnderCore: <IP_ADDRESS>
Minecraft: 1.10.2
Forge: <IP_ADDRESS>7

Your most recent log file where the issue was present:
There Is no crash log. But here is a pic https://drive.google.com/file/d/0B2IewM42X78HaTBzODZCQkVDakE/view?usp=sharing

#4261

@HenryLoenwind I even search this time Sorry about that and thanks for the link.


Xcert upgrade proposal

This is a very quick proposal that adds:

Xcert allows a single asset by exposing the convention serial number.
Xcert includes metadata for use in atomic swaps.

Further discussion is expected.

Upgrades:

removing chainable Xcert
replacing proof with data (hash)
erc165 for Xcert conventions


New Theme: Neovim

New Theme
I created a port of the Dracula color scheme for Neovim. Check it out.
I haven't deleted the samples folder yet, so you guys can see if everything is okay with the highlights groups.
https://github.com/maxmx03/dracula.nvim

Can you expand on how it's different from dracula/vim (and please don't say the only difference is Lua)?

Can you expand on how it's different from dracula/vim (and please don't say the only difference is Lua)?

Of course, it's not just because of Lua. One significant difference is that the newer version of Neovim incorporates semantic highlighting, which can enhance the overall highlighting experience.

You can read more here: https://gist.github.com/swarn/fb37d9eefe1bc616c2a7e476c0bc0316

Got it. Is the semantic highlighting colorscheme code something that can be added to dracula/vim ? We already have some treesitter support, for example.

Yes, you can add semantic highlighting to Dracula/Vim by adding the necessary code to your lua/dracula/init.lua file, or just write everything in vimscript
  local c = {
     fg = "#f8f8f2",
     bg = "#282a36",
     float_bg = "#23252f",
     currentline = "#44475a",
     comment = "#6272a4",
     cyan = "#8be9fd",
     green = "#50fa7b",
     orange = "#ffb86c",
     pink = "#ff79c6",
     purple = "#bd93f9",
     red = "#ff5555",
     yellow = "#f1fa8c",
     cursor_fg = "#282a36",
     cursor_bg = "#f8f8f2",
     sign_add = "#50fa7b",
     sign_change = "#ffb866",
     sign_delete = "#ff5555",
     error = "#ff555b",
     warning = "#ffb866",
     info = "#8be9fd",
     hint = "#8be9fd",
     other = "#bd93f9",
     blended_orange = "#483F3E",
     blended_red = "#48303B",
     blended_cyan = "#374754",
  }

  local hl = vim.api.nvim_set_hl

  -- TreeSitter
  hl(0, "@comment", { fg = c.comment, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.comment, bg = 'NONE' })
  hl(0, "@error", { link = 'Error' })
  hl(0, "@operator", { fg = c.pink, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.fg, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0, "@string", { fg = c.yellow, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.red, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0, "@character", { fg = c.yellow, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0, "@function", { fg = c.green, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.green, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.green, bg = 'NONE' })
  hl(0, "@method", { fg = c.green, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.green, bg = 'NONE' })
  hl(0, "@constructor", { fg = c.pink, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ link = '@type' })
  hl(0<EMAIL_ADDRESS>{ link = '@type' })
  hl(0, "@parameter", { fg = c.orange, bg = 'NONE', italic=true, })
  hl(0, "@boolean", { fg = c.purple, bg = 'NONE' })
  hl(0, "@number", { fg = c.purple, bg = 'NONE' })
  hl(0, "@float", { fg = c.purple, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.cyan, bg = 'NONE' })
  hl(0, "@exception", { link = 'Exception' })
  hl(0, "@type", { link = 'Type' })
  hl(0<EMAIL_ADDRESS>{ link = '@type' })
  hl(0<EMAIL_ADDRESS>{ fg = c.cyan, bg = 'NONE', italic=true, })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE', italic=true, })
  hl(0<EMAIL_ADDRESS>{ link = 'Type' })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0, "@storageclass", { fg = c.pink, bg = 'NONE' })
  hl(0, "@attribute", { fg = c.pink, bg = 'NONE' })
  hl(0, "@field", { fg = c.fg, bg = 'NONE' })
  hl(0, "@property", { link = '@field' })
  hl(0<EMAIL_ADDRESS>{ fg = c.cyan, bg = 'NONE' })
  hl(0, "@variable", { fg = c.fg, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.purple, bg = 'NONE' })
  hl(0, "@constant", { fg = c.purple, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.purple, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.purple, bg = 'NONE' })
  hl(0, "@namespace", { fg = c.fg, bg = 'NONE' })
  hl(0, "@symbol", { fg = c.purple, bg = 'NONE' })
  hl(0, "@text", { fg = c.yellow, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.orange, bg = 'NONE', bold=true, })
  hl(0<EMAIL_ADDRESS>{ fg = c.green, bg = 'NONE', italic=true, })
  hl(0<EMAIL_ADDRESS>{ fg = c.yellow, bg = 'NONE', strikethrough=true, })
  hl(0<EMAIL_ADDRESS>{ fg = c.cyan, bg = 'NONE', bold=true, })
  hl(0<EMAIL_ADDRESS>{ fg = c.green, bg = 'NONE', italic=true, })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.cyan, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.pink, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ link = 'Todo' })
  hl(0<EMAIL_ADDRESS>{ link = 'WarningMsg' })
  hl(0<EMAIL_ADDRESS>{ link = 'ErrorMsg' })
  hl(0<EMAIL_ADDRESS>{ link = 'DiffAdd' })
  hl(0<EMAIL_ADDRESS>{ link = 'DiffDelete' })
  hl(0, "@tag", { fg = c.pink, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.green, bg = 'NONE' })
  hl(0<EMAIL_ADDRESS>{ fg = c.fg, bg = 'NONE' })

  -- semantic highlight
  hl(0<EMAIL_ADDRESS>{ link = '@namespace' })
  hl(0<EMAIL_ADDRESS>{ link = '@type' })
  hl(0<EMAIL_ADDRESS>{ link = '@type' })
  hl(0<EMAIL_ADDRESS>{ link = '@type' })
  hl(0<EMAIL_ADDRESS>{ link = '@type' })
  hl(0<EMAIL_ADDRESS>{ link = '@structure' })
  hl(0<EMAIL_ADDRESS>{ link = '@parameter' })
  hl(0<EMAIL_ADDRESS>{ link = '@variable' })
  hl(0<EMAIL_ADDRESS>{ link = '@property' })
  hl(0<EMAIL_ADDRESS>{ link = '@constant' })
  hl(0<EMAIL_ADDRESS>{ link = '@function' })
  hl(0<EMAIL_ADDRESS>{ link = '@method' })
  hl(0<EMAIL_ADDRESS>{ link = '@macro' })
  hl(0<EMAIL_ADDRESS>{ link = '@function' })
  hl(0<EMAIL_ADDRESS>{ link = '@constant' })
  hl(0<EMAIL_ADDRESS>{ link = 'Italic' })
  hl(0<EMAIL_ADDRESS>{ link =<EMAIL_ADDRESS>})

and then importing it to your colors/dracula.vim. Please note that this will only work for Neovim users.
if has('nvim-0.8.0')
lua << eof
require('dracula')
eof
endif

I can see where you're coming from, but we already have some NeoVim support. Ripping it out now and suggesting people use a different plugin would cause a lot of unnecessary churn, to me; worse, maintaining it in the face of a different plugin would lead to lots of questions about the discrepancies. Heck, having 2 different plugins is likely to lead to that.
Perhaps @dsifford has thoughts as both dracula/vim maintainer and (IIRC) NeoVim user?

Neat! I would encourage you to consider contributing back to dracula/vim instead of trying to maintain two parallel themes… it would be great to see the best of both worlds in one place, I think.

Hi, I get that you wanted to have everything in a single repository, but I believe it would be better to have separate themes for each text editor. That way, you don't have to worry about Neovim and its plugins. I think it would be easier for Dracula/Vim if they focused solely on improving the Vim syntax instead of trying to support Neovim

I completely agree with this take. The current dracula/vim theme, lacks a lot of support especially when you lazy load plugins (cmp for example), also the last issues in the dracula/vim are mostly about neovim.
Why not have different repository?  They are different editors.
This way I also think it will help a lot the neovim community because this is written in lua hence it would be a lot easier to contribute and read the code.

The current dracula/vim theme, lacks a lot of support especially when you lazy load plugins (cmp for example)

I've been meaning to try to address this, but it's not trivial to detect when a plugin is loaded in the middle of the editor running. I think the solution is for dracula/vim to unconditionally link plugin related groups? cf. https://github.com/dracula/vim/pull/301 (And why lazy load anyway? If most code is behind autoload, it shouldn't be a big deal, but maybe I misunderstand the purpose of lazy load?)

also the last issues in the dracula/vim are mostly about neovim.

To me this indicates interest in improving dracula/vim to include NeoVim compatibility, not necessarily interest in splitting off entirely.

Why not have different repository? They are different editors. This way I also think it will help a lot the neovim community because this is written in lua hence it would be a lot easier to contribute and read the code.

How different the editors are is debatable, given the compatibility focus for many major features.
The actual highlighting code is almost the same in Lua as in Vimscript, and the layout of all the Lua files is harder to follow if you aren't used to the Lua+Vim structure… Stringly-typed APIs like the one in this link are also not the easiest to use, IMO.
What is being done in Lua that couldn't be done clearly in the original?
From poking around, the major differences I see are

the "soft" variant (what is this?)
the initialization code is more complicated because of Lua
the require/Lua organization practically re-invents all of the existing :colorscheme name mechanism (which just executes the corresponding file), switching it for a setup method

I don't want to be hostile to change. I just believe fragmenting things is the wrong way to go, and getting a dedicated group together to improve the state of dracula/vim would be better than two independent groups trying to keep up with each other in ad-hoc ways. I'm sure @dsifford and I are willing to take on more maintainers with a NeoVim focus (my focus is typically very Vim, since that's what I use, but I try to review NeoVim-related things where I can).

the "soft" variant (what is this?)

I had to add that to the VSCode theme because of the overwhelming amount of issues related to how macos renders colors. Soft just reduces the saturation of the colors to make them appear more normal on mac displays. (that said, I use a mac for work and still don't personally use "soft" in vscode)

the "soft" variant (what is this?)

I had to add that to the VSCode theme because of the overwhelming amount of issues related to how macos renders colors. Soft just reduces the saturation of the colors to make them appear more normal on mac displays. (that said, I use a mac for work and still don't personally use "soft" in vscode)

Ah, thanks. Haven't had this trouble in Vim, presumably, so I wasn't aware of it.


Remove deprecated packages

:tada: This PR is included in version 8.0.0-alpha.3 :tada:
The release is available on:

v8.0.0-alpha.3
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 8.0.0-beta.1 :tada:
The release is available on:

v8.0.0-beta.1
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 8.0.0 :tada:
The release is available on:

v8.0.0
GitHub release

Your semantic-release bot :package::rocket:


Add onValidationFailed handler

The default behavior is return undefined but some times we want to raise some exception.
For this and other any behavior, we will add this middleware.
Proposal:
{
  "*": "string",
  onValidationFailed: (schema, originalValue, errorCode) => {
    //...
  }
}

Added in 0.11.0


[REQUEST] Create GitHub Actions to help toolchain to transform design tokens

Suggestion/Concern
Related to, and depends on, #770.  Once a Theme Builder instance is associated with a GitHub repo and pushes a new design system or a design system update to GitHub, we probably need GitHub Actions defined that will run when the target folder has changes and will transform the generated design token JSON and turn it into CSS.
Resources given to me by @lwnoble:
https://github.com/tokens-studio/sd-transforms
https://configurator.tokens.studio/
Proposed Solution
The idea here is that when a design system is updated by Theme Builder, we should be able to pass the artifacts to the designers for use in Figma (design tokens) and to the developers for use in applications (CSS)

@aaronreed708 may i give it a try

Hi @sagnik3788! We love that you have interest in the project, but I'm not sure that everything is in place here, yet (e.g. there might be dependencies missing). And we really don't want you to feel frustrated right out of the gate. @lwnoble , can you verify whether the JSON is stable or is it still in flux?  I guess that progress could be made on this issue with test data and without #770 in place

Work on this can begin on @lwnoble latest code.  Will update this issue once the branch is ready.  Will need to use instructions from #829 to setup the environment in order to work on this issue.


chore(main): release 0.1.38

Auto generated release
0.1.38 (2024-12-31)
Build System

deps-dev: bump laravel/pint from 1.18.3 to 1.19.0 (#135) (a6496e6)
deps-dev: bump phpstan/phpstan from 2.0.3 to 2.0.4 (#134) (728a0c2)
deps: bump nanoid from 3.3.7 to 3.3.8 (#132) (8ecdae8)
deps: bump nesbot/carbon from 3.8.2 to 3.8.4 (#136) (8b2854b)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v0.1.38
:sunflower:


Typescript Issue with API Route Redirects

I've implemented the typed helpers per the docs, but see this warning when I try to redirect an API route that is totally valid by Next.js standards!

It does work without errors like this though!


Prometheus with Telegraf

Hi,
I am using Telegraf to ship or say expose my metrics on /metrics path for my prometheus operator service monitor to capture application metrics....
But i am facing a problem like since i am parsing my logs using telegraf so for every new timestamp it is creating a new time series..
For example - metrix_x(time=t1), metric_x(time=t2)
So logically it should create a single time series with these timestamps as my base timestamp but i am not sure how to achieve that via telegraf and prometheus.
Need help on this...

It makes more sense to ask questions like this on the prometheus-users mailing list rather than in a GitHub issue. On the mailing list, more people are available to potentially respond to your question, and the whole community can benefit from the answers provided.


Add a nix setup using flake

This adds a basic nix build which makes it possible to easily include BLAKE3 as a flake dependency.

This feels like a lot of toplevel stuff to have in the repo, for what would be a small minority of users. Would it be possible to maintain it in a separate repo, or at least in a subdirectory?

@oconnor663 I can move it to my own repo. Though you should check out nix as a universal build tool.

I converted this to an independent solution.


Update .eslintignore

@rpokorny for whatever reason it seems that the build task was reading the ignore file from root fs and not cwd ¯\_(ツ)_/¯
When you get a chance :+1: if this works for you
CC @akonsowski @clarkjefcoat
https://github.com/ozone-development/ozp-center/issues/676

:+1:

:+1:


Calling partial that returns value without arguments (or a zero argument) just silently fails

I get that this is was a bug in the expected behavior. However, this fix broke my particular use case:
{{ with partial "helpers/get-image" .image }}
  <!-- do something with image if found -->
{{ end }}

In my case .image may be empty, and in this case it is completely correct to silently "fail" and return nil from the partial.
Just my five cents: maybe better to make a warning out of similar cases as opposed to a full error, so current behavior is not broken?


Add section on RunOptions

Inspired by this UserVoice request, we received a request to share how to avoid failure when the user is in cell edit mode. Since this is a RunOption passed to Excel.run it made the most sense to add the information here. If there's a better location, please suggest it.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/excel/excel-add-ins-core-concepts.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/excel/excel-add-ins-core-concepts.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Thanks @Rick-Kirkham and @kbrandl. Updated accordingly.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/excel/excel-add-ins-core-concepts.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/excel/excel-add-ins-core-concepts.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Cant't seem to get anything to work when using some sort of shared folder

Describe The Bug
I have a linux host machine on which I develop. I have a windows virtual machine on which I want to build the project due to it being a windows driver. My plan was to create a shared folder containing the project to be able to build the project from the windows virtual machine. This sadly does not work however much I try it. At first I used a virtiofs filesystem with WinFSP. Even though I configured it properly according to this issue it did not work and gave me the following error:
[cargo-make] INFO - Execute Command: "rust-script" "target\\_cargo_make_temp\\persisted_scripts\\D4060E7434B3779E78A683E8BA00D06A5D08BE8C95BC432359E22F06CB30EF1C.rs"
Error: IoError(Os { code: 87, kind: InvalidInput, message: "The parameter is incorrect." })
[cargo-make] VERBOSE - Output: Ok(
    Output {
        status: ExitStatus(
            ExitStatus(
                1,
            ),
        ),
        stdout: "",
        stderr: "",
    },
)
[cargo-make] VERBOSE - Executed rust code, exit code: 1
[cargo-make] ERROR - Unable to execute rust code.
[cargo-make] WARN - Build Failed.

Then I thought "No problem let's just try good ol trusted SAMBA". Yet fate again was not nice to me. I got a new error but that hardly made this less frustrating:
[cargo-make] INFO - Execute Command: "rust-script" "target\\_cargo_make_temp\\persisted_scripts\\D4060E7434B3779E78A683E8BA00D06A5D08BE8C95BC432359E22F06CB30EF1C.rs"
Error: IoError(Os { code: 4390, kind: Uncategorized, message: "The file or directory is not a reparse point." })
[cargo-make] VERBOSE - Output: Ok(
    Output {
        status: ExitStatus(
            ExitStatus(
                1,
            ),
        ),
        stdout: "",
        stderr: "",
    },
)
[cargo-make] VERBOSE - Executed rust code, exit code: 1
[cargo-make] ERROR - Unable to execute rust code.
[cargo-make] WARN - Build Failed.

My project is exactly what is described here. As soon as I copy the project from the shared folder to any location on disk like the desktop the build works as expected.
To Reproduce
Setup a windows virtual machine on a linux host. Follow the steps described here to set up a driver project. And the simply try to build the project with cargo make.
Error Stack
As described above
Code Sample
/

@kedom1337 thanks for notifying me.
i have to admit i'll probably not setup a windows vm like you did as i'm running on a weak arm machine so will need your help here.
the flow to run a script is

create a temp file
add some pre script content (like cd back to original location) - not relevant for your rust usecase
add user content
run it

for rust scripts, i run it by using the rust-script crate and you can see the actual command i'm running.
so error i feel is originating from that crate.
my suggestion is to create a simple test case without cargo make

create a rust file with your code
run it as i did
see error

and open an issue for that crate with link to this one.
wdyt?

thanks for verifying. i'll close it for now and feel free to reopen if needed


Floyd Warshall Algorithm in C++

@aanchal009  contribute here https://github.com/Ishaan28malik/Get-Dev-Resources


Don't work

Hello,
Installed as recommended, don't work.
Message from Log.txt:
X-Plane 11/Resources/plugins/TypeIt/64/win.xpl : Error Code = 126 : Impossível localizar o módulo especificado.
Thanks.

Shame that not even a response was given...


Corrected rule variable

Variable 1_4_1 used for task 1_5_1.

Hello,
We have been working on a shift and some re-writes for this role and addressed this issue. In that work. Thank you for being part of the community and contributing to the project!
-George


fix当成员是std::variant类型时.get_fields_name 会有重复字段名bug;2.添加get_field_any接口,具体类型转换延迟到用户端代码

Why

fix当成员是std::variant类型时.get_fields_name 会有重复字段名bug
添加get_field_any接口,具体类型转换延迟到用户端代码

What is changing
Example

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

用clang-format13 格式化一下代码。

你能把这个pr也提到iguana那边吗？


How are you hiding the dropdown windows when the user clicks away from it?

This isn't a bug or a feature request. I was just curious how you implemented this functionality. I already have my own ags config which includes a bar for each monitor and a dropdown window. Right now, the only way to hide or show the dropdown window for my config is to click on a button on the bar. I'd love to be able to close the dropdown window by clicking away from it anywhere.
I tried looking at your config but couldn't figure out how you accomplished this. I'm guessing it has something to do with all the EventBox widgets in DropdownMenu?

I have 2 EventBoxes. The parent EventBox that covers the entire screen (className: "parent-event"). This parent event box will then close the window if it registers a click.
Then I have a child EventBox inside of the parent EventBox above. This child also registers clicks but it returns true instead. This allows you to interact with it's children.

Let's go ahead and move this conversation over in Discussions.


Azure NetApp Files - create snapshot_policy operation throws exception

Package Name: azure-mgmt-netapp
Package Version: 0.13.0
Operating System: Linux - Ubuntu 20.04.1
Python Version: 3.8.5

Describe the bug
While creating Snapshot Policy for Azure NetApp Files, the operation fails with Deserialization error. The value that the error mentions as missing has been passed in the same way mentioned in the documentation.
Code to create Snapshot Policy
netapp_client = get_client_from_cli_profile(                         AzureNetAppFilesManagementClient,                         subscription_id=subscription_id)         sp_creation_ops = netapp_client.snapshot_policies.create(             body=payload['body'],             resource_group_name=payload['rgpName'],             account_name=payload['anfAccount'],             snapshot_policy_name=payload['spName']         )
'body' parameter for create operation
    "body": {         "location": "westeurope",         "daily_schedule": {             "snapshots_to_keep": 7,             "hour": 1,             "minute": 30         },         "enabled": true     }
Exception:
Azure Error: DeserializationError Message: Deserialization error for object SnapshotPolicyEntityModel - Could not find member 'snapshots_to_keep' on object of type 'DailyScheduleEntityModel'. Path 'dailySchedule.snapshots_to_keep'.
To Reproduce
Steps to reproduce the behavior:

Use the above code to create a snapshot policy
Pass the body given above as parameter

Expected behavior
A snapshot policy with daily schedule as given above should be created
Additional context
Snapshot policies will allow us to automate NetApp volume backup by setting policy while we create the volumes itself. Policies are set at account level and can be assigned to independent volumes

Hi @jsntcy, @changlong-liu - any update on this?

Thank you for addressing this issue. Since the issue has been open for very long, we'll evaluate the necessity of change in current solution and will raise a separate issue linked to this one if needed.


Update e2e script for CI

Move screenshots folder to the top-level: https://coreos.slack.com/archives/C02EQERSV3N/p1655841794720949
Add execute permission to the E2E script.

/lgtm

/approve


fix Issue 16291 - EncodingScheme.create fails

add EncodingScheme.register overload that references the registered class
just adding the FQN name of a class does not reference that class, so
it must not end up in the binary and subsequently EncodingScheme.create failed
This used to work by chance b/c all the EncodingScheme implementations
were in a module w/ static ctor. Any user of std.encoding did drag in
that ModuleInfo, which in turn referenced all classes of std.encoding.
Since moving the static ctor to std.internal.phobosinit to break a
cycle, the classes were no longer referenced by a ModuleInfo w/ shared
ctor, so they wouldn't end up in the binary unless explicitly
referenced elsewhere.
deprecate the old EncodingScheme.register(string fqn) b/c relying on
Object.factory is slow, error prone (linkage), and really unnecessary
import encodinginit in std.encoding so that the
outsourced static ctor actually gets run

depends on #4839

Unfortunately we can't really test for that bug from within phobos, need to add some additional tests like we have for druntime, but that's somewhat out of scope.

Nice, LGTM.
I think we merge this and close the other PR.

Auto-merge toggled on


Add unicode support.

Hello! It is a wonderful editor, but it does not support Unicode.
For example if i insert phrase "Arabic (Arabic: العَرَبِيَّة, al-ʻarabiyyah [ʔalʕaraˈbijːah]" i get incorrect result.
Please add support of Unicode BIDI algorithm, context glyph forms and etc.

I'm trying to type those characters in Vim or just in the terminal and it seems to be broken there to. I'm using iTerm2. Are you using a terminal where arabic characters display correctly in another editor (Vim? Nano?)

I tested those terminal emulators:

gnome-terminal   terminal - bad, micro - bad, nano - bad, vim - bad
konsole - terminal good, micro - bad, nano - good, vim - bad

emacs good on gui mode

https://punpun.xyz/1jfy.png

Looks like micro displays tem pretty well, although the width/spacing is off.
Not only spacing, but wrong context forms.
For example العَرَبِيَّة  vs العَرَبِيَّ ة
The most left symbol the same in both cases, but has different context form final and isolated.
Also symbols has wrong logical order.

Unicode Arabic/Persian characters are displayed correctly. But RTL / BIDI does not work in any terminal-based text editor or even text viewer (even less or more, except for an old Arabianized vim that did not work properly too), and BIDI is not fully implementable in terminals.
For editing Arabic/Persian, a graphical text editor is required.
I'm Persian and I have a lot of experience with internationalization in Linux. and I think there is no ways that a terminal-based editor can handle it correctly.
I think it's safe to close this issue.

Thanks for your input. I'll close the issue.