Change how we assemble user names in ASC

Use a DOMAIN\Username format to avoid accidental detection of
a name that has a dotted domain as an enterprise name.
Fixes #44

@frozencemetery @kvv81 I added another patch to handle this better on the get_enterprise_name() part as well.
Please (re)review.

I wonder if I should get away with get_enterprise_name() and just make a parse_user_name() function that processes everything.

Ok I pushed a proposed fixup version where I get all username processing in a single parse_user_name() function.
Let me know if this looks better. The behavior is unchanged from the previous patch, it is just moving around stuff.

Oops, it seems like enterprise name use-case is not going well...

I see a problem with escaping in this flow:

gssntlm_accept_sec_context is called, security context has DOMAIN=(NULL) and<EMAIL_ADDRESS>call to ntlm_decode_auth_msg is done, however this routine does not add escaping for (at) character, so on the output we have DOMAIN=(NULL) and<EMAIL_ADDRESS>During the call of gssntlm_import_name -> gssntlm_import_name_by_mech -> parse_user_name finds only 'at' character without 'sep' because escaping was not done; as a result, this use-case is handled as standard foo@BAR form.
winbind is called with wrong domain/username, NO_SUCH_USER error.
NTLM CRAP authentication for user [gmail.com][username] returned NT_STATUS_NO_SUCH_USER

Very good point, thanks for testing this is exactly the kind of validation I was hoping to see.
So we have two options here:
Always compose as if a domain was there, and treat \user@foo as valid with result:
DOMAIN=NULL, USERNAME=user@foo
or always escape the username therefore allowing domain\user@foo as valid with result:
DOMAIN=domain, USERNAME=user@foo
The first form has the advantage of avoiding escaping, and introduces a new way to pass in an enterprise name.
The second is more consistent with the current documentation, but requires internal escaping and a change in parse_user_name()
Finally we have a third option, reactitect ASC to be able to pass in a gssntlm_name instead and avoid translations, but this does not solve the case where a user need to handle this stuff from a client application.
I am leaning toward the first form because ti will make it easier to deal ith from client apps as well without escaping,
WDYT ?

I changed the code according to the preference I expressed in my previous comment.
@kvv81 it would be really nice if you could retest in the failing scenario to assure we properly handle it now.

Well, I have tried to test all those corner-cases in-depth and got unexpected result...
Using special characters like (at) and (backslash) doesn't work well even in native Windows environment when using 'net use' command.
First of all, special characters are forbidden by ActiveDirectory itself, when trying to add such you will see this warning:

As the result, pre-win2k user logon name is fixed to NOT contain any (at) or (backslash). When 'DOMAIN\user' old form is used, special characters MUST be replaced with underscores.
New (unchanged) user logon name can be used with UPN form only (username@domain.com), BUT in this case special characters like (at) or (slash) are also forbidden for reason of collision with old format and (at) separator character.
I.e. the usage of new (post-win2k) user logon name is probably due unlimited length of user logon name (pre-win2k has a limit of 20 characters, new logon name list much longer - 256?) and maybe due other special characters. Although (at) and (backslash) characters are allowed in new logon name, you simply can't use it...

Bottom line - I do not see any benefit to support enterprise names like<EMAIL_ADDRESS>(username_example.com must be used instead). Could you please advise in what environment did you see this usage before? Probably my reproduction was not full enough (I used 'net use' and RemoteDesktop apps to check usernames with special chars - not handled there).

As the test itself for your changes - I got a new problem with 'plain username' use-case in my scenario.
I.e. when I run "net use \<IP_ADDRESS>\share /user:test_user", I get the error.
In tcpdump I see following things:

There are TWO attempts of login, the 1st is done immediately (with default NULL domain and empty password?) and usually fails with LOGIN_FAILURE. From winbind side I see this line for 1st try:
NTLM CRAP authentication for user [VAST-SMB-DEV][test_user] returned NT_STATUS_WRONG_PASSWORD
The 2nd try is done later (after correct password is entered) BUT domain name is "<IP_ADDRESS>" (set by client).
Winbind doesn't understand this trick and this try is failed too.
NTLM CRAP authentication for user [<IP_ADDRESS>][test_user] returned NT_STATUS_NO_SUCH_USER

I understand that here we have problem driven from client-side, but still we need to solve it somewhere.
The 1st try also fails on WindowsServer (due bad password, we can't do anything here).
But the 2nd try (with domain=server_ip) passes properly on windows. Probably we need to make some work-around here

The \test_user issue cropped up also in the test suite, and I have already rebased yesterday to fix it, can you check you got the latest release ?

In a domain named foo.com you can set a UPN like<EMAIL_ADDRESS>in this case<EMAIL_ADDRESS>is an enterprise name, as it has nothing to do with the actual AD domain name. Enteprise names need special resolution in AD, which is why they are passed as<EMAIL_ADDRESS>DOMAIN=NULL
That's all there is, paring<EMAIL_ADDRESS>as USERNAME=user DOMAIN=bar.com would fail as there is no such bar.com AD (or NETBIOS) domain in the forest.

Btw the <IP_ADDRESS> "domain name" looks like a bug in whatever client you are using, I am not aware of that being a valid name at all, sounds like a resolution issue in some fallback.

The \test_user issue cropped up also in the test suite, and I have already rebased yesterday to fix it, can you check you got the latest release ?

Sorry, I don't understand what is the "latest release"...
I have got both commits from your enterprise_name_issues branch, and the latest commit from main is dated by September (also present).
What new code do you mean?
$ git branch
* enterprise_name_issues

$ git log

commit 2d5c7d1bc66e1d481b028b2c1dda2015b838ba14
Author: Simo Sorce<EMAIL_ADDRESS>Date:   Mon Oct 12 16:32:54 2020 -0400

    Change how we assemble user names in ASC
...
commit 75ed138925c748622ef9cef51085d46747e4c092
Author: Simo Sorce<EMAIL_ADDRESS>Date:   Tue Oct 13 14:36:55 2020 -0400

    Introduce parse_user_name() function
...
commit ecda1a5a82300b72af47260f1e90c181c04a2270
Author: Volodymyr Khomenko<EMAIL_ADDRESS>Date:   Tue Sep 29 14:00:05 2020 +0300

    Fixed memory leaks found by valgrind

That's the right top commit, but now I am baffled.
In a previous push I had this issue that I was getting \username as well, so I rebased an push the current commit to properly address that.
Now the test suite shown no such  behavior, and it can't because the code properly parses a \username as USERNAME=username, DOMAIN=NULL and the \ is gone, unless somehow someone passes in \username as a username ?
But that is not what you do in the Windows commandline, so I am baffled about how this can happen.
Would you be able to capture the NTLM negotiation before we hand data over to Winbind and see if the \ is already present in the username there? ATM I cannot explain how else a username can end up having a leading \ ...

I'm afraid that UPN use-cases are still problematic with winbind.
This time I'm using this "happy path" matrix (it passes for "net use" command in case of Win-Server):

What I see that UPN scenarios are failed due splitting enterprise name on parts.
Note - for cases 2 vs 3 (and 4 vs 5) winbind receives exactly the same input data (domain and account),
also used password '12345' is the same, BUT we still have different auth result.
I think that password hash also depends on account name, so using improper account  name leads to wrong password result.
1. net use v: \\<IP_ADDRESS>\root 12345 /user:test_user

useratdom=\test_user
winbind_srv_auth: account_name=test_user, domain_name=VAST-SMB-DEV, workstation=CNTOS-VOLODYMR2
[1603112103] ALLOK: gssntlm_srv_auth() @ src/gss_auth.c:446 [0:0]

NTLM CRAP authentication for user [VAST-SMB-DEV]\[test_user] returned NT_STATUS_OK

2. net use v: \\<IP_ADDRESS>\root 12345 /user:vast-smb-dev.local\test_user

useratdom=vast-smb-dev.local\test_user
winbind_srv_auth: account_name=test_user, domain_name=vast-smb-dev.local, workstation=CNTOS-VOLODYMR2
[1603112266] ALLOK: gssntlm_srv_auth() @ src/gss_auth.c:446 [0:0]

NTLM CRAP authentication for user [vast-smb-dev.local]\[test_user] returned NT_STATUS_OK

3. net use v: \\<IP_ADDRESS>\root 12345<EMAIL_ADDRESS>
<EMAIL_ADDRESS>winbind_srv_auth: account_name=test_user, domain_name=vast-smb-dev.local, workstation=CNTOS-VOLODYMR2
[1603112407] ERROR: gssntlm_srv_auth() @ src/gss_auth.c:420 [851968:13]

NTLM CRAP authentication for user [vast-smb-dev.local]\[test_user] returned NT_STATUS_WRONG_PASSWORD

4. net use v: \\<IP_ADDRESS>\root 12345 /user:vast-smb-dev\test_user

useratdom=vast-smb-dev\test_user
winbind_srv_auth: account_name=test_user, domain_name=vast-smb-dev, workstation=CNTOS-VOLODYMR2
[1603112492] ALLOK: gssntlm_srv_auth() @ src/gss_auth.c:446 [0:0]

NTLM CRAP authentication for user [vast-smb-dev]\[test_user] returned NT_STATUS_OK

5. net use v: \\<IP_ADDRESS>\root 12345 /user:test_user@vast-smb-dev

useratdom=\test_user@vast-smb-dev
winbind_srv_auth: account_name=test_user, domain_name=vast-smb-dev, workstation=CNTOS-VOLODYMR2
[1603112570] ERROR: gssntlm_srv_auth() @ src/gss_auth.c:420 [851968:13]

NTLM CRAP authentication for user [vast-smb-dev]\[test_user] returned NT_STATUS_WRONG_PASSWORD

Please note that showing "\test_user" is a wireshark side-effect, it uses (backslash) as a separator to show domain + username in one field. This just means that domain=NULL and username=test_user.
I think that using domain=server_ip_address is some client-specific, and now this behaviour has done (so use-case with plain username as logon_name passes now).

So it looks like what you are saying is that Windows maintain an enterprise name as is, while Winbindd seem to somehow decide to "reparse" the username after we hand it in, and gets it wrong ?

Can you get debug level 3 logs from Winbind ?
there is a log line that should show username and domain as:
[] pam auth crap domain:  user:
I would like to see what you get there for the "enterprise name" cases.

net use v: \\<IP_ADDRESS>\root 12345<EMAIL_ADDRESS>
log from gssntlmssp (+I have added few verbose print-outs there):
<EMAIL_ADDRESS>[1603121394] ALLOK: parse_user_name() @ src/gss_names.c:206 [0:0]
winbind_srv_auth: account_name=test_user, domain_name=vast-smb-dev.local, workstation=CNTOS-VOLODYMR2
[1603121394] ERROR: gssntlm_srv_auth() @ src/gss_auth.c:420 [851968:13]

wibind logs:
[ 4795]: pam auth crap domain: [vast-smb-dev.local] user: test_user
NTLM CRAP authentication for user [vast-smb-dev.local]\[test_user] returned NT_STATUS_WRONG_PASSWORD
process_request_done: [4795:PAM_AUTH_CRAP]: NT_STATUS_WRONG_PASSWORD

<EMAIL_ADDRESS>[1603121394] ALLOK: parse_user_name() @ src/gss_names.c:206 [0:0]
winbind_srv_auth: account_name=test_user, domain_name=vast-smb-dev.local, workstation=CNTOS-VOLODYMR2
[1603121394] ERROR: gssntlm_srv_auth() @ src/gss_auth.c:420 [851968:13]

This is surprising, so you are saying that gssntlmssp is failing to provide:
<EMAIL_ADDRESS>DOMAIN=NULL
and is instead splitting and providing
USERNAME=test_user
DOMAIN=vast-smb-dev.local

I think the issue may be in winbindd_get_creds() which should be invoked by gss_acquired_cred_from() right before external_srv_auth(0 is called.

@kvv81 I added a tweak to how represent enterprise username internally, by changing from using a NULL domain to an empty "" domain.
Can you check if this makes things better ?
It should make us not try to set a domain in winbindd_get_creds() for example.

No, the tweak has no effect.
winbindd_get_creds() is not a source of a problem because the name is split earlier by parse_user_name().
In this use-case we have both 'at' and 'sep' pointers set and sep < at so parse_user_name() goes deeply inside the complicated logic I don't fully understand. I'm afraid the code of parse_user_name has become too cumbersome and it's too hard to troubleshoot it.
I suggest to write a unit-test for all use-cases you wrote in the header of parse_user_name - it will be more easy to run a test with gdb than try to guess all issues from source-code...
Having those print-outs added:
--- a/src/gss_names.c
+++ b/src/gss_names.c
@@ -317,6 +317,11 @@ uint32_t gssntlm_import_name_by_mech(uint32_t *minor_status,
                                  input_name_buffer->length,
                                  &name->data.user.domain,
                                  &name->data.user.name);
+        if (retmaj == GSS_S_COMPLETE) {
+           gssntlm_debug_printf("parse_user_name result: domain=%s, name=%s\n",
+              (name->data.user.domain) ? name->data.user.domain : "<null>",
+               (name->data.user.name) ? name->data.user.name : "<null>");
+        }
     } else if (gss_oid_equal(input_name_type, GSS_C_NT_MACHINE_UID_NAME)) {

         name->type = GSSNTLM_NAME_USER;
diff --git a/src/gss_sec_ctx.c b/src/gss_sec_ctx.c
index 2464571..81ab805 100644
--- a/src/gss_sec_ctx.c
+++ b/src/gss_sec_ctx.c
@@ -878,6 +878,7 @@ uint32_t gssntlm_accept_sec_context(uint32_t *minor_status,
             memcpy(&useratdom[uadlen], usr_name, ulen);
             uadlen += ulen;
             useratdom[uadlen] = '\0';
+            gssntlm_debug_printf("useratdom=%s\n", useratdom);

             usrname.value = useratdom;
             usrname.length = uadlen;

Running the command:
net use v: \<IP_ADDRESS>\root 12345<EMAIL_ADDRESS>I get those logs:
<EMAIL_ADDRESS>[1603127368] ALLOK: parse_user_name() @ src/gss_names.c:206 [0:0]
parse_user_name result: domain=vast-smb-dev.local, name=test_user
winbind_srv_auth: account_name=test_user, domain_name=vast-smb-dev.local, workstation=CNTOS-VOLODYMR2
[1603127368] ERROR: gssntlm_srv_auth() @ src/gss_auth.c:420 [851968:13]

Allright, I am guilty of not adding tests, that is bad, I just did not realize that parse_user_name() was failing at correctly parsing the name, my bad.
I'll add test cases before asking you for any more live tests.

Ok added unit tests, which revealed 2 issues in parse_user_name, one of which would cause the form \user@foo to not be properly recognized.
Now it should work as expected.
@kvv81  sorry for not adding this test earlier and  making you run in circles.

Well, we are one step closer to the good result :-)
I see that parse_user_name now returns expected values. But as you mentioned before, winbindd_get_creds() is called after that and domain name is modified: NULL->default_winbind_domain. So probably winbind uses name as plain username and fails with NO_SUCH_USER.
Again, testing matrix is 5 use-cases that should all pass properly. 'Plain username' and 'old pre-win2k logon name' works properly, but 'UPN as login name' fails.

Nevermind.

And we have finally came to the point when we rely on winbind's behaviour, this is tough topic for testing...
Thanks for helping us!

I can try to change the code to not add the domain at all in the enterprise case, by passing in an empty domain.
But it is unclear if Winbindd will work properly.
Would you happen to have a capture of NTLM authentication done against a windows server with a UPN name ?
I want to check how windows deal with the user vs domain name in NTLM packets in this case.

I pushed a smal patch that should make winbindd_get_creds() not add a domain part.
Wonder if this is sufficient to get it working.

Would you happen to have a capture of NTLM authentication done against a windows server with a UPN name ?
I want to check how windows deal with the user vs domain name in NTLM packets in this case.

Yes, I have reproduced all visible variants of logon name (down-level, UPN, plain) against WinServer before to verify that those cases are really supported. The result of captured traffic for SMB SESSION_SETUP is summarized in 'NTLM domain' field of the table:

For example, for<EMAIL_ADDRESS>you can see NTLM domain=NULL, meaning that domain name in SESSION_SETUP packet was set to NULL and the rest of information was passed as username (test_user@vast-smb-dev.local).

As a summary, I can guess following behaviour during SMB auth:

Client search for down-level format separator (backslash) in logon name. If found, the first part of logon name is encoded as domain, the rest comes to username (backslash is stripped).
If backslash is not found in logon name, SMB client DOES NOTHING with logon name (i.e. client doesn't parse domain from UPN because we may have some alternate UPN suffix that doesn't match to domain name). Thus UPN format is always come as is (by single piece) in username with domain=NULL.
(GUESS) SMB server (while handling NTLM authentication) is likely to pass NTLM domain and username as they are to Domain Controller (again, if UPN is inside NTLM username, it can be alternate UPN suffix a.k.a enterprise name that doesn't match to actual domain name, so SMB server can't predict correct domain name for UPNs).
Domain Controller uses NTLM Domain part as domain (if not NULL) and NTLM username as username. If domain is NULL, username can be either plain username or UPN<EMAIL_ADDRESS>or username@ALTERNATE.upn). Domain controller has all required information (short+fqdn for all DOMAINs, all the list of alternate UPN suffixes) to recognize username in UPN format and parsing is done properly here (on DC).

And YES, this time all five cases works properly!
Now UPN works fine, plain username and down-level logon names are ok (as before).


Adding Ionic Pro Overview Slides

Initial upload of Ionic Pro Overview Slides.

Seems fine for now. If we re-use this we will probably want to rename that to be something more generic. We'll see what happens.


All lints need tests

We should have a meta-test, that enforces that any lint file has an associated test file.

I went through and looked at which lints don't have tests by name. Here is the list that we should investigate:

[ ] lint_generalized_time_does_not_include_seconds.go
[ ] lint_generalized_time_includes_fraction_seconds.go
[ ] lint_subject_locality_without_org.go
[ ] lint_dsa_improper_modulus_or_divisor_size.go
[ ] lint_dh_params_missing.go
[ ] lint_rsa_no_public_key.go
[ ] lint_subject_province_without_org.go
[ ] lint_subject_not_dn.go
[ ] lint_ext_authority_key_identifier_no_key_identifier.go
[ ] lint_subject_common_name_disallowed.go

In case it helps, this is the code I used to check:

import sys

e = set()
for l in sys.stdin:
    l = l.strip()
    if "test" not in l:
        e.add(l)
    else:
        l = l.replace("_test", "")
        e.remove(l)

print e```


Grav Will Not Update to 1.6

Grav will not update and continues to give me the following error:

This continues to show up, no matter which version of PHP (7.1 and 7.2 attempted) is installed on the server

I found the file index.php that says $ver = PHP_VERSION that appears to have a related call to the one that is throwing an error. How do I get this call to pull the correct PHP information so I can complete the upgrade?

Looks like your server is still using the old version of PHP; you need to change your server configuration to use PHP 7.2.

Just because the terminal shows php 7.2 doesn’t mean the web server is using that version.  Check the config -> info tab or turn on debugger to see what version you are running in the server.

Thank you @mahagr for pointing me in the right direction & @rhukster to show me where Grav tracks the information. Finally got PHP fully upgraded!


[5.10][TypeChecker] Prevent invalid attribute on accessor from failing if i…

…t's a warning
Cherry-pick of https://github.com/apple/swift/pull/70163
Follow-up to https://github.com/apple/swift/pull/69369

Explanation:

Pre-Swift 6 use of a global actor attribute on an accessor would be diagnosed as a warning and shouldn't fail GlobalActorAttributeRequest.

Scope: Accessors with global actor attributes in pre-Swift 6 mode.

Main Branch PR: https://github.com/apple/swift/pull/70163

Resolves: rdar://118934711

Risk: Low

Reviewed By: @hborla

Testing: No tests since the behavior is not currently observable.

@swift-ci please test


[AutoPR peering/resource-manager] [WIP] Adding readme.cli.md file for peering

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/7419

(message created by the CI based on PR content)


Add remaining optional TCK tests from MP JWT Auth

Adds the optional servlet, EE security, and JACC tests from the MP JWT Auth TCK. That includes the following classes:

org.eclipse.microprofile.jwt.bridge.tck.container.servlet.ServletTest
org.eclipse.microprofile.jwt.bridge.tck.container.jaxrs.RolesAllowedTest

Just the testNeedsGroup1Mapping test case.

org.eclipse.microprofile.jwt.bridge.tck.container.jaxrs.jwe.RolesAllowedSignEncryptTest

Just the testNeedsGroup1Mapping test case.

org.eclipse.microprofile.jwt.bridge.tck.container.jacc.SubjectTest

For #5

@sberyozkin Here are the only remaining optional tests from the MP JWT Auth spec. For now I have lumped all the tests to be included and run in the base suite as well as the full suite. We can discuss later if we want to organize the suites differently (e.g. have core, web, and platform suites instead).

Thanks @ayoho, checking PR now


Import Dockerfile

It would be nice to be able to open a Dockerfile on my hard drive and have the image created.

@felixgborrego @nathanila great. Do you know how to compile? I cant find my scala ..js output files. (see newest issue)
I realy want a Dockerfile import :D.
Thanks for the great work.


How to register own key / key code

Hello!
I'm struggling with some problem. I want to create own key code and name it pupa (i.e. 'key_code': 'pupa' and set it instead of capslock), and use pupa as kinda HYPER one. I've seen solutions like "set hyper as all right modification keys", but I just want to have clear/new one for my experiments. I was trying to find it somewhere on the Internet, but I didn't saw anything like that.
Is it somehow possible in Karabiner?

 Like a hyper...
This is an example: https://pqrs.org/osx/karabiner/json.html#typical-complex_modifications-examples-change-right-shift-x2-to-mission-control
Another example (SpaceFN): https://pqrs.org/osx/karabiner/complex_modifications/json/spacefn.json
I hope it helps...


Build shared library with F# fails on macOS

I followed the NativeLibrary sample to build a shared library in F#. Here's my code.
It fails with
ld: illegal text-relocation to '___typemanager_indirection' in obj/Debug/netcoreapp2.2/osx-x64/native/FSNativeLibrary.o from '___RMHSignature_System_Linq_System_Linq_Enumerable__Empty' in obj/Debug/netcoreapp2.2/osx-x64/native/FSNativeLibrary.o for architecture x86_64

Building as a static library works. I have only tested on macOS 10.13.6

Fixed by https://github.com/dotnet/corert/pull/7249


Definition of eccdna_status

Hello there,
I am confued by  the definition of 'eccdna_status' in the output of identify_wgls. What's the difference between cyclic and non-cyclic eccDNA? How could a eccDNA be non-cyclic?
Thanks a lot.

@20191600 The non-cyclic eccDNAs could be the artifacts of the prediction or the quality of data.
The goal of CReSIL is to capture all possible results and it's up to the user to define the cutoff.

I suppose cyclic eccDNA is all i need. Thanks for answering.


Fix for new jb

This is a quick fix for running under the latest jbrowse
The process of getting a subadapter can be async so it needs to run like this
This isn't a perfect scheme (some ts-ignores, some relying on behavior of getRefNames called first) but it is a quick fix
CC @elliothershberg

Thanks for looking at this. I'll probably try to add some types and merge.
Just for reference, when/why did this behavior change?

Lazy loading adapters


Targetting resets scene fx

https://www.dropbox.com/s/hld2ez3sp6fbmua/TargetEnhancementsSceneFXIssue.mp4?dl=0

reported by @rwr2 in Discord


Rust's orphan rule doesn't seem to hold

Aim
I'm able to define impls for traits in modules in which neither the trait nor the type are defined. My understanding is that this is illegal (the so-called 'the orphan rule').
Bug
mod serde {
    trait Serialize<let N: u32> {
        fn serialize(self) -> [Field; N];
    }
}

use crate::serde::Serialize;

// Even though Field is primitive, this should be illegal - the definition should go inside the serde
// module (where the trait is defined).
impl Serialize<1> for Field {
    fn serialize(self) -> [Field; 1] {
        [self]
    }
}

mod many_field {
    struct ManyFieldStruct {
        a: Field,
        b: Field,
    }
}

use crate::many_field::ManyFieldStruct;

// In this case neither the struct nor the trait have been defined in this module, yet I can still provide
// an impl and break the orphan rule.
impl Serialize<2> for ManyFieldStruct {
    fn serialize(self) -> [Field; 2] {
        [self.a.serialize()[0], self.b.serialize()[0]]
    }
}

#[test]
fn test() {
    let bar = ManyFieldStruct { a: 13, b: 42 };

    let bar_ser = bar.serialize();
    assert_eq(bar_ser[0], 13);
    assert_eq(bar_ser[1], 42);
}

I tried wrapping trait with an inner module (i.e. serde::foo::Serialize), and then place a second impl for Field there, and got a duplicate impl error, so at least that seems to work as expected.

I think this is working as expected. The orphan rule seems to be about not allowing these across crates, but here everything is defined in the same crate.

(put another way: I couldn't reproduce this issue if mod serde is in a different crate)

Closing as this is how it works in Rust too, but feel free to reopen if what I'm saying is wrong.

Ahh you seem to be right, sorry! For some reason I was sure modules had to contain both definition and impl. This makes much more sense.


tsserver feature request: improve compiling functionality

First of all thanks for implementing the feature: https://github.com/Microsoft/TypeScript/issues/19336
It allows us to remove own implementation of the command "reload" in the next versions of WebStorm.
In the latest WebStorm release we have rewritten the  service integration and now we use own extension of tssever only for compiling.
For complete removal the extension we need a couple new features:

Add a new API method for compiling projects by tsconfig.json.

It must return a list of diagnostics, emitted and affected files.
Optional flag to skip check 'compileOnSave' value;
Optional flag "forced" (similar to CompileOnSaveEmitFile)

Add new optional flags to CompileOnSaveEmitFile:

option to skip checking 'compileOnSave' value;
option to return list of diagnostics, emitted and affected files;
option projectFileName;

If i understand the requests correctly, these are all achievable in the API today. The API is intentionally split into smaller pieces to avoid long running operations on the server. the server is single threaded, and has no way to correctly interrupt/resume long running background tasks. we defer to the host to schedule these and cancel them as needed.

@mhegazy

If i understand the requests correctly, these are all achievable in the API today

no, we cannot achieve some of the functions using current API (without a session extension).
Right now I see there problems:

CompileOnSaveEmitFile doesn't return list of output files (I mean ".js" and ".js.map" files). We need this information for refreshing the files in the IDE. There is no way to get the output files (please correct me if I am not right)
getCompileOnSaveAffectedFileList doesn't return files if compileOnSave = false.

CompileOnSaveEmitFile doesn't return list of output files (I mean ".js" and ".js.map" files). We need this information for refreshing the files in the IDE. There is no way to get the output files (please correct me if I am not right)

the file would change on disk, is not that sufficient to update your state? we can make it send a response with the list of files written. but seems weird that you need that to refresh your state, these files could change in an out-of-band fashion any ways, e.g. a command line build command outside the editor.

getCompileOnSaveAffectedFileList doesn't return files if compileOnSave = false.

not sure what you expect here. if the user chose to disable the feature why do you want to force enable it?
why is this different from any other flag, e.g. typeAquision, or noEmitOnError?

but seems weird that you need that to refresh your state, these files could change in an out-of-band fashion any ways, e.g. a command line build command outside the editor.

We use this information in several ways:

Exclude files from the project (we don't want to index generated files)
If file was created first time (and for example parent directories) it can take a long time to get the information from FS but we get it immediately using the compile result.

not sure what you expect here. if the user chose to disable the feature why do you want to force enable it?

We use "compileOnSave" flag for compiling on the fly. But WebStorm also have explicit button "compile"(and "before run" task for compiling ts files) and users expect that the compiling will not taking into account the "compileOnSave" option. In fact it is not "compileOnSave" it just "compile" using the service.

We use this information in several ways:

Should be a simple change to add. we can return a list of files written to disk.
Feel free to send us a PR for the change.

We use "compileOnSave" flag for compiling on the fly. But WebStorm also has the explicit button "compile"(and "before run" task for compiling ts files) and users expect that the compiling will not taking into account the "compileOnSave" option. In fact it is not "compileOnSave" it just "compile" using the service.

Seems like you have a custom setup there. we make some assumptions based on compile-on-save being set, and decide not to keep a builder for instance. it should be possible for you to override the implementation of CompileOnSaveAffectedFileList in session, the functions it call are all public. I do not think this should be a supported scenario in the server through a new command.

it should be possible for you to override the implementation of CompileOnSaveAffectedFileList in session, the functions it call are all public. I do not think this should be a supported scenario in the server through a new command.

Yes, we have a similar solution for the problem. But the session api (even public) is not reliable (in
2.6RC you broke ~ 5 public methods that we used in previous versions).
Anyway thank, I will try to send you a PR for the first scenario.

Yes, we have a similar solution for the problem. But the session api (even public) is not reliable (in
2.6RC you broke ~ 5 public methods that we used in previous versions).

Sorry about that. we have added new tests to check the API changes at build time. so hopefully we do not get these unintentional changes in future versions.

Thanks @anstarovoyt for creating this issue. I had created a similar issue than you at https://github.com/Microsoft/TypeScript/issues/17630
@anstarovoyt I tell me how to you support the case of user save a ts file which must be excluded. I suggested

compileOnSaveAffectedFileList should throw error when the given file is excluded of tsconfig.json. It will give the capability to display an error dialog when user wishes to compile a ts file which is excluded by tsconfig.json

and @mhegazy answered me:

do not you have this already from projectInfo?

Thanks for your answer.

@angelozerr as I said before we extend the ts session and have own commands so we can do all the required check on the tsserver side

I forgot about #17630. this issue seems like a duplicate of #17630 then. thanks @angelozerr .
closing in favor of #17630.

@angelozerr as I said before we extend the ts session and have own commands so we can do all the required checks on the tsserver side

OK I understand more now, but your goal is not to use tsserver instead of implementing your own ts session? Perhaps tsserver should provide a new isInScope(fileOrDirName)  command?

you should be able to get that with ProjectInfo command.

Sorry @mhegazy you have already suggested me this solution and I hav enot give you an answer. Indeed I could consume ProjecInfo command but I'm afraid with performances.
In Eclipse builder word, we receive a list of files or folders to compile, and you need to filter it ot not. If tsserver could provide an isInScope(fileOrDirName) command, I could speed the build to avoid calling CompileOnSaveAffectedFileList for each ts files which must be excluded (ex : inside node_modules)

maybe i do not understand what isInScope supposed to do.. can you elaborate.

Ok my goal is to use tsserver to compile ts files instead of using tsc with watch mode:

I can manage progress monitor and stop compilation process when user click on stop.
it avoid having double parsing (one for tsserver and one for tsc)
I can consume language services (like tslint), but perhaps tsc can support that?

When an Eclipse project is opened, an Eclipse builder is executed and loop for each files of the project. In each loop I check if ts file must be compiled by calling CompileOnSaveAffectedFileList. Imagine you have a lot of ts files in your node_modules folder, I will check for each ts files of  node_modules if ts y calling file must be compiled by calling  CompileOnSaveAffectedFileList, although if I could consume isInScope("node_modules") which returns false in this case, I could stop the loop.
You suggested me to consume ProjectInfo, but this command works only if you know a ts file which belong to a ts project. Me I don't know ts file which belong to a ts project. I could have several tsconfig.json files. If projectInfo could work by calling it with tsconfig.json file (instead of ts file), I could perhaps consume it:

I could track the list of tsconfig.json in the project with Eclipse API.
call for each tsconfig.json  the projectInfo to build a list of ts files to compile. I could use this list to check if ts file must be compiled or not (like you suggested me).

But I'm a little afraid with performance when projectInfo is called with a lot of ts files (the JSON response could be big).
To support compile with tsserver with clean mean and good perforamnce, I will need:

call projectInfo with tsconfig.json file to support full build. Perhaps it will support the feature of @anstarovoyt (But WebStorm also has the explicit button "compile")
provide an isInScope(fileOrDirName) which basicly will call projectInfo and check if it contains fileOrDirName. isInScope returns just a boolean compare to projectInfo which returns a big ts files list.

Hope you will understand my need.

Imagine you have a lot of ts files in your node_modules folder, I will check for each ts files of node_modules if ts y calling file must be compiled by calling CompileOnSaveAffectedFileList, although if I could consume isInScope("node_modules") which returns false in this case, I could stop the loop.

That is the part i am confused about. you have some starting point, like a list of open files for instance, get the projectInfo for these, and that all you need to build.

If projectInfo could work by calling it with tsconfig.json file (instead of ts file), I could perhaps consume it:

We could make that work. but earlier you said the build loop works on .ts files and not the tsconfig.json.

That is the part i am confused about. you have some starting point, like a list of open files for instance,

The list of files is not necessary opened. When you open an Eclipse project the first time, ther eare none opened files, but builder is called with a list of files (closed) of the project.

get the projectInfo for these, and that all you need to build.

You suggested me to consume projectInfo for each files of the list? If it that:

I'm afraid with performance.
if file is not opened, I think projectInfo doesn't work.

We could make that work. but earlier you said the build loop works on .ts files and not the tsconfig.json.

the build loops for the whole files of the project (*.java, *.ts, *.json files ), but in my case I filter only ts files.

do you assume all projects have tsconfig.json?

do you assume all projects have tsconfig.json?

Yes, I assume even that a projct can contains several tsconfig.json files.

so filtering the tsconfig.json files, and then calling projectInfo would be a workable solution for you? if so, as i noted earlier we can make projectInfo work on tsconfig.json as well as .ts/.js files.

so filtering the tsconfig.json files, and then calling projectInfo would be a workable solution for you?

I think it can be a good solution.

if so, as i noted earlier we can make projectInfo work on tsconfig.json as well as .ts/.js files.

It should be very cool!

A PR would be appreciated.


corrections submit to server

Current dependencies on/for this PR:

main

PR #2

PR #3

PR #9

PR #10   👈

This stack of pull requests is managed by Graphite.


Write unit tests for sorting EM-gain, k-gain and non-linearity calibration data

This is a feature that comes as one of the to-do tasks of this parent issue about EM-gain, k-gain and non-linearity: https://github.com/roman-corgi/corgidrp/issues/226
The task is:
Write unit tests that cover the file diversity that mat happen when collecting calibration data for EM-gain, k-gain and non-linearity

Finished in PR #247


Terser output for ES6 import

Feature request

Option to optimize away extraneous property access for ES6 module imports.
What is motivation or use case for adding/changing the behavior?
With ES6 modules a typical import looks like
import {fn} from 'lib';
fn();

...resulting in something like
var _lib = __webpack_require__('lib');
_lib.fn();

I understand this allows for non-constant exports, handling of circular dependencies etc.. But in 99% (my gut feeling) of the cases the extra propertly lookup is unnecessary and you can do fine with (assuming ES2015 output)
const {fn} = __webpack_require__('lib');
fn();

This may look minor, but has some advantages:

Easier to parse for the eye during debugging
Easier to run intermediate code during debugging (e.g. calling fn() manually) since you don't have to lookup the name of the generated module import
Faster (I admit - no benchmarks)
Allows for smaller bundle size

The hard part:
Need to determine if the exported variable is a constant and that there are no circular dependencies between the modules in question.
How should this be implemented in your opinion?
This could technically be implemented as a minification step, but I think it'd be hard to do the module dependency detection. And...the benefit that it'd be easier to read in development wouldn't apply.
That leads me to think that this should be handled in webpack itself.
Are you willing to work on this yourself?
yes - I would benefit from some directions. More specifically about how to detect circular dependencies.

Webpack supports ES5 compatible browsers. Sadly, destructuring is not part of that.
I also doubt there would be any performance improvement here.

@Janpot ES5 compatibility is of course important. I’m not suggesting this should be a default behavior, only someting turned on through some config propeerty - primarily for use during development and possibly when targeting new browsers only.
As for speed - yeah consider it a minor (if applicable at all)

I also want to add that the import statement
import {fn} from 'lib';

is not the same as destructuring assignment. Even though it looks very similar.
for instance
import {a:{b}} from 'lib';

Is invalid syntax even though
const {a:{b}} = foo;

is perfectly valid.

Technically you are right we could optimize some cases. Scope Hoisting already optimize most cases in production mode. I guess it's not worth the work.

I don’t see how that’s relevant.

Nevermind, seemed like you were suggesting these were equivalent constructs. Just wanted to make sure.
To be honest, I don't understand what's the optimisation here. no speed or correctness improvements. Would this allow for better tree-shaking? Maybe there is a slightly improved readability in code that is not meant to be read. At the cost of added complexity and an extra configuration point.

I don't understand what's the optimisation here

@Janpot As I wrote in my original post, the primary benefit would be easier debugging.

Maybe there is a slightly improved readability in code that is not meant to be read.

What do you mean with "not meant to be read"? My code is full of Object(myLib['fn'])(). It's meant to be read when in the debugger. People (including myself) spend significant time there. You and @sokra may not think "it's not worth the work" - a normative statement where we clearly disagree. I guess it's a matter of how much time you spend reading (and manually executing) code in the debugger.

What do you mean with "not meant to be read"?

It's transpiled code. It's not optimized to be read. Aren't sourcemaps working as expected for you here?

@sokra may not think "it's not worth the work" - a normative statement where we clearly disagree.

Making transpiled code debuggable is definitely worth the work. I don't think we disagree on that. What we probably disagree on is the way to achieve this.

Aren't sourcemaps working as expected for you here?

Source maps destroy the mapping between shown variable names and "real" variable names, meaning I can no longer

hover a variable to see its value
type the value in a watch inspection or the console to inspect its value

Furthermore, they are slow. Hence they're not an option for me (and I'm sure many others)

Making transpiled code debuggable is definitely worth the work. I don't think we disagree on that. What we probably disagree on is the way to achieve this.

Ok, nice! What's your solution to this problem then?

Also I might add that const and destructuring is by no means required. You might as well go with something like
var _lib = __webpack_require__('lib'), fn = _lib.fn;
fn();

Note that afaik FF debugger can handle these import indirection in SourceMaps and allows to use fn in console and watch.

@sokra Just tested with FF 60.0.1. Did not work

Is it possible to statically verify a binding will never be updated if it isn't declared as const? I'm not sure the export const ... satisfies 99% of the use-cases as you claim.

Great question. Yes, I think this is definitely the challenging part. I did a quick search on my current code base (sure, not scientific at all, but at least a hint). ~98% (const, function, class). ~2% let. The same with angular (@6.0.0) yielded 100% const,function, class (and re-exports).  But...if you import the non-ES2015 angular module, you wouldn't have const etc. there so that'd be harder. You could resort to statical analysis, or always err on the safe side for libraries then. Or provide a config flag like unsafeLibImportShortening for libraries (since I can't recall ever stumbling upon a library that did some reassignment to an export, but of course there exists such a library). Left then are circular dependencies of course.

Sounds like there's a problem with sourcemaps. Do you think this is rather a problem with the specification, a missing browser implementation, or a missing webpack implementation?

Again, good question. Judging by this thread https://bugs.chromium.org/p/chromium/issues/detail?id=327092 it seems the symbol mapping feature exists in Chrome, so an example such as
import * as lib from 'lib';
debugger
console.log(core);

...should work, but it doesn't. I should really read up to give you a better answer but if I were to guess, I'd say that:

Webpack should provide symbol mappings for the case above, and it doesn't (?)
I'd guess the source map spec probably doesn't support symbol mapping to a property access of another symbol, which is ultimately what this use case is about.

Ok, nice! What's your solution to this problem then?

My opinion is that this is an issue with sourcemaps, and needs to be solved (or attempted to be solved first) in that area. I think your proposal will be incompatible with ES6 modules spec (live bindings) and will hinder future alignment with the spec. I don't think what you propose is significantly more readable, and I don't think readability of output should come before correctness or speed.
Prior discussion: https://github.com/facebook/create-react-app/issues/1557.

yes, because of 2.

Note that SourceMaps don't have range mappings only position mappings.
I tried to map Object(a.b) to x and x yields Object. I also tried to map a.b to x and x yields a.
so this is a bit broken (in the spec?)

Note that you can throw function and class in the same category as const, because they are not const:
export class A {}
export function B {}

// Don't do that, but it works
A = 1; B = 2;

Next challenge: const is not really const. It actually has two different values. TDZ and the value.
// a.js
// ...
export const X = 1;
// ...

// somewhere else
import { X } from "./a"; // never throw here
export function Y() {
  return X; // may throw TDZ error here
}

Rewriting this with your proposed style changes the semantics.
const { X } = ...; // may throw here too
export function Y() {
  return X; // may throw TDZ error here
}

so this is a bit broken (in the spec?)

@sokra thanks for clarifying

Note that you can throw function and class in the same category as const, because they are not const

Good point. This is a strong argument for how hard it would be to build a ES2015 compliant fool-proof solution

Next challenge: const is not really const. It actually has two different values. TDZ and the value.

Sure, but doesn't this only apply to circular dependencies? There'd be no TDZs in the unidirectional dependency graph case, right? Or am I missing something?

@Janpot

My opinion is that this is an issue with sourcemaps, and needs to be solved (or attempted to be solved first) in that area.

I'm not against this, but don't see it happen anytime soon. Also, source maps take significant time to produce in webpack builds, so they're not this unicorn either. In general I'm very skeptical to the "should be solved by someone else" argument. There is a problem, here and now. Deal with what you can control to solve it.

I think your proposal will be incompatible with ES6 modules spec (live bindings) and will hinder future alignment with the spec.

I think there are basically two ways of implementing import rewriting - one pragmatic way and one robust way. In the pragmatic way people would have to turn it on if wanted during development, potentially facing some incompatibilities for edge cases, because it's a tradeoff but they still value this. For the robust case, a prerequisite for rewriting would be guaranteed compatibility. Then by design they would be compatible (or no rewriting should take place). But it would be much harder to pull that off.
I fail to see how any of these approaches would hinder alignment with future TC39 work. For the pragmatic approach it's an opt-in that you go for knowing about the risks. Or, for the robust approach compatibility should be guaranteed. I think @sokra made a good case for that you actually need to statically analyze if a variable is overwritten for e.g. function cases. Not impossible, but lots of work it seems, potentially hurting performance. There's always gonna be edge cases, e.g. with eval where it'd be pretty close to impossible to always be correct.

I don't think what you propose is significantly more readable, and I don't think readability of output should come before correctness or speed.

Speed should be faster or the same if you mean outputted code. If you mean build time you may be right depending on ambition. Again, my strongest argument has been ease of debugging - which you do not address. As for correctness, I think it only matters if it's ever noticeable. Building the robust approach above, that should be a requirement. Going for the pragmatic approach, then it's an opt-in and you'd pay the consequences. But if the compatibility is bad, I can understand why the webpack core couldn't/wouldn't stand behind it
So, assuming a solution that builds on a plugin or opt-in configuration...let's sum up the arguments
+ better debugging experience
+ somewhat better readability
- potentially slower builds
- edge case incompatibilities that will increase webpack support issues or problems
I clearly value the pros higher than the cons.
So, maybe I'll go with a plugin instead.


fixed incorrect return type for bool ORBSet::erase(...)

bool ORBSet::erase(...) originally had a return nullptr. It should return a false instead.

@mcharleb Could you please have a look and comment?

Yes, looks good. Please merge.


Update imgui package to 1.70

I've followed this guide
step by step carefully. Yes

I've tested this package remotely and have excluded all broken builds.
Here is the links to the Travis/AppVeyor with status "All passed":

https://ci.appveyor.com/project/caseymcc/hunter/builds/24908177
https://travis-ci.org/caseymcc/hunter/builds/538965599

Testing:

https://ci.appveyor.com/project/ingenue/hunter/builds/24915437
https://travis-ci.org/ingenue/hunter/builds/539091697?utm_source=github_status&utm_medium=notification

Released: https://github.com/ruslo/hunter/releases/tag/v0.23.181


Add the Cumulative Residual Entropy

The CRE would make a good extension module: http://citeseerx.ist.psu.edu/viewdoc/download?doi=<IP_ADDRESS>.6754&rep=rep1&type=pdf
This should be a relatively easy thing to code, at least in the discrete case.

Such discipline with the documentation. I'm taking notes...


Rely on WN '/bin/env' for the trace command (SOFTWARE-3387)

https://opensciencegrid.atlassian.net/browse/SOFTWARE-3387
Also part of the problem on Fermicloud VMs was the location for env that got added to the job submission existed on EL7 but not on EL6:
$ which env
/usr/kerberos/bin/env

Should be /usr/bin/env, not /bin/env.
Fermicloud VMs have an annoying property that /usr/kerberos/bin is a symlink to /usr/bin and comes before /usr/bin in $PATH.


put license and notice in all jars

I updated the NOTICE to match what we have in main pekko repo
From my understanding, we do not need to include HPack license - or even include details in our NOTICE file

it is Apache licensed
https://github.com/twitter/hpack has no NOTICE so we don't have copy that non-existent notice into our notice

From my understanding, we do not need to include HPack license - or even include details in our NOTICE file
* it is Apache licensed

* [twitter/hpack](https://github.com/twitter/hpack) has no NOTICE so we don't have copy that non-existent notice into our notice

Not sure I fully agree.

We should note somewhere that we included code and the notice file seems a good place
Since we are adapting akka-http we still have to keep akka-http's NOTICE file (based on the Apache License terms), so we cannot just delete it because we think the information is not useful to keep...


document_diagnostics not automatically refreshing

Not sure when it started happening, but I'm no longer seeing document diagnostics automatically refreshing in my loclist when I make correction or introduce new errors. Workspace diagnostics still correctly auto-update, but document level one don't. My setup isn't too far out of the ordinary, but I'll try to slim it down to help provide a minimal repro setup!
I'm running latest off main.

hmmm was able to trouble to https://github.com/nvim-lua/kickstart.nvim and everything worked as expected! Must be an issue buried somewhere in my config. Strange though, because I reverted back to a time where I swear I had things working! I'm sure I'll figure it out. Thanks!


Feature/delete endpoint

This PR adds support for an /uninstall endpoint.
The uninstall action uninstalls a release provided a release name.
It also supports additional flags provided by helm.

Codecov Report

Merging #7 (afa88c4) into master (3154982) will decrease coverage by 3.54%.
The diff coverage is 54.54%.

@@            Coverage Diff             @@
##           master       #7      +/-   ##
==========================================
- Coverage   68.44%   64.90%   -3.55%
==========================================
  Files          11       14       +3
  Lines         225      302      +77
==========================================
+ Hits          154      196      +42
- Misses         54       83      +29
- Partials       17       23       +6

Impacted Files
Coverage Δ

pkg/helmcli/client.go
46.87% <0.00%> (-11.95%)
:arrow_down:

api/uninstall/uninstall.go
57.57% <57.57%> (ø)

api/uninstall/service.go
73.33% <73.33%> (ø)

pkg/helmcli/uninstall.go
100.00% <100.00%> (ø)


aframe-layout-component: line layout along Y and Z axis

The line layout seems to be limited to X axis only. It would be nice to support line layout along Y and Z.

Agreed. In order to achieve a line along the Y axis, I rotated the group by 90 degrees and each element by -90. ;-)

I think rotating it is sufficient than expanding the API?

For me rotating is ok.

Actually, I will work on this because this is preferable to rotating. Rotating changes the whole object transform which can be annoying.

Fixed.
layout="type: circle; plane: xy"
layout="type: line; plane: yz"
layout="type: box; plane: xz"


Need Enterprise-class configurability

If Teams is really supposed to supplant Skype in the near future, then some effort needs to be put into making it Enterprise-ready!  I can't have users getting popups from Windows firewall and just tell them to click Cancel!  How is that possibly acceptable.  GPO-based configuration for the client (along with an enterprise-deployable package) is needed, and at least documentation on how to configure the client-side firewall rules.  Telling users to "click Cancel" just sets a bad precedent for user behavior.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6457d204-d229-9154-28fd-ca9bbb4bddcd
Version Independent ID: f5d82846-41cb-3548-6997-ce23bd8f89c6
Content: Get clients for Microsoft Teams
Content Source: Teams/get-clients.md
Service: msteams
GitHub Login: @LolaJacobsen
Microsoft Alias: lolaj

@KyleSchroeder Thank you for your feedback.
I want to get this to support and our Teams team so it can be tackled on both ends. I am getting this sent over to the SharePoint teams backlog.
I also need you to send this in as a request to the support team.
Here is a quick video on opening a ticket: https://www.youtube.com/watch?v=puuOy0sEg1Y
If you don't mind please keep us posted here on the resolution and I really want to get this information you are discovering back into the docs.

@KyleSchroeder - I know the issues you brought up are on the radar of the Teams product team. There is an active Enterprise community discussing and up-voting these items on teams.uservoice.com and the product team uses that input as they work on their feature plan. I also know some of the Teams product teams and will bring these points up to them in person and if you can up-vote items you agree with on teams.uservoice.com that will be super helpful too. Thanks for taking the time to submit this feedback and make the product better for Enterprise. All the feedback I can provide to the Teams product team is beneficial and they listen to all of it I know for a fact. Thanks, Ken

@kenwith - I attempted to access teams.uservoice.com and it is reported as no longer available.  Where can we go to make our needs known?  I'm looking to deploy Teams in a non-persistent virtual desktop. There is no way I can expect my users to download and install this every day. I need an installer for the master image, and GPOs to help with configuration.

@cbujak Here is the url I use and I just checked that it works for me. https://microsoftteams.uservoice.com/forums/555103-public

@KyleSchroeder  Thank you for submitting this feedback.
From our understanding the issue has been resolved. If you feel it hasn't been resolved please re-open this issue.

@officedocsbot close


Introduce AiService class and method names into GenAI metrics

This is done using the Duplicated Context as currently LangChain4j doesn't provide a more fine-grained way of providing information to the ChatModelListener.
Nevertheless, the use of Duplicated Context is done in way that ensures that the tags will only be added if the Duplicated Context exists. If they don't, metrics continue to work.
An example metric using this is:
gen_ai_client_estimated_cost_total{ai_service_class_name="io.quarkiverse.langchain4j.sample.MyAiService",ai_service_method_name="writeAPoem",currency="USD",gen_ai_operation_name="completion",gen_ai_request_model="gpt-4o",gen_ai_response_model="gpt-4o-2024-05-13",gen_ai_token_type="output",} 0.00327

@jmartisk the org.acme.example.openai.RAGWithMetadataFilterTest is flaky in quarkus-langchain4j-integration-test-rag-pgvector-flyway.
Any chance you can take a look?
Thanks

@jmartisk the org.acme.example.openai.RAGWithMetadataFilterTest is flaky in quarkus-langchain4j-integration-test-rag-pgvector-flyway.
Any chance you can take a look?
Thanks

Ok looking

🙏🏽

I ran the tests (rag-pgvector and rag-pgvector-flyway modules) locally 200 times in a loop and no failure...
The error ("cache lookup failed for type 16386") looks like something from deep inside postgres that we may not be able to control, apart from trying to use a different postgres version.
Or it may be related to the reuse of containers between test modules (I tried with Ryuk disabled as well as enabled).
I've submitted https://github.com/quarkiverse/quarkus-langchain4j/pull/944 but whether that will help anything, I don't know


Image resizer - option to always save as jpg (or whatever format)

Description of the new feature / enhancement
I would like to be able to resize and convert with one click. I have a lot of PNGs and sometimes I want to send a smaller copy in email or whatsapp or whatever. That is when I most often use the resizer. I would like to have the option to automatically save the resized file as a jpg even if the original was a PNG.
Scenario when this would be used?
I think I would leave the option on and use it every time I use the image resizer.
Now I just have to do an extra conversation step.
Supporting information
No response

/dup https://github.com/microsoft/PowerToys/issues/2686


[WIP] Rework types, remove protos

Still reviewing, but don't we lose the ability now to do something like "given a JSON object, turn it into the correct dataclass type"? With proto we could do this because there was a type attached to it.

I'm going to go back to the single List[Entity] version of the document entities, and make it work in the JSON conversion code (attempt to?), so this won't be missing anymore.

@ddanco let me know when you want me to review again!

@ddanco let me know when you want me to review again!

@vontell Ready!

Sorry for the delay! Looking at this today @ddanco

The serialization code for a Document is in document.py (load_fnd_doc_from_json), that will serialize a Document and all the Entities inside of it, but I should add a separate function for just Entities, I'll do that.


Enable Scala 2.11 cross-compile, upgrade sbt version.

Design Review: None
Testing Done: Ran test. Looking for review comments on what to test.

LGTM


Include jsii-rosetta examples

The AWS CDK API reference includes example code snippets for most constructs, classes, structs, and enums. Example. These are generated from information within jsii "tablet" files (if you install aws-cdk-lib, within the package you should find a .jsii.tabl.json file).
However, these are not displayed when you generate an API reference with jsii-docgen (and hence, they're not shown on Construct Hub). Example. We should find a way to include these since many users find them helpful given that CDK is vended in multiple languages.

The AWS CDK API reference includes example code snippets for most constructs, classes, structs, and enums. Example. These are generated from information within jsii "tablet" files (if you install aws-cdk-lib, within the package you should find a .jsii.tabl.json file).
However, these are not displayed when you generate an API reference with jsii-docgen (and hence, they're not shown on Construct Hub). Example. We should find a way to include these since many users find them helpful given that CDK is vended in multiple languages.

+1
Ideally based on the target language specified in the args to jsii-docgen, it should parse all @example and .md files, run rosetta to generate the tablet and then substitute into the generated docs the language appropriate substitution. This will allow the ability to simply pipe this into something like mkdocs to generate a micro site for any language binding with minimal additional work.

Did a little digging...
It appears https://github.com/cdklabs/jsii-docgen/blob/main/src/docgen/schema.ts#L498 is not adding the example to the returned doc. Additionally in https://github.com/cdklabs/jsii-docgen/blob/main/src/docgen/render/markdown-doc.ts#L104 the doc renderer is not emitting a line for the example. This can be easily fixed by adding the following:
schema.ts
function extractDocs(docs: reflect.Docs, language: Language) {
...
    return filterUndefined({
       ...
        example: docs.example.length > 0 ? docs.example : undefined,
        language: language
    });

markdown-doc.ts
docs(docs) {
    ...
    if (docs.example) {
        this.code(docs.language, docs.example);
    }
}


Load json file for topology

Some updates:

Enable loading json file to draw topology graph
Add example json file

Thanks for the fast review! Merging now !


Disable shallow git clone in Travis

Travis sets a git clone depth to 50 by default.
This change makes it effectively unlimited.
Resolves a warning in SonarCloud:
"Shallow clone detected during the analysis. Some files will miss SCM
information. This will affect features like auto-assignment of issues.
Please configure your build to disable shallow clone."

Codecov Report

Merging #1185 into master will decrease coverage by 11.55%.
The diff coverage is n/a.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c4bc78...e2f3f4d. Read the comment docs.


@everyone Setup on PC

Please set up the project on your PC and start familiarising yourself with the codestructure.
Everyone should have a rough understanding of the project and be able to set it up and run it.
The people in this project with roles of the more organizational type have agreed to help the developers when necessary.
Not having the project setup already, and therefore not being familiar with it, creates unnecessary overhead, when additional manpower is needed.
Please react to this issue with a comment or otherwise to signal that you have successfully started the project on your PC and have driven a virtual car via your smartphone.

Funktioniert

Funktioniert nicht ganz, wäre cool wenn wer helfen könnte die Tage

funktioniert

@devrimy7 Eine Ahnung woran es liegen könnte?

funktioniert

funktioniert

Setup has worked for everyone.


Remote-SSH: multiple remote connections in a single window

The Remote-SSH extension should support multiple SSH targets; indeed, when working on multiple machines (e.g. one for development and one for test), it would be great to have the whole situation under control in a single window.

Sorry, our architecture doesn't support this, and it's not likely to be supported in the near future.


Snips addon ver 5.3 : issue on the NLU engine

The last version of Snips addon (5.3) is running with a NLU model (ver 0.19.0) not aligned with the model used bt the Snips console (ver 0.20.0).
Following the error message when start the Snips addon  with an assistant.zip made by the Snips console:
INFO:snips_nlu_hermes: loading nlu engine "/usr/share/snips/assistant/nlu_engine"
ERROR:snips_nlu       : Unable to read file '/usr/share/snips/assistant/nlu_engine/nlu_engine.json'
-> caused by: Mismatched model version: model is 0.20.0 but runner is 0.19.0

What needs to be done to fix this. I'm unable to use this addon because I'm starting on this version and don't have any previous assistants to fall back on

Maybe an version update?


allErrors not working with multiple forms on the page

When I have two forms on a page as follows:
	const {
		allErrors: form1AllErrors
	} = superForm(data.form1)
	const {
		allErrors: form2AllErrors
	} = superForm(data.form2)

and the forms are initialised on the server with errors: true, $form2AllErrors contains the errors for form 1.

I've just fixed it by specifying IDs for each form when initialising them on the server, but feel this is something which the library could handle internally.

Id handling will be pretty much automatic in 1.0, unless the forms are using the same schema, which requires setting the id manually.

Cloning this, as it will be fixed in 1.0.


dashing!

how about animating the camera distortion effect

should the camera distortion effect be positive (left) or negative (right) @notangoose


fix(analyticsfunction): made all param types optional

In order to give users more flexibility on how they use the analytics function, all arguments are now optional

:tada: This PR is included in version 1.20.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add FalconMessenger by @RMizin

[x] Project URL: https://github.com/RMizin/FalconMessenger
[x] Update contents.json instead of README
[x] One project per pull request
[x] Avoid iOS or open-source in description as it is assumed
[x] Use this commit title format if applicable: Add app-name by @github-username
[x] Use approved format for your entry

@RMizin thank you!


Rendering 99% of the gaussians despite not looking at their direction

Hello,
I am wondering if there is a way to make this more effective? As of now, it seems that the renderer is still rendering the majority of the splats despite not looking at their direction. If I for example look away to the emptiness, I might render around 6% of the splats.
What are the factors that determine how much is rendered? See image as an example where I am high above the ground and looking at the distant (non-important) splats. Thank you! :)

Any idea?

The "Rendering" line in the info panel actually refers to the number of splats being sorted for rendering. The reason you're not seeing the expected drop in that number is that the viewer is intentionally selecting a significant number of splats that are outside the view frustum to be sorted. This is so that if the user quick rotates the camera, there won't be a bunch of black space on screen during the time when the sorting algorithm is catching up to the new camera orientation. These splats are also sent to the GPU to be rendered, but the shaders discard them if they are outside the frustum.

Thank you!!


Support cell array of strings and matrices for colours

This would be better than just a char array, and could allow mixing line spec strings for colours along with arrays.

Also, would be good to support 'none' values mixed in with actual colours.


SLING-4377, add JSR 305 annotations to Sling API

I am going to merge another cleaned up pull request (https://github.com/apache/sling/pull/64).


[1.17] Add support for multiple schemes to the SUI

Hmm. I think it would be better to handle this in the viewmodel. Right now a reader needs to keep in their head that the setting is displayed if there's a Dark color scheme, but then it just edits CurrentColorScheme. What does that mean?
IMO, "Has Dark" and "Edit Both" should be hidden as implementation details inside the VM.

Originally posted by @DHowett in https://github.com/microsoft/terminal/pull/14064#discussion_r1023275324

Talked with Dustin. Punting this to try and tighten up 1.17. We'll think about servicing this back to stable 1.17.

Removed from Blocking, removed from 1.17 Servicing


Horizon showing multiple IPs, nova client showing TAP devices on multiple hosts

Customer is hitting this after a churn test.  It may be the same as @DaveLangridge's #515 bug.  They were running 0.17.
In horizon, a VM shows as having mutliple IP addresses.  Listing ports at the nova CLI, they found that nova believes that there are two TAP devices on different hosts with the same UUID.  One of them is active and attached to the host but felix thinks the other one is.

Now think this is a dupe of #515


set up app container networking before containers start

Currently 'weave run' sets up the app container's interface into the weave network after the container has been launched with 'docker run -d'. That means the network may not be available to the container process straight away. Depending on what the container is doing, that can be benign, annoying, or disastrous.
Containers can themselves ensure that the interface is available, by running s.t. like https://github.com/jpetazzo/pipework/blob/master/pipework#L30, i.e.
while ! grep -q ^1$ /sys/class/net/ethwe/carrier 2>/dev/null
do sleep 1
done

before starting the actual container process, but this of course requires containers to have been constructed with weave in mind, which is limiting.
There is no way around this issue w/o some changes to docker.

One challenge with the "rewrite entrypoint to a volume-mounted exe" approach is making it work when we don't have direct access to the docker host and instead run weave itself in a container (as per #312)? Where would the exe live s.t. we can volume-mount it into containers?
The answer is --volumes-from... As part of weave launch we start a named container that copies the required exe into a volume created with -v. For example,  we could add the exe, and the program to copy it, to the weavetools image, in which case we would start that container with something like docker run --name weavetools -v /home/weave zettio/weavetools /bin/copy-entrypoint-exe /home/weave. We can then start all other containers with --volumes-from=weavetools.
Note that weavetools doesn't need to remain running for this to work, though the container needs to continue to exist. We'd need to take care of removing it with docker rm -v on weave stop.

Trying to extract the orginal entrypoint from the command line (in case it was specified there), and determining the image from the command line (so we can lookup the entry point specified in that), is fiendishly hard. Go's option parser library flag, which is what docker uses, allows options to be specified in a myriad of ways. Moreover, there is no generic parsing function; parsing requires a specification of all the options. This not not just a matter of missing APIs; the option grammar is ambiguous in the absence of an option specification - consider the difference between docker run -i a b (a is the image and b is the command) and docker run -h a b (a is the host and b is the image).
The best option I can think of here is to invoke docker's runconfig.Parse function. This performs nearly all of the docker run option parsing and appears to require little/no context. However, it is missing some options, because the function is also used to for parsing docker create, so any options not shared with docker create would have to be added by us before invoking the function.
Furthermore, the whole thing is highly version specific; there's probably not been a single docker release when the docker run options haven't changed.
Finally, there is no pretty-printer, so one cannot, say, parse a command line, then remove/add/modify some options, and then produce a modified command line. Which is what we need to do here.

Trying to extract the orginal entrypoint from the command line (in case it was specified there), and determining the image from the command line (so we can lookup the entry point specified in that), is fiendishly hard.

Am I understanding right that this implies docker run --entrypoint=foo will not reflect .Config.Entrypoint? This doesn't look like it's been reported yet, is it considered a feature?
Separately from "entrypoint" overwriting approach, have we considered docker pause (or SIGSTOP) with consequent docker unpause (or SIGCONT) approach? The question is whether set-up of network namespace continues while container process is paused?

Hey guys sorry for delay have been out of action for a coupla months...
@rade Great idea r.e. mounting the exe from a volume - makes this whole thing much more portable and allows the exe to be distributed as a container.
It's mad how fiendishly hard parsing the docker run args can be - I made start in go (which is new for me) and hit that same problem of 'where do the docker arguments end and container arguments begin'
Had not thought of using dockers runconfig.Parse however - that plus some extra wrapping in it's own go library called parse-docker-args would be a good start - I will pick this up again this week and see where my beginner golang skills get me :)
@errordeveloper pausing the container would be great if it works!
The challenge I suppose would be how to check /sys/class/net/ethwe/carrier from the outside of the container and if the initiation of the network continues for a paused container.

@errordeveloper

Am I understanding right that this implies docker run --entrypoint=foo will not reflect .Config.Entrypoint? This doesn't look like it's been reported yet, is it considered a feature?

Not sure what you mean. The whole point of --entrypoint is to be able to override the entrypoint configured in the image.

have we considered docker pause (or SIGSTOP) with consequent docker unpause (or SIGCONT) approach?

That's just racy; there's no way to guarantee we'd be able to pause the container before it attempts to access the network.

@squaremo suggested an alternative technique for modifying the entrypoint: run a docker proxy. i.e. a process (in a container, naturally) that the weave script would perform all docker actions on (using DOCKERHOST=... docker ... or docker -H ...), and that forwards requests/responses to/from the real docker daemon. In the process it could rewrite the entrypoints, which, since it is speaking the docker remoting protocol, are easy to pick out and modify.
That same proxy could also take care of #251.

@rade yes the proxy would be a great way to bypass the cli arguments issue - I've been messing with a docker proxy in node - mdock - and it works a treat because as you say, the arguments are now in nice, already processed JSON format.
When doing this, I had lots of fun and games realizing that docker run was (as #251 points out) in fact a sequence of commands rather than a single action.
So perhaps I can make a simple container based on mdock to quickly test this idea (which feels much better than command line processing the arguments).

@binocarlos Do you have a fork of mdock? It looks like it's meant for single-host <=> many-hosts, rather than single-host docker API proxying.
I'm working on my own single-host Docker proxy in Twisted, but would be curious to see the node version.

@lukemarsden I've published mdock onto npm here (so npm install mdock will grab the code locally).  You are right though - it's trying to be a single entry point to multiple docker servers - not the single destination proxy we need here.
Slight distraction: I had originally called this library flocker (as in flock of dockers) - then I realised clusterhq existed and so renamed it to mdock (multiple dockers) - you are welcome to take over the npm package flocker at any time :)
@rade  - here is a rough plan for the proxy I have in mind - I have no idea if a) it will work or b) how it might integrate into weave so feedback is welcome!
We need a container that can be named 'weave-proxy' and:

has the weave command
has --net=host -v /proc:/hostproc so weave can mess with the system (i.e. #230)
has access to the docker host (via HTTP 2375 or mounted /var/run/docker.sock)
publishes a tcp port and listens for HTTP (.e.g. port 2475)

We need another container that contains a wait-for-weave binary in a volume.  This binary will wait for the weave network to be ready and then execute its arguments as a command.  So echo "weave rocks" becomes wait-for-weave echo "weave-rocks".  Any container that has '--volumes-from=wait-for-weave' can then access that binary.
The HTTP proxy port (2475) will forward all requests to the docker server (2375) by default.  Any requests to POST /containers/create will be intercepted and the JSON packet processed as follows:

add a --volumes-from=wait-for-weave
change the entry-point to wait-for-weave $@

Any requests to POST /containers/:id/start will:

ensure the entry-point is wait-for-weave $@
extract $CIDR from the env or auto-create it
forward the request to the docker server
the container is now running but blocked by wait-for-weave
capture the returned $CONTAINER_ID
run a weave with_container_netns $CONTAINER_ID attach $CIDR style command
run a weave tell_dns PUT $CONTAINER_ID $CIDR command
return the $CONTAINER_ID to the docker client

We point the docker client at our proxy:
$ export DOCKER_HOST=tcp://<IP_ADDRESS>:2475

Now we can run long-lived servers:
$ docker run -d --env WEAVE_CIDR=<IP_ADDRESS>/24 mystack/mysql

And interactive jobs:
$ docker run --env WEAVE_CIDR=<IP_ADDRESS>/24 ubuntu bash -c "ping <IP_ADDRESS>"

The interactive job will be blocked by wait-for-weave before attempting to ping <IP_ADDRESS>.
There are a couple of problems I can see:

point number 3 from #230 - i.e. how do we know what CIDR address to instruct weave with
returning the $CONTAINER_ID to the docker client before wait-for-weave has run the container's original entry point (perhaps adding a delay might help)

I'll give this a try this week and discover all the things that I've missed out :)

@binocarlos Thanks for this. By the way, do you live in Bristol? (So do I...)

@lukemarsden No problem - yes I live in Bristol - its a great city!
Perhaps I should come and say hello sometime - I met Richard Wall a few months ago @ a meetup in the engine shed and he was telling about what you guys were doing upstairs.

I'm making good progress with a container that:

runs weave from inside a container
presents a docker API proxy that will

rewrite the entry-point for /containers/create
add --volumes-from=weavetools

will run weave attach before returning the container id for /containers/:id/start

In the meantime - I've uploaded wait-for-weave which is the golang program that will wait for /sys/class/net/ethwe/carrier before running its args as a command.
This binary is what will be mounted in a volume in a container called weavetools
@rade if you could check this (golang being very new to me) I would be most grateful :)

@binocarlos you may want to use EnsureInterface

@rade thanks! - that is far more civilized :)
I've added an exit with a non-zero code if it decides there is no ethwe to be found - I'm assuming this is a good plan rather than try to run the entrypoint anyway.

I've added an exit with a non-zero code if it decides there is no ethwe to be found - I'm assuming this is a good plan rather than try to run the entrypoint anyway.

Agreed.

Hi, @rade
Is there any update in docker that can let docker wait for NIC ready in a Docker way? I searched but only found this thread is almost same with what I needs. I saw that you are using another container to replace entrypoint and then run the original one after weave device ready.

@tangzhankun The answer is Docker Network Plugins. Which obviously does a whole lot more than just wait for an entrypoint.

hi, @rade
oh. Thanks for the quick reply. Writing a plugin is obviously overkill for my needs. I think using the wait in entrypoint is enough. BTW, if I want to wait custom NIC name, like eth0, or if0. Will wait-for-weave support this in the future? Or I should build a wait-for-NIC?

if I want to wait custom NIC name, like eth0, or if0. Will wait-for-weave support this in the future? Or I should build a wait-for-NIC?

Submit a PR :) This should be a straightforward extension.

hi, @rade
:) I'll do it if I have time in the future. Haha. Thanks again.

Is there somekind summary, what is correct solution to fix this. There is situations like DNS server in docker & weave networks, which needs to be up and running before other containers in reboot.

@apassi, what are you looking for exactly?
Weave Net's Docker proxy already waits for the NIC to be ready before your container gets started, and Weave Net's Docker plugin relies on Docker to do that, but leads to the same outcomes, so all dependencies should be ready by the time your container's ENTRYPOINT gets run.

Basically i like to understand how weave and containers handle host's reboot situation. I havent able to find any "internals" document about weave. After i run weave launch, there seems to run new weave related containers, weavedb etc. but i havent been able to find documents for those..

@apassi, I understand what you meant now, thanks clarifying it.
Indeed, details on the internals of Weave Net, when present at all, are very much diluted in the online user docs. You can find lower-level details in our developers docs, but ultimately the code is the source of truth.
If you think a "Weave Net's internals" documentation could be useful, feel free to open a new GitHub issue (as this is unrelated to this one) and to list exactly what you would like to see there, since as developers, we are biaised and may think some details are obvious. Thanks!


Editing logpoint condition results in editing logpoint message

Testing #114345

Create logpoint
In breakpoints viewlet, Right click > Edit condition...
The logpoint message editor comes up.

Yeah that is expected behavior.
For breakpoints that are tied to a source location clicking on Edit Condition -> we open the breakpoint widget in the source location.
I guess you would expect that the Edit Condition would automatically set the dropdown to condition for logpoints which I think makes sense. I can leave this open for now to get more user feedback.

Yeah that is expected behavior.
For breakpoints that are tied to a source location clicking on Edit Condition -> we open the breakpoint widget in the source location.
I guess you would expect that the Edit Condition would automatically set the dropdown to condition for logpoints which I think makes sense. I can leave this open for now to get more user feedback.

Well, when I right-click the logpoint in the editor itself it says 'Edit Logpoint...', and then to me it makes sense that the message editor shows up.
I'm wondering if for the breakpoint viewlet, right-clicking logpoints should also bring up 'Edit Logpoint...' or something along the lines of 'Edit message...' instead of 'Edit Condition...'

Well, when I right-click the logpoint in the editor itself it says 'Edit Logpoint...', and then to me it makes sense that the message editor shows up.
I'm wondering if for the breakpoint viewlet, right-clicking logpoints should also bring up 'Edit Logpoint...' or something along the lines of 'Edit message...' instead of 'Edit Condition...'

Yes, I like that idea! Let me assign to this week to look into fixing that
Thanks for feedback

Yes, I like that idea! Let me assign to this week to look into fixing that
Thanks for feedback

We currently do not allow commands with generic titles, so I would have to introduce a new command just for a different title "Edit Logpoint". It makes sense but is a bit clunky. Thus pushing out of this milestone for now.

We currently do not allow commands with generic titles, so I would have to introduce a new command just for a different title "Edit Logpoint". It makes sense but is a bit clunky. Thus pushing out of this milestone for now.


chore(ci): separate fork and local test runs

https://ci-fix--design-website--adobe.hlx3.page/

@pfulton when you get a chance, could you please give this a look? This separation brings design-wesbite in sync with the project boilerplate and will split tests run for fork-based PRs from local ones. It will likely fix the sporadic linting failures seen in PRs of this project.


Add release notes for 0.6.0.

Codecov Report

Merging #44 (17d21b0) into main (60bade1) will not change coverage.
The diff coverage is 0.00%.

:exclamation: Current head 17d21b0 differs from pull request most recent head daee42b. Consider uploading reports for the commit daee42b to get more accurate results

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@          Coverage Diff          @@
##            main     #44   +/-   ##
=====================================
  Coverage   0.00%   0.00%
=====================================
  Files          7       7
  Lines        498     486   -12
=====================================
+ Misses       498     486   -12

Impacted Files
Coverage Δ

zyte_common_items/base.py
0.00% <0.00%> (ø)

zyte_common_items/components.py
0.00% <ø> (ø)

zyte_common_items/util.py
0.00% <ø> (ø)


Add cmake support for OCIO_LIBNAME_SUFFIX

This patch from Lee Kerley makes it easier for people making custom builds to add an optional suffix to the library name itself.
The purpose is to prevent a studio build for internal tools from conflicting with a different build of OCIO that might be embedded in a DCC app. The existing option to customize the inner namespace does a good job preventing symbol clashes at link time, but does not address the problem where both the DCC app and the plugin we write end up callled libOpenColorIO.so, so whichever is first in the LD_LIBRARY_PATH will win and the other won't be found.
This is a patch against RB-1.1, because that's what we need ASAP, but after this is merged, we will submit another one against master (the build system has changed enough it won't just cherry pick).
Signed-off-by: Lee Kerley<EMAIL_ADDRESS>Signed-off-by: Larry Gritz<EMAIL_ADDRESS>

The system won't let me review this, because I'm the one who submitted the PR. But actually, Lee wrote it, I just filed the PR to save him some time, so consider it reviewed and approved from me.
This is an important fix for us, so if at all possible, it would be really helpful to get the reviews and merge on this ASAP.


make_fake_fixtures.py does not create accurate foreign key mapping for member chapters.

Summary of the Bug
The fake fixtures script does not create valid data - the members objects have chapter IDs that do not correspond to the actual chapter IDs.
See [this comment[(https://github.com/usgo/agagd/pull/129#issuecomment-706909833) for more context.
Steps to Reproduce the Behaviour

docker-compose build
docker-compose up
docker-compose exec db /bin/bash
mysql -p
use agagd; select chapter_id from members limit 5; select member_id from chapters;

The Expected Behaviour
chapter_id in the members table should correspond to a member_id in the chapters table.

https://github.com/usgo/agagd/pull/132 fixed this.


[FEATURE] Change role of user from SDK

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
I have no way to change the role of an existing user without creating a new user.
Describe the solution you'd like
A clear and concise description of what you want to happen.
A method to change the role of an existing user from the SDK.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Usually the change is from a more restrictive to a less restrictive role e.g. annotator > admin or admin > owner.

@nataliaElv The use case is clear.
Server side @jfcalvo @frascuchon It seems like the user endpoints have not been carried over to v1 and most user operations are still in the v0 api. https://github.com/argilla-io/argilla-server/blob/main/src/argilla_server/apis/v1/handlers/users.py . We should start off by creating these server endpoints. I'm happy to take this on.
Client side: @frascuchon Once the the server operations exist then the SDK work is straightforward.


Implement Vector Class - void add(int index, Object element)

please assign it to me

This is same issue as #200
Please see to it @JyotiJauhari maam

This is same issue as #200
Please see to it @JyotiJauhari maam

I will close it, thanks for telling @plazzy99 .


[Request] Doctest highlighting for Python

Is your request related to a specific problem you're having?
I'm frustrated when syntax highlighers ignore doctests in Python docstrings. When working in gitlab its hard for people to review doctests I write because the syntax isn't highlighted. I understand that this library is what gitlab uses for highlighting, so implementing support here should solve that issue.
The solution you'd prefer / feature you'd like to see added...
I would like to request support for highlighting doctest syntax in docstrings.
This feature is being added to rouge-ruby: https://github.com/rouge-ruby/rouge/pull/1932  and has been supported by vim for years:

How would we know that is a docstring vs just a normal string?
We do currently support highlighting the >>> blocks within """ strings, as the meta scope (not as code itself) ... but we don't do anything to try and handle "output blocks" (green in your snap above I believe)...
I think we might be open to someone working on a PR to improve this, though we may have a small problem in that there is no way to specify "return to normal code formatting"... so once you're in a string you're stuck with the styling from that... there isn't any great way to interleave code into the string and reset the highlighting accordingly - you'll still be inheriting styling from the comment block.
We do have subst scope for inline code, but that's not guaranteed to match the default coloring (ie, normal code that's unhighlighted)... perhaps we need a new scope for "default"...?

Do you have an example from gitlab?

Here is an example of a doctest in a gitlab snippet: https://gitlab.com/-/snippets/2502446

How would we know that is a docstring vs just a normal string?

This would only happen in triple-quoted string. The way that vim handles it is that any tripple quoted string where a new line starts with a >>>   and then the syntax transitions into the "want" part of the docstest (where you put the expected output) when it finds a new line that doesn't start with >>>  or ... , and then it goes back to a normal string on the next empty newline.
More percicely this is how vim tags start and end tokens:

" Triple-quoted strings can contain doctests.
syn region  pythonString matchgroup=pythonQuotes
      \ start=+[uU]\=\z(['"]\)+ end="\z1" skip="\\\\\|\\\z1"
      \ contains=pythonEscape,@Spell
syn region  pythonString matchgroup=pythonTripleQuotes
      \ start=+[uU]\=\z('''\|"""\)+ end="\z1" keepend
      \ contains=pythonEscape,pythonSpaceError,pythonDoctest,@Spell
syn region  pythonRawString matchgroup=pythonQuotes
      \ start=+[uU]\=[rR]\z(['"]\)+ end="\z1" skip="\\\\\|\\\z1"
      \ contains=@Spell
syn region  pythonRawString matchgroup=pythonTripleQuotes
      \ start=+[uU]\=[rR]\z('''\|"""\)+ end="\z1" keepend
      \ contains=pythonSpaceError,pythonDoctest,@Spell

...

" Do not spell doctests inside strings.
" Notice that the end of a string, either ''', or """, will end the contained
" doctest too.  Thus, we do *not* need to have it as an end pattern.
if !exists("python_no_doctest_highlight")
  if !exists("python_no_doctest_code_highlight")
    syn region pythonDoctest
	  \ start="^\s*>>>\s" end="^\s*$"
	  \ contained contains=ALLBUT,pythonDoctest,pythonFunction,@Spell
    syn region pythonDoctestValue
	  \ start=+^\s*\%(>>>\s\|\.\.\.\s\|"""\|'''\)\@!\S\++ end="$"
	  \ contained
  else
    syn region pythonDoctest
	  \ start="^\s*>>>" end="^\s*$"
	  \ contained contains=@NoSpell
  endif
endif

And here is the patch that implemented it in rouge: https://github.com/rouge-ruby/rouge/pull/1932/files
The vim higlighter isn't perfect. It breaks on this string within a string test in xdoctest (although it's a lot better than nothing).

But it handles most cases fairly gracefully:

though we may have a small problem in that there is no way to specify "return to normal code formatting".

Is there not a way to say that a string can contain a special higlighted pattern, and have rules for when that pattern ends?

Here is the doc that describes how gitlab is using this for highlighting: https://docs.gitlab.com/ee/user/project/highlighting.html
I'm not very knowledgable in terms of CSS / JS, but thank you for considering this improvment. Better doctest support in highlighters in genral will I think help a fair number of people in the longer run.

I think the important part there might be "and the rouge gem". If they are using HLJS they are using a heavily modified version- I couldn't even recognize the markup, so I wouldn't make assumptions about their upgrade timing.


BUG: special.logsumexp: fix type promotion

Reference issue
https://github.com/google/jax/issues/23496
What does this implement/fix?
gh-21149 included an unintended change in return dtypes. This PR reverts to the original behaviour to avoid a breaking change.
Additional information
cc @dfm

@mdhaber the previous CI failures from array-api-strict disallowing cross-promotion of integral and floating dtypes probably explains why you wrote the promotion code the way you did initially. I've rewritten it now to consider floating and integer promotion separately, let me know if that looks alright.

Let's see if this passes.

I've given up on defining integer to float promotion rules for array_api_strict (the only library that doesn't define them). When we go back and use this utility in the stats functions we've already translated to array API, this might mean we need to modify tests (e.g. add . all over the place) and add some skips for array_api_strict. Another possibility is to try-except the result_type calculation and define the promotion if the library refuses to. Maybe we'll do that if it gets too anoying to make exceptions for array_api_strict, but let's avoid that if possible.

I disagree with this - I think we should respect that the mixed promotion is undefined in the standard, and define our own promotion. I'm happy for this to sit behind a try-except. I'm not very comfortable with adding skips for array-api-strict unless they are obviously temporary.

I think we should respect that the mixed promotion is undefined in the standard

If we respect that mixed promotion is undefined, shouldn't that mean that we don't define it?
But sure - let's add the try-except. I just didn't think that looked as nice but it would save us a lot of trouble. I'll push that.

Undefined by the standard => we need to cover all bases if we want to support it, IMO (or ask the standard to change). Perhaps we could have some caveat that “your library must support mixed promotion for this function to work”, but I think that’s more complicated that defining it ourselves.

On 10 Sep 2024, at 15:08, Matt Haberland @.***> wrote:
I think we should respect that the mixed promotion is undefined in the standard
If we respect that mixed promotion is undefined, shouldn't that mean that we don't define it?
But sure - let's add the try-except. I just didn't think that looked as nice but it would save us a lot of trouble. I'll push that.
—
Reply to this email directly, view it on GitHub https://github.com/scipy/scipy/pull/21528#issuecomment-2340930285, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMI2QFZWCWL3PH3ERRIEM43ZV34N5AVCNFSM6AAAAABN4BCHKWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBQHEZTAMRYGU.
You are receiving this because you authored the thread.

xp.asarray(1.).dtype, I think (although I don't have a good argument for it, I think it's fine to use the default float whenever floating data is needed and non-floating data is provided).

Works for me, and I don't think all CI needs to run again, but please check one last time locally. LMK if squash merge is OK.

all fine locally, thanks!

@dfm please let us know if you still see problems

Thank you!! Our nightly tests will run again tomorrow and I'll report back if there are any issues.


Create a public online service to quickly collect Data samples for OCR and setup automated release of trained models based on availability of new dataset

This task not much relevant now since public data like https://www.gundert-portal.de/?page=malayalam exists .
They have line-wise correct translated text and its respective scanned images