Modifying ICE Candidates in offer SDP?

Hi all,
I've been trying to find a way to modify the ICE candidates in the SDP.
The reason is that when we make an offer (200 OK) there are multiple ICE candidates being gathered, and most of them are not relevant at all as they're some virtual NICs on my PC. Most of the times, the non-relevant ICE candidates are getting higher priority and it is making a big delay in a call.
Question is if you know a way to change ICE candidates or making chosen one being prioritized (although I doubt it is in the SIP.JS level, right?).
What I tried to do is doing the following:
        var sdpModifier = function (description: RTCSessionDescriptionInit) {
            console.log(description);
            //description.sdp = description.sdp?.replace(....... making some replace REGEX here .....);
            console.log(description.sdp?.toString());
            return Promise.resolve(description);
        };

        var invitationAcceptOptions = {
            "sessionDescriptionHandlerModifiers": [sdpModifier]
        };

        invitation.accept(invitationAcceptOptions);

But when I see in the console.log I'm making in the sdpModifier function is that it has the whole SDP body, all except the ICE candidates themselves, so I can't really modify something here. So I assume that the ICE gathering process happens somewhere after this step. Can I modify it after candidates are gathered and put into the SDP? Or any other better solutions?
Thanks!

I'm experiencing the same issue, however, I think your solution isn't the most viable, I'm thinking that the problem here is that SIP.js doesn't support Trickle ICE. (or does it?)
Also, I only experience this in Firefox, on Chrome it is fast to establish the call. (might be a good idea to try Chrome on your side)

Oh and I tried setting iceCheckingTimeout on the SessionDescriptionHandlerOptions of UserAgentOption to 500 and didn't notice any difference at all.
I am using the version 0.15 of the Lib.

@egreenmachine Any updates on this?

Oh and I tried setting iceCheckingTimeout on the SessionDescriptionHandlerOptions of UserAgentOption to 500 and didn't notice any difference at all.
I am using the version 0.15 of the Lib.

Hi!
Regarding your previous reply, I tested it on Chrome & FireFox, and it behaves the same for me.
Regarding the "iceCheckingTimeout", for me I can tell for sure it will not help because the problem on my end is actually that the RTPEngine is the one that takes the list of the ICE candidates and try to do ICE checking for all candidates. The issue is that the offer (200 OK) or an initial INVITE from SIP.JS (although SIP.JS lib is not to blame here) is placing multiple ICE candidates with some strange priorities (at least for me) so RTPEngine makes the ICE checking according to this priority list, so when it tries to check the first one which is not accessible for him it delays the whole process, and that's why I tried to approach it with deleting irrelevant ICE candidates when there's "onicecandidate" event (while they're gathered).

@slavikbialik Yes, I understood your issue, I was just commenting mine because it had the same "outcome" (the delay). 😋

OK, I was able to find the proper way to code what I initially wanted in my post.
First needed a way to detect the creation of the session description handler and only then I could hook a delegation of the "onicecandidate" event. So something like that (only I didn't accomplished the removal of ice candidates that I don't need, but it is possible within this code as far as I recall):
        CallTaking.inviter.delegate = {
            onSessionDescriptionHandler: (
                sessionDescriptionHandler: SessionDescriptionHandler,
                provisional: boolean
            ) => {
                var sdh = (CallTaking.inviter.sessionDescriptionHandler as Web.SessionDescriptionHandler);
                sdh.peerConnectionDelegate = {
                    onicecandidate(event: RTCPeerConnectionIceEvent) {
                        console.log(event);
                    }
                };
                console.log("A session description handler was created");
            }
        };

Oh and I tried setting iceCheckingTimeout on the SessionDescriptionHandlerOptions of UserAgentOption to 500 and didn't notice any difference at all.
I am using the version 0.15 of t

Can you please assist me with the code you implemented to change iceCheckingTimeout?
I'm trying to understand how to code it to change this timer, and I'm not able to get a hold of it.
Although I'm using SIP.JS 0.17, so it might be a little different (hope not).

@slavikbialik Try setting it in the sessionDescriptionHandlerOptions of your SessionInviteOptions.

@slavikbialik Try setting it in the sessionDescriptionHandlerOptions of your SessionInviteOptions.

Thanks! I just found out now that iceCheckingTimeout is not part of the SIP.JS 0.17, cannot find it in the code and in non of the documents for that version. No idea how I can reduce the time for building up the INVITE, as it takes me around 5 seconds until the INVITE is being ready and sent out. Opened a new issue on that.

Weird, because I found it on the docs here in this GitHub.


[doc] add back beta notices for 6.8

This PR add back the Beta notices for 6.8 branch. Only releases >= 7.7.0 are GA and eligible to support.

errors not related to this PR => merging


"ValueError: DataFrame.dtypes for data must be int, float or bool. Did not expect the data types in fields" even for the columns with type as

Dataframe has below columns :
Age                              int64
Worked_in_field           int64
Major                           int64
GPA                              float64
Projects                        int64
Experience                    int64
Bootcamp                     int64
Ethnicity                        int8
BusinessTravel               int8
DailyRate                      int64
Department                  int8
DistanceFromHome      int64
Education                     int64
EducationField              int8
EnvironmentSatisfaction       int64
Gender                         int8
HourlyRate                   int64
JobInvolvement           int64
JobLevel                       int64
JobRole                        int8
JobSatisfaction            int64
Attrition                       int64
Trained the model to predict Attrition and using DiCE for counterfactuals
d = dice_ml.Data(dataframe=df, outcome_name='Backed_out_before_Round2',continuous_features=['Age', 'Worked_in_field', 'Major', 'GPA', 'Projects', 'Experience'])
m = dice_ml.Model(model=xgb, backend="sklearn")
exp = dice_ml.Dice(d, m, method="random")
query=df[1:2].drop(['Attrition'],axis=1)
e = exp.generate_counterfactuals(query, total_CFs=1, desired_class="opposite",random_seed=0)
e.visualize_as_dataframe(show_only_changes=True)
0%|          | 0/1 [00:00<?, ?it/s]
ValueError                                Traceback (most recent call last)
Cell In[162], line 1
.......
183     msg = """DataFrame.dtypes for data must be int, float or bool.
184             Did not expect the data types in fields """
--> 185     raise ValueError(msg + ', '.join(bad_fields))
187 if feature_names is None and meta is None:
188     if isinstance(data.columns, MultiIndex):
ValueError: DataFrame.dtypes for data must be int, float or bool.
Did not expect the data types in fields Bootcamp, Ethnicity, BusinessTravel, DailyRate, Department, DistanceFromHome, Education, EducationField, EnvironmentSatisfaction, Gender, HourlyRate, JobInvolvement, JobLevel, JobRole, JobSatisfaction
Issue : Even though columns are of datatype int64, int8 the error is somewhat not clear.

@aakash086 could you send the entire stack trace? If you could share a sample notebook with dataset that will help a lot.


Idea: multiple templates

I've thought about how markmeld could be used to produce a bunch of PDFs for a mail merge or something. In the past I've done it just by having a separate target for each letter, but then I have to build each separately, or put them all into a pre-build for a mega-target or something.
It would be more convenient if one target could produce multiple outputs, particularly if those outputs are highly similar. Two use cases for this type of function would be 1) mail merge; 2) static site generator. What if targets could have multiple templates, with multiple output files, and the command just ran for each of them?
Case 1: mail merge
So maybe something like:
targets:
  build_letters:
    output_file: '{person}.pdf'
    outputs:
      - data_variables:
        person: person1

Here we don't need to specify an output file for each person, but each person does get an individual output file, it's just populated by the person variable value.
Case 2: static site generator
You could basically use this to implement a static site generator...
targets:
  build_letters:
    outputs:
      - index.html:
          md_template: index.jinja
          output_file: "index.html"
      - people.html:
          md_template: people.jinja
          output_file: "people.html"

For the mail merge, the md_template doesn't need to be included because it's the same for each output. For the static site generator case, different templates and output files are included in each element of the outputs attribute. Maybe the output file could be kept as {output_file} and then you'd use a data variable to specify it, or something.
Case 3: only read large data in once for web updates
This could also solve the problem of updating multiple literature threads, while reading in large data only once. Because it's just 1 target, it only has to load the data a single time. I guess this is basically the 'static site generator' use case, just a specific example where it makes sense.
Implementation
All I need to add is a loop that goes through each element in output, updates the main structured data object, and then runs the command. The generic case would be unchanged, really... if no outputs are specified, then a single output is expected with the main data object, no 'outputs update' required.
I think it could use the same functionality as the way a 'target' data package updates a 'general' data package... we'd just copy the 'target' data package and then loop through the build, updating it with each 'output' data package.

What does this look like for a static site generator?
loop:
  loop_data: pages
  assign_to: page

pages:
  - about:
    output_file: _site/about.html
  - home:
    ...

But this requires enumerating every page.


Branching strategy

Here are some examples:

https://github.com/rancher/helm3-charts/branches:

master
dev

https://github.com/rancher/system-charts/branches:

release-v2.6
dev-2.6

We can keep master as the release branch, and then add a new dev branch. That way, I can configure Rancher with the dev branch to test any changes, without impacting official releases.


OCPBUGS-32547: Fix delete generation for mirror to mirror flow

Description
This a fix addresses the bug raised for delete images generation for the mirror to mirror flow
Fixes # OCPBUGS-32547
Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Executed the following isc
kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v2alpha1
mirror:
  platform:
    channels:
    - name: stable-4.14
      minVersion: 4.14.22
      maxVersion: 4.14.22
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages:
      - name: aws-load-balancer-operator
    - catalog: oci:///home/lzuccarelli/go/src/github.com/openshift/oc-mirror/rhopi4-14
      targetTag: "v14"
      targetCatalog: "lmz/catalog"
      packages:
      - name: windows-machine-config-operator
    - catalog: oci:///home/lzuccarelli/go/src/github.com/openshift/oc-mirror/rhopi4-14
      packages:
      - name: cincinnati-operator

Then ensured a mirror to mirror workflow using this command
bin/oc-mirror --config ocpbugs-32547.yaml --workspace file://ocpbugs-32547 --v2 docker://localhost:5000/ocpbugs-32547 --dest-tls-verify=false

Created a delete-isc.yaml from the isc as described above
Once completed I executed the following to generate the delete files.
bin/oc-mirror delete --config delete-32547.yaml --workspace file://ocpbugs-32547 --v2 --generate docker://localhost:5000/ocpbugs-32547 --dest-tls-verify=false

And finally actually doing a delete of the remote registry
bin/oc-mirror delete --config delete-32547.yaml --delete-yaml-file ocpbugs-32547/working-dir/delete/delete-images.yaml --v2 docker://localhost:5000/ocpbugs-32547 --dest-tls-verify=false --force-cache-delete

Expected Outcome

All indicated steps should not fail
All unit and e2e should pass

/jira refresh

/test images

/test images

/retest

/label qe-approved

/lgtm


FIX: hostmetrics didnt include anything except filesystem metrics

Without setting these to null, it disable them completely, so no hostmetrics are included except filesystem metrics

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Rcplx is broken on R-devel

Originally filed at https://github.com/extendr/libR-sys/issues/149.
R-devel changed the definition of Rcomplex a few days ago (https://github.com/r-devel/r-svn/commit/862f9f816ff3ff3cb3f851603f19e99f60a98475) in order to match the C representation of complex with Fortran's representation (see https://bugs.r-project.org/show_bug.cgi?id=18430 for the details).
In short, Rcomplex is now a union, while the current exendr code assumes it's a struct. So, now extendr won't compile with the following error. We have to deal with this. I hope I can have time to fix this soon, but I'd appreciate if anyone can take a look because I've never used complex numbers...
error[E0560]: union `libR_sys::Rcomplex` has no field named `r`
  --> extendr-api\src\robj\into_robj.rs:93:20
   |
93 |         Rcomplex { r: 0., i: 0. }
   |                    ^ `libR_sys::Rcomplex` does not have this field
   |
   = note: available fields are: `__bindgen_anon_1`, `private_data_c`

error[E0560]: union `libR_sys::Rcomplex` has no field named `i`
  --> extendr-api\src\robj\into_robj.rs:93:27
   |
93 |         Rcomplex { r: 0., i: 0. }
   |                           ^ `libR_sys::Rcomplex` does not have this field
   |
   = note: available fields are: `__bindgen_anon_1`, `private_data_c`

error[E0784]: union expressions should have exactly one field
  --> extendr-api\src\robj\into_robj.rs:93:9
   |
93 |         Rcomplex { r: 0., i: 0. }
   |         ^^^^^^^^

error[E0609]: no field `r` on type `libR_sys::Rcomplex`
   --> extendr-api\src\robj\mod.rs:348:56
    |
348 |                     CPLXSXP => R_IsNA((*COMPLEX(sexp)).r) != 0,
    |                                                        ^ unknown field
    |
    = note: available fields are: `__bindgen_anon_1`, `private_data_c`
help: one of the expressions' fields has a field of the same name
    |
348 |                     CPLXSXP => R_IsNA((*COMPLEX(sexp)).__bindgen_anon_1.r) != 0,
    |                                                        +++++++++++++++++

error[E0609]: no field `r` on type `libR_sys::Rcomplex`
   --> extendr-api\src\scalar\rcplx_default.rs:105:28
    |
105 |         Rcplx(c64::new(val.r, val.i))
    |                            ^ unknown field
    |
    = note: available fields are: `__bindgen_anon_1`, `private_data_c`
help: one of the expressions' fields has a field of the same name
    |
105 |         Rcplx(c64::new(val.__bindgen_anon_1.r, val.i))
    |                            +++++++++++++++++

error[E0609]: no field `i` on type `libR_sys::Rcomplex`
   --> extendr-api\src\scalar\rcplx_default.rs:105:35
    |
105 |         Rcplx(c64::new(val.r, val.i))
    |                                   ^ unknown field
    |
    = note: available fields are: `__bindgen_anon_1`, `private_data_c`
help: one of the expressions' fields has a field of the same name
    |
105 |         Rcplx(c64::new(val.r, val.__bindgen_anon_1.i))
    |                                   +++++++++++++++++

Some errors have detailed explanations: E0560, E0609, E0784.
For more information about an error, try `rustc --explain E0560`.
error: could not compile `extendr-api` due to 6 previous errors
warning: build failed, waiting for other jobs to finish...
error: could not compile `extendr-api` due to 6 previous errors

I 'like' how they introduce a breaking change in a minor version.

Yeah, was going to say.  Has anyone asked on the mailing list about this? Also isn't it a bit weird that the struct definition is part of the public interface when most of the other R structs are hidden?

According to this screenshot from the bug discussion, they have now to union definitions inside Rcomplex. Technically it changes nothing, but (even rustc notices that) we need to reference not Rcomplex.r but Rcomplex.__bindgen_anon_1.r.

Also isn't it a bit weird that the struct definition is part of the public interface when most of the other R structs are hidden?

Well, R works over primitives, and the complex number is the only type that is not 'default' and its implementation can be opinionated. You cannot have many opinions on how to store an i32, but two f64 -- a different thing. Including alignment issues. Hence they expose it.

I think the R core just doesn't consider this as a breaking change, considering there's no mention in the NEWS. But, yaah, I was wondering how this can be a non-breaking change to C/C++ users... Could any of you ask on the mailing list? I think I need more time to figure out what's going on.
Note that, as you can see the source code, the easiest fix for now would be to define R_LEGACY_RCOMPLEX to stick with the old definition, though I'm not sure how long this will be allowed.

Well, R works over primitives, and the complex number is the only type that is not 'default' and its implementation can be opinionated. You cannot have many opinions on how to store an i32, but two f64 -- a different thing. Including alignment issues. Hence they expose it.

But why not treat it like SEXP where all external interfaces just treat it as a pointer, and can't do anything with it other than using public function interfaces. Then you don't need to know the layout of the struct.

According to this screenshot from the bug discussion, they have now two union definitions inside Rcomplex. Technically it changes nothing, but (even rustc notices that) we need to reference not Rcomplex.r but Rcomplex.__bindgen_anon_1.r.
Do you advise doing this, or is there a better way? I'm a bit out of my depth

I think the R core just doesn't consider this as a breaking change, considering there's no mention in the NEWS. But, yaah, I was wondering how this can be a non-breaking change to C/C++ users... Could any of you ask on the mailing list? I think I need more time to figure out what's going on.

I'm happy to ask, but I'm not sure I have the vocabulary to explain it. Do I just say "the definition of Rcomplex has changed from a struct to a union, breaking our generated bindings"?

But why not treat it like SEXP where all external interfaces just treat it as a pointer, and can't do anything with it other than using public function interfaces. Then you don't need to know the layout of the struct.

SEXP is a pointer, but Rcomplex is a struct, not a pointer to a struct stored somewhere (like character in R). When you pass it around, you still need to communicate to the CPU you are moving 128 bits of data, together.

Note that, as you can see the source code, the easiest fix for now would be to define R_LEGACY_RCOMPLEX to stick with the old definition, though I'm not sure how long this will be allowed.

Actually, it feels like we need a set of features that describe what R version is used. E.g. if it is R >= 4.3. Then we could provide different implementations based on the breaking changes introduced.

Actually, it feels like we need a set of features that describe what R version is used. E.g. if it is R >= 4.3. Then we could provide different implementations based on the breaking changes introduced.

Don't we have config parameters that are set already based on the R version? Surely we can use those to handle this case.
Also, if we define R_LEGACY_RCOMPLEX but we try to link to an R interpreter using the new unions, won't everything explode?

Actually, it feels like we need a set of features that describe what R version is used. E.g. if it is R >= 4.3.

Yes, that would be the proper fix. We can define a new cfg for R 4.3 here.
https://github.com/extendr/extendr/blob/master/extendr-api/build.rs

Also, do you happen to know if this change is going to land in R 4.3.0, or is it in the version after that? Not sure if they have branches etc in the dev R.

It is possible in the libR package to define what version of r is being picked up. Those can be then picked up by extendr build.rs. But as far as I know, it is not possible to affect the features enabled 😕 from a build.rs.
Of course we can ask chat gpt and see if it knows a solution.

It is possible in the libR package to define what version of r is being picked up.

As you can see in the build.rs, it's defined as envvars.

I wasn't asking a question. As far as I know, you cannot set a feature cfg by an envvar.
I might be wrong though.

I believe Yutani has linked above exactly the location where we already set cfg using an envvar. It seems to be using this logic: https://doc.rust-lang.org/cargo/reference/build-scripts.html#outputs-of-the-build-script

That's amazing. Thank you for teaching me thing. I truly never knew about this..

Yes, I meant it. Sorry, my explanation was a bit too terse.

Can anyone explain why R >= 4.1 is use_r_ge_version_14? Where does 14 come from?
https://github.com/extendr/extendr/blob/6e15390d2d4dbcd53878f4b31312c4fb68de25e6/extendr-api/build.rs#L21

It's the version of the Graphics Device API.
https://github.com/extendr/libR-sys/blob/d113d56dd0c362ddf002e489b456face8c4c348d/bindings/bindings-linux-x86_64-R4.1.rs#L81

Here's the mailing list thread. Nothing interesting yet: https://stat.ethz.ch/pipermail/r-devel/2023-April/082517.html.
I've just had the realisation that I'm probably embarrassing myself though, because the C compiler might actually insulate the user from this particular layout change. It seems likely that only Rust would be affected because it forces you to handle the union in a way that C probably doesn't, because it doesn't attempt to be safe in the same way.

Yep, that seems to be the case!

So what I'm thinking is this:

Firstly, we wait until this patch actually hits a stable branch (?). The patch author pointed out to me that r-devel is unstable and that we shouldn't worry too much about it
Secondly, once it does hit stable, we implement the cfg stuff discussed above
Using this cfg flag, if we're on R > 4.4, we access the struct as cplx.__bindgen_anon_1.i. Otherwise we access fields as cplx.i. I will ask if they can rename the anonymous struct to make this less ugly
Lastly, if and when https://github.com/rust-lang/rust/issues/49804 is implemented in stable Rust, we simplify our implementation using the changes

As @CGMossa has pointed out, this has now hit the NEWS, meaning that it will be in R 4.3.0, so we're on to step 2.

I have a proposal -- currently, we use use_r_ge_version_14, which is tied to graphics API version. We should have a separate cfg for R versions. An example could be r_ge_4_4 or r_4_4_or_greater. Ideally, we should rename graphics cfg to something like r_graphics_ge_14, but I am not sure if this is going to be painful for consumers.

Yep, it looks like the cfg mechanism basically treats everything as a boolean, so it makes sense to have GT/LT config options since we can't do the numerical comparison within the Rust code. I agree with a ge -> graphics change.

An example could be r_ge_4_4 or r_4_4_or_greater

The intention that I didn't chose the name like r_4_1 was for not losing the context why we had to distinguish the version, but feel free to rename them.

I am not sure if this is going to be painful for consumers.

These cfg are for internal logics, so I don't think this affects the users of extendr.

@multimeric, I believe here ge means greater than or equal to, and it is important to keep that this feature is enabled for graphics version 14 and higher.
@yutannihilation, I did not suggest to remove it completely, only that if we have two cfgs both saying it is R version, it can be misleading. But separating it into graphics (what is used now) and an overall R version seems reasonable.
My only concern is that if somewhere someone relied on purely libR-sys and graphics cfg, their code will break if we change this. But on the other hand, we are in the pre-release phase, so we can occasionally break things.

I now think the proper fix to this is to overwrite the definition in libR-sys. Although they have different names, C's Complex is also translated as a struct of two f64 fields. If I understand correctly, it's impossible to distinguish them over FFI anyway, because it's passed via a pointer without type information (if it were a tagged union, we could?). All we can care is their memory layouts, and it should be okay as long as their memory layouts agree, so I think this union is unnecessary.
#[repr(C)]
#[derive(Copy, Clone)]
pub union Rcomplex {
    pub __bindgen_anon_1: Rcomplex__bindgen_ty_1,
    pub private_data_c: __BindgenComplex<f64>,
}

#[repr(C)]
#[derive(Debug, Copy, Clone)]
pub struct Rcomplex__bindgen_ty_1 {
    pub r: f64,
    pub i: f64,
}

#[derive(PartialEq, Copy, Clone, Hash, Debug, Default)]
#[repr(C)]
pub struct __BindgenComplex<T> {
    pub re: T,
    pub im: T,
}

So both variants of the union are a struct with two fields that are f64? In that case I'm happy to just override bindgen here. It would make everything a lot easier for sure.

To be quite honest I don't understand the point of the complex change in the first place. It's something to do with aliasing, and forcing the C compiler to handle something about the Fortran complex data type but it's weird to me that it doesn't actually change anything about the layout. This is what Tomas said on the mailing list:

Not necessarily only gfortran, the problem arises anytime when an
Rcomplex value is (without explicit conversion) treated as C99 _Complex
double. The layout is (under some assumptions) the same, but not the
aliasing, so the information about that it can be treated as either
needs to be carried over. You would run into a similar problem when
calling say a library function expecting a C99 _Complex double, one
could argue it is not correct and one should do an explicit conversion,
but the question is how much code does it correctly. Then in theory
there could be LTO across C/Fortran/Rust and I can't rule out you
couldn't get to a similar situation (e.g. Rust calling Fortran). Ideally
really Rust should be able to translate this valid C11 code correctly.

In case clarification is needed, the Fortran compiler will not see the
definition of Rcomplex at all, it will use complex numbers as defined in
Fortran, and in case of gfortran this is C99 _Complex at least on the
interface to C. The definition is for the C compiler, which compiles
code using ".r" and ".i" to access real and imaginary parts of complex
numbers. When compiling this code, the C compiler will be made aware
that values set via ".r" and ".i" may also be accessed as C99 _Complex
double objects. It will also store this information in object files, in
the intermediate representation used during link-time optimizations. The
link-time optimizer will then know that e.g. when passing Rcomplex from
(originally) C to COMPLEX*16 in (originally) Fortran, values set on the
C side via ".r" and ".i" are visible on the Fortran side in a
corresponding array of COMPLEX. Without knowing that, the link-time
optimizer could remove say stores to the ".r" and ".i" when calling to
Fortran, this is what has been actually seen to happen.

Sorry, I forgot to say thank you for asking it on the mailing list. Yeah, I too don't understand it completely, but... I was thinking if it's really possible for Rust to handle a complex number over FFI at all after I found this description.

Note that Complex<F> where F is a floating point type is only memory layout compatible with C’s complex types, not necessarily calling convention compatible. This means that for FFI you can only pass Complex<F> behind a pointer, not as a value.
https://docs.rs/num-complex/latest/num_complex/struct.Complex.html#representation-and-foreign-function-interface-compatibility

Though I don't understand what this exactly means either, I guess that "Rust calling Fortran" is the case; they probably pass values, considering they share the complex definition. If they are passed as a value, it might get optimized wrongly by the compiler. But, if they are passed as a pointer, it will be explicitly cast on Rust's side, so it won't be a problem.

Can we override the name and visibility of the generated Rcomplex? I think what we can do is just override its definition and then have some build-time test (like we already have in libR-sys) to verify that whatever gets bindgened from headers has an identical layout to our custom Rcomplex(f64, f64).

@Ilia-Kosenkov I found out that there is a override mechanism in bindgen, it unfortunately only works for variables and functions.
Further this is going to be a little more painful. The type Rcomplex is used everywhere in the bindings. We cannot simply rename it to say _Rcomplex, because that would be the type we'd then have to pass around everywhere where Rcomplex is used (considering only the bindings here), ~ 138 references.

Can we define an impl for it? We can leave it as is but then provide a convenient API to work with, like accessing re() and im(), for instance. This will be a breaking change for extendr, but libR-sys will have the 'proper' implementation.

I'm attempting this over at https://github.com/extendr/libR-sys/pull/156
Feel free to provide comments and tasks on that PR continuously, as I've no idea when it is sufficiently done.

Fixed by https://github.com/extendr/libR-sys/pull/156


Fix typo

Fixes #
Proposed Changes

Thanks for the help @preetikr, when should I expect to see the connector published in the console?


Remove deprecated createGooglePhotorealistic3DTileset key option in 1.126

In https://github.com/CesiumGS/cesium/pull/12299, createGooglePhotorealistic3DTileset(key) was deprecated in favor of an options object: createGooglePhotorealistic3DTileset({key}). It should be removed for the 1.126 release.

Hi @ggetz,
I would like to contribute to resolving the issue #12328 regarding the removal of the deprecated createGooglePhotorealistic3DTileset(key) function. Could you please assign this issue to me?
Thank you!


created initial about.md file for feature extraction pipeline

Added Use cases- Sentence classification & Inference
Will adding more as I research on the topic. Thanks

partially fixes #355

Hi @merveenoyan , I added a code snippet to train a Random Forest on embeddings. Just a little confused if I should place it in Task variants or Use cases. Also, I'm working on improving the technical writing you mentioned. Thanks

@shloksah do you need help on this? 🙂

@shloksah do you need help on this? 🙂

Hi @merveenoyan , I'm working on the suggestions sorry for a bit delay - I have my end semester exams this week. Thanks 😄…

@shloksah no worries, best of luck!!! (I'm also a master's student 😂)

@shloksah we deployed feature extraction here so there's a merge conflict, would you like to make additions on top of it instead? https://huggingface.co/tasks/feature-extraction

Hey @shloksah 👋
We have moved the codes for Tasks project to huggingface.js repository so if you feel like continuing this pull request you can move your work over there. Many thanks! 🤗


Bumpversion to 1.3.3

Codecov Report

Merging #388 (757feea) into main (6a26ebc) will decrease coverage by 0.10%.
Report is 1 commits behind head on main.
The diff coverage is 100.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@            Coverage Diff             @@
##             main     #388      +/-   ##
==========================================
- Coverage   72.91%   72.82%   -0.10%
==========================================
  Files         184      184
  Lines        8864     8864
  Branches     1447     1446       -1
==========================================
- Hits         6463     6455       -8
- Misses       2213     2223      +10
+ Partials      188      186       -2

Files
Coverage Δ

orchestrator/__init__.py
100.00% <100.00%> (ø)

... and 6 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Zeppelin no longer reads example workbook into application on load

I think it might have been a change to the directory structure for Zeppelin 9 that for whatever reason just got picked up on release 2.5.2 of bigdata-docker-compose, but there was a significant change to directory structure that Zeppelin reads from disk. However, you can still use the json file in the repository to import the notebook into the application instead of getting it on load of the application.
Would probably be a good idea to save the new directory structure into the bigdata-docker-compose repo for clarity eventually.

Looks like the introduced the concept of a .zpln file, which looks like just a json file for zeppelin notebooks.

I was wondering if it's just me or the new version of Zeppelin actually doesn't even migrate the old notebook format to the new one automatically... Turns out it's not just me! It was a no-brainer - I just renamed the old .json notebook to .zpln and it all worked.
Meh. I really wish Zeppelin developers would do that.

Fix pushed - feel free to close if it works for you.


handle missing controller gracefully

ATM if the admin forgot to activate the controller service or point to an external instance with CONTROLLER_HOST / CONTROLLER_PORT_SSH, for both entry points (for_production/presentation.sh) we have a very unfortunate behaviour:

Manager logs show the workflow is started, nothing more (no error that Controller cannot be reached)
Monitor shows the job as terminated (because no connection can be established to Controller at all)
script does not terminate

Manager logs show the workflow is started, nothing more (no error that Controller cannot be reached)

That's actually not true: the Manager logs will "unwind" from the correct error message
ocrd-manager process_images.sh: ssh: Could not resolve hostname ocrd-controller: Temporary failure in name resolution#015

That log is directly accessible as ocrd.log in the inactive job list.

script does not terminate

Also not true.
@SvenMarcus is there something I am missing? (I dimly recall you found the original problem...)


ts.transpile() doesn't seem to do type check

TypeScript Version:
1.8.5
Code
function FeedPig(what:number)
{
	print("the pig ate " + what + " pigs at 8:12");
}

FeedPig([1,2,3]);

Compiler Invocation
Using typescriptServices.js:
ts.transpile(source, null, filename)
Expected behavior:
Type error
Actual behavior:
Code compiles successfully and produces the output:
the pig ate 1,2,3 pigs at 8:12

That is correct. The transpile AP, is just for transpilation, I.e. generating JavaScript from typescript code. This only works on a single file, and does not load any files other than the ones needed.
Type checking is done as part of a full program compilation. You can use the ts.createProgram API for that.

@mhegazy This means what you said at this comment is already happening:

ts.transpile is the correct entry point. we are looking of slimming it down. for the short term it still needs the checker code, but in the future it should be possible to do it with just the parser and the emitter, which is much smaller.

In other words: is .transpile() now the smaller version or still the same?

it still needs the code in the checker. no errors are reported, no files are loaded. the comment was mainly about the internals of the API call, and is not related to this question.


A11y_Azure Data Studio extension 'Azure SQL Migration'_Database backup_ScreenReader: Narrator is not announcing column header name while navigating using ctrl+alt+arrow keys in "target database" grid

"Check out Accessibility Insights! - Identify accessibility bugs before check-in and make bug fixing faster and easier.”
Github Tags:
#A11yMAS; #A11yTCS; #A11ySev2; #Benchmark; #SH-AzureSQLMigration-Win32-Jun2022; #DesktopApp; #Win32; #SQL Migration Control Plane; #WCAG1.3.1; #ScreenReader; #Narrator;
Environment and OS details:
Azure Data Studio Version 1.37.0(User Setup)
Windows Version 21H2 (OS Build 22000.675)
Note:

This issue is also observed for Databases grid in Databases for assessment step
This issue is also observed for Authentication keys grid present in "Azure Database Migration service" step
This issue is not observed with NVDA

Repro Steps:

Turn on Narrator
Install Azure data studio app and install Azure SQL migration extension
Connect to SQL server, tab till respective server, right click and select manage
Tab till Azure SQL Migration in sitemap and press enter key
Tab till migrate to azure sql and press enter
Navigate to databases grid in "Databases for assessment" step
Select few databases and navigate to Database backup step by filling all mandatory fields in all steps
Navigate to target database grid and verify whether Narrator is not announcing column header name while navigating using ctrl+alt+arrow keys or not

Actual Result:
Narrator is not announcing column header name while navigating using ctrl+alt+arrow keys in "target database" grid
Expected Result:
Narrator should announce column header name while navigating using ctrl+alt+arrow keys in "target database" grid
Attachment:

This is a narrator issue and we do not optimize for specific screen-readers. Please close this bug

Closing this issue as it is a narrator specific bug.

@aasimkhan30
Is it external to narrator could please provide a sample app we will be routing this bug to Narrator Team

@aasimkhan30 As mentioned above bug is external to narrator Could you please provide us the stand alone sample app to route the bug to external team. thanks!

@aasimkhan30
Is it external to narrator could please provide a sample app we will be routing this bug to Narrator Team?

#Closed;
Since issue not observed with NVDA we can close this bug since Azure Data studio is an electron app.


Reexport kira

Currently, to create a StaticSoundData for AudioSource, you have to import kira with the correct version. I'd like... to not do that lol.
Also, in the case that #63 would be implemented, it is likely that users would have to reimport kira just to implement their own SoundData.
A solution to this is to reexport kira itself.

Not sure about reexporting Kira here. I would prefer to keep the API backend agnostic. For #63 there will likely be a new trait from bevy_kira_audio. We could do the same for StaticSoundData.
Would that be fine?

I think reexporting StaticSoundData, SoundData, and Sound would be fine. However, when a new type from kira is used in the public api, then it should also be exported.
Is there a way for Github Actions to know that a foreign item is used, but not reexported?


.Net: Example58_ConfigureRequestSettings.cs - rename to execution settings

We now use the term "prompt execution settings" (or just "execution settings") instead of "request settings". This sample should be renamed and comments should be updated accordingly.

Done


bug report，Cluster alias update failed after master failover

If this is a bug report, please provide a test case and the error output. Useful information:

your orchestrator.conf.json config file/contents
1、Except for the account configuration, other parameters remain unchanged. （raft: false）

"HostnameResolveMethod": "none",
"MySQLHostnameResolveMethod": "@@report_host",
"RecoverMasterClusterFilters": [
   "10.*"
 ]

your topology (e.g. run orchestrator-client -c topology -alias my-cluster)

<IP_ADDRESS>:61106   [0s,ok,8.0.20-11,rw,ROW,>>,GTID]
+ <IP_ADDRESS>:61106 [0s,ok,8.0.20-11,rw,ROW,>>,GTID]
+ <IP_ADDRESS>:61106 [0s,ok,8.0.20-11,rw,ROW,>>,GTID]

what did you do?
1、First, i set cluster alias（cluster_name: <IP_ADDRESS>:61106, alias: eeo) by web api.
2、After auto master failover, Cluster architecture changed to

frenky@frenkydeMacBook-Pro bin % ./orchestrator-client -c topology -alias <IP_ADDRESS>:61106
<IP_ADDRESS>:61106   [0s,ok,8.0.20-11,rw,ROW,>>,GTID]
+ <IP_ADDRESS>:61106 [0s,ok,8.0.20-11,rw,ROW,>>,GTID]

>  new cluster:（cluster_name: <IP_ADDRESS>:61106, alias: <IP_ADDRESS>:61106），alias is cluster_name.

frenky@frenkydeMacBook-Pro bin % ./orchestrator-client -c topology -alias eeo
<IP_ADDRESS>:61106 [unknown,invalid,8.0.20-11,rw,ROW,>>,GTID,downtimed]

>  failed cluster:（cluster_name: <IP_ADDRESS>:61106, alias: eeo），alias is eeo.

what did you expect to happen?
1、The cluster structure I would like is shown below

>  new cluster:（cluster_name: <IP_ADDRESS>:61106, alias: eeo），alias is eeo.
>
>  failed cluster:（cluster_name: <IP_ADDRESS>:61106, alias: <IP_ADDRESS>:61106），alias is <IP_ADDRESS>:61106.

what happened?
1、Since I have aliased the cluster, the code follows the following logic。

if alias := analysisEntry.ClusterDetails.ClusterAlias; alias != "" {
       inst.SetClusterAlias(promotedReplica.Key.StringCode(), alias)
}

2、When the above function is executed, the new cluster name should be eeo。
new cluster:（cluster_name: <IP_ADDRESS>:61106, alias: eeo）

3、During master failover,  because the 'go inst.UpdateClusterAliases()'  executes this SQL every 5 seconds
replace into cluster_alias (alias, cluster_name, last_registered)
select
	 cluster_name as alias, cluster_name, now()
from  database_instance
group by  cluster_name
having
sum(suggested_cluster_alias = '') = count(*)
`)

value: ('<IP_ADDRESS>:61106', '<IP_ADDRESS>:61106', now())

4、Cluster alias was replaced because of unique constraint on the table. The end result was not what we expected.
（cluster_name: <IP_ADDRESS>:61106, alias: eeo）   was replaced to （cluster_name: <IP_ADDRESS>:61106, alias: <IP_ADDRESS>:61106)

CREATE TABLE `cluster_alias` (
  `cluster_name` varchar(128) CHARACTER SET ascii COLLATE ascii_general_ci NOT NULL,
  `alias` varchar(128) NOT NULL,
  `last_registered` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`cluster_name`),
  UNIQUE KEY `alias_uidx` (`alias`),
  KEY `last_registered_idx` (`last_registered`)
) ENGINE=InnoDB DEFAULT CHARSET=ascii

How to fix ？

topology_recover.go

if alias := analysisEntry.ClusterDetails.ClusterAlias; alias != "" {
       + inst.SetClusterNameByAliasOverride(after, alias)
       inst.SetClusterAlias(promotedReplica.Key.StringCode(), alias)
}

cluster_alias.go

func SetClusterNameByAliasOverride(newClusterName string, alias string) error {
	return updateClusterNameByAliasOverride(newClusterName, alias)
}

cluster_alias_dao.go

func updateClusterNameByAliasOverride(newClusterName string, alias string) error {
	writeFunc := func() error {
		_, err := db.ExecOrchestrator(`
			update cluster_alias_override set cluster_name = ? where alias=?
			`,
			newClusterName, alias)
		return log.Errore(err)
	}
	return ExecDBWriteFunc(writeFunc)
}

Unique constraints are recommended for alias field of cluster_alias_override

I'm facing exactly the same issue. If the cluster alias would change, it's inconvenient to locate a certain cluster as there is not an identifier

I'm facing exactly the same issue. If the cluster alias would change, it's inconvenient to locate a certain cluster as there is not an identifier

I'm facing exactly the same issue. If the cluster alias would change, it's inconvenient to locate a certain cluster as there is not an identifier

The above repair measures have been applied to our production environment and operate normally. In abnormal or normal master switching scenarios, you can test it to see whether it meets your requirements


Add S3 as destination with datasets registration to glue catalog/snowflake external tables

What
Nowadays warehouses like Bigquery/Redshift/Snowflake are not the only used destinations.
Often companies used S3 (or objects storage as destinations).
It will be great to support S3 as destination - JSON format is okish (ideally parquet is supported).
When adding data to s3 destinations it will be great to register the dataset to Glue catalog , to achieve that conside the usage of:

create_json_table
create_parquet_table

Something similar can be done for snowflake - after ingesting the data in s3 the dataset can be registered as external table in snowflake.
Notes
It's generally a good idea to use S3 as destination also when using Snowflake, because pure S3 storage is cheaper than snowflake storage - in this case it's possible to use snowflake eternal tables in combo with dbt to pick only what is needed and stage transformed data - ideally this is done incrementally.

This is clearly useful!

we are planning to add s3 (and other bucket storage) support still in March. this will be soon ticketized here
registration in catalogue services like AWS Glue... sounds like a powerful tool and good idea! all depends when we can find more contributors :)

btw. (2) is on our radar for some time. you could build such pipeline with dlt + duckdb right now. we were experimenting with something like this:
json -> dlt -> duckdb -> dbt-duckb
dbt-duckdb can materialize to parquet on s3 bucket and register/update in AWS Glue
@nicor88 are you building something similar(json - s3 - glue)?  if so and if you want to use dlt we can assist you in some PoC.

We have all the above already in place (except snowflake external tables).
We do often extract data from APIs (JSON) and persist to our bronze layer using AWS data wrangler ](https://aws-sdk-pandas.readthedocs.io/en/stable/), then we perform the remaining transformation using dbt-athena.

if so and if you want to use dlt we can assist you in some PoC.
I would say we are not interested for now, my interest in the above is just to help you to build an amazing product that cover more uses cases/setups :)

A feature like this would make dlt fit in perfectly into our architecture. At the moment we're trying to figure out how to use a glue crawler to catalog the data landed in s3. So far it's not playing well because of the file name format in the filesystem destination. Will probably have to implement our own table creation process.
As an added bonus allowing the definition of partitions would be excellent.

@lukas-gust we plan to start working on this next week. lack of filesystem destination was the blocker and now we are ready. it would be very helpful to get any requirements for file name format for filesystem from you. this is something that could be configured - currently we have a first attempt on it and we need feedback.
do you mean partitions in parquet or in table catalogue? both are possible, in the first case we would need to copy a folder with partitioned parquet

@nicor88 not sure if you still are interested in this, but if you could share code that creates and updates AWS Glue catalogue. it looks trivial to add parquet files as external tables in Redshift and Snowflake but I lack experience on how you manage updates to data and schemas in external tables. I'll talk with @adrianbr but if you have any code to share that would be fantastic

As far as I know for the glue catalog integration it doesn't matter too much what the files look like, but it might simplify implementation if a table was represented in a folder/key e.g. /datalake/<table_name>/<load_id>.<file_id>.<file_format> or something. Then the glue catalog could just reference the table directory and infer that all the files in there belong to the table when it's read. As it stands now I think there would a be a challenge using, but I'm not super familiar, yet, with using awswrangler or the Glue Table API. Most of my experience is with using Athena DDL statements.

@rudolfix We leverage external schemas in Redshift. Meaning if you can handle the schema drift in the glue catalog then the external schema will just pick up the new external tables automatically without having to write create external table statements. From the redshift docs:

If your external table is defined in AWS Glue, Athena, or a Hive metastore, you first create an external schema that references the external database. Then you can reference the external table in your SELECT statement by prefixing the table name with the schema name, without needing to create the table in Amazon Redshift. For more information, see Creating external schemas for Amazon Redshift Spectrum.

So basically you'd catalog the table in Glue with the same mechanism and add the external schema to redshift. Though they do recommend you manage external dbs/schemas/tables through redshift.
Does that make sense?

So basically you'd catalog the table in Glue with the same mechanism and add the external schema to redshift. Though they do recommend you manage external dbs/schemas/tables through redshift.
Does that make sense?
yes and adding AWS Glue as a destination does not look overly complicated ie. similar to what dbt-duckb does here (https://github.com/jwills/dbt-duckdb/blob/master/dbt/adapters/duckdb/plugins/glue.py - mentioning this because I know this code, it may not be the best example :)

going back to external redshift tables - it looks like that if we just add EXTERNAL keyword and location of the files to our existing Redshift destination - it will just work. so we'll investigate that as well

@rudolfix As far as I know for the glue catalog integration it doesn't matter too much what the files look like, but it might simplify implementation if a table was represented in a folder/key e.g. /datalake/<table_name>/<load_id>.<file_id>.<file_format> or something. Then the glue catalog could just reference the table directory and infer that all the files in there belong to the table when it's read. As it stands now I think there would a be a challenge using, but I'm not super familiar, yet, with using awswrangler or the Glue Table API. Most of my experience is with using Athena DDL statements.
OK - we can let people choose their preferred layout by using format specifiers with placeholders like "{table_name}/{load_id}..." so users can define their layout. maybe with some more specifiers like current day etc. I'll work on a ticket with requirements and I'll ping you to get feedback

By partitions I mean native to s3 and the glue table and/or athena. We could also go down the open table route which would be neat.
what do you mean by open table route? that we maintain ie iceberg hive locally or in a bucket and maintain the schema ourselves? there my knowledge is quite limited

OK - we can let people choose their preferred layout by using format specifiers with placeholders like "{table_name}/{load_id}..." so users can define their layout. maybe with some more specifiers like current day etc. I'll work on a ticket with requirements and I'll ping you to get feedback: https://github.com/dlt-hub/dlt/issues/510

Very nice! I think this will add a healthy amount of flexibility with the filesystem destination.
RE: Redshift external schema/tables. How are you/we planning to handle the need for an IAM role? Or is that handled already by the Redshift destination? I.e. the external queries need policies in place to allow the Redshift user/service account to read the data in the S3 buckets.

what do you mean by open table route? that we maintain ie iceberg hive locally or in a bucket and maintain the schema ourselves? there my knowledge is quite limited

RE: partitions. I worded that a bit weird, they were meant to be separate ideas. By mentioning open table formats I mean supporting (where it's possible) hive, iceberg, hudi, and delta etc. More or less allowing the 'merge' write disposition to be possible for a the filesystem destination OR is it a new destination at this point?
I'm only just learning more about Iceberg in particular, but I've used Delta tables before. Are there specific questions about schema evolution and management that you can ask so that I can help clarify?

re. Redshift: you can provide an iam role or if not provided dlt will forward the AWS credentials used by staging to copy files to s3 (https://dlthub.com/docs/dlt-ecosystem/destinations/redshift#staging-support)

RE: partitions. I worded that a bit weird, they were meant to be separate ideas. By mentioning open table formats I mean supporting (where it's possible) hive, iceberg, hudi, and delta etc. More or less allowing the 'merge' write disposition to be possible for a the filesystem destination OR is it a new destination at this point?
This is most probably a new destination. To say more I'd need to do a deeper research. We'd need a query engine somewhere to interact with those tables ie. https://aws.amazon.com/about-aws/whats-new/2023/07/amazon-redshift-querying-apache-iceberg-tables/ or https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-iceberg.html (this just looks like an option when creating tables)

I'm only just learning more about Iceberg in particular, but I've used Delta tables before. Are there specific questions about schema evolution and management that you can ask so that I can help clarify?
We are prioritizing tasks today. I'll get back to you with more questions.

One thing to clarify what I'm referring to regarding Redshift is the following quote from https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html

In addition to external tables created using the CREATE EXTERNAL TABLE command, Amazon Redshift can reference external tables defined in an AWS Glue or AWS Lake Formation catalog or an Apache Hive metastore. Use the CREATE EXTERNAL SCHEMA command to register an external database defined in the external catalog and make the external tables available for use in Amazon Redshift. If the external table exists in an AWS Glue or AWS Lake Formation catalog or Hive metastore, you don't need to create the table using CREATE EXTERNAL TABLE. To view external tables, query the SVV_EXTERNAL_TABLES system view.

This may be a desired integration for some, essentially being able to "load" data into redshift via adding an external schema that is managed by say a glue crawler that crawls event files landed by a kinesis stream, or some other process outside of dlt. Thoughts?

We'd need a query engine somewhere to interact with those tables ie. https://aws.amazon.com/about-aws/whats-new/2023/07/amazon-redshift-querying-apache-iceberg-tables/ or https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-iceberg.html (this just looks like an option when creating tables)

Hence where, in the case of AWS, Athena could enter as query engines to manage the crud operations.
https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-managing-tables.html
I don't imagine the destination would be something like 'iceberg'. It'd be more along the lines of 'athena' or 'presto'.

@lukas-gust I have been reading the aws docs for Athena and Glue and to me it seems at this point (and please correct me if I am incorrect, I have just started on this topic) that a s3 + Athena destination would make sense for us a this point. dlt knows the schema of the current load package and can use Athenas HiveQL DDL statements to create and update table schemas based on sets of s3 files. There is no need for the glue crawler to infer the schema and you can then query your tables via Athena and by extension also from Glue. Does what I write make sense to you or am I missing something? I will also think about how to add partitions. Most likely it seems to me at this point, that we could add a field on the resource to add the keys on which to partition on, but that would also mean extending the dlt schema with these partition keys, so probably a bigger project.

@sh-rp You're 100% correct. It's also possible to use the Glue API or what @nicor88 mentioned with aws_wrangler library to create tables and such.
The glue crawler was meant to be an example, but it is not necessary and would always be in the hands of the implementation and not dlt. And if #510 is implemented faster that is what I will be utilizing in the short term. At the moment I'm planning my implementation around building my own destination the best I can.
Partitions is a nice to have IMO. The catch is that you can only add 100 physical partitions at a time in Athena so if there were more for say a full refresh or something you'd have to batch the partition add statements, which I have done by hand at one point. It also looks like if you enforced hive style partitions you would not have to do so, but would need to run a repair command, might have some control there. https://docs.aws.amazon.com/athena/latest/ug/partitions.html
Athena and Glue Catalog are synonymous if you add something via one it's available from the other. One gripe I have is that it's only a two-level namespace `..

@lukas-gust thanks for your detailed answer. After more reading it seems like ticket 510 is necessary in any case, as its seems Athena tables always reference a full folder and never a subset of items in a folder, so we might change our default filesystem storage output to have one "folder" per table anyway. Will keep you posted here or in slack about our further thoughts and progress on this.

@lukas-gust I have updated the tickets with my thoughts, feel free to let me know if there is stuff missing or anything else comes to mind.

@lukas-gust another question: Would there be any sense in writing all the data into iceberg tables? What I have in mind now is uploading parquet files to s3 and referencing folders with Athena/Glue Catalog. An alternative seems to be to create iceberg tables and updating those via the sql interface. I think this might have a few upsides with regards to merging datasets, but It seems to be that this approach might actually be quite slow in the end.

TLDR; landing as JSON or Parquet is perfectly acceptable IMO. Iceberg would be a nice addition for merging. Partitioning is important to optimize cost and performance.
I think it's better to leave the extracted data closest to original form, this is a general approach I prefer and isn't the end all be all. I could see an implementation with Iceberg tables being helpful, but they're managed tables so Iceberg owns the files, whereas the default hive table is just a reference to a set of files.
I think the biggest benefit you get with loading via Iceberg is merging capability and maybe some partition mechanics. I'm not ultra familiar with iceberg yet. Not sure what slowness you might be referring too.
dbt-athena allows both, where merging updates is only possible via iceberg tables. https://github.com/dbt-athena/dbt-athena#iceberg. But I don't think it started there.


Increase Banshee timeout to 360s

I think the 180s are a bit too optimistic for one of the tests. Let's double it, still better than 5h :-)

Agree 👍 Did you manually catch the trailing white-space or was that your editor and if so, which and configured how?

That was my editor :) VS Code using the editorconfig plugin which should pick up the .editorconfig in the root of this project. But I think I also manually configured it to trim all whitespaces.


NPM integration FAQ fix

Fixes for FAQ for 2.2 according to https://github.com/brunch/brunch/pull/1120

:+1:


Email notifier stuck in testing and throwing error

Rancher versions: 2.0 master 2/22
Steps to Reproduce:

Setup an email notifier
Click on test

Results:  Testing button stuck in testing -
Error in dev console -

@tfiduccia  I can not reproduce the issue. Could you provide me the env or the request data?

@tfiduccia I tested with the latest rancher:master and could not reproduce the issue. Can you check with the lateset rancher:master?

Screen shot if I input the valid data.

This issue can be reproduced if user re-type the value for port. Then UI is passing a string instead of the int to backend which causes it.

@vimniky please fix it. thanks.

This is because the test flow doesn't go through the normal validate() call and save flow.  HTML <input> values are always a string, but if the schema says a field is an int/float it's converted during validation (in ember-api-store/addon/models/resource.js validationErrors()).
Checking for validation errors before doing the test will be both useful and fix this.

@vincent99  Thanks!  Fixed

Version - 2.0 master 2/27
Verified fixed


Add all packages back to the "Scheduled release of all packages" workflow

Due to GitHub's error "Too many workflows are referenced, limit is 20", the "Scheduled release of all packages" workflow is currently missing a few packages.
From https://github.community/t/too-many-workflows-are-referenced-limit-is-20/211391 it seems that "work is in progress to change this to a distinct count", which would effectively allow us to build all packages, as we are only referring to two distinct worflows.
Once the limit is lifted:

0e979b4b5d0ac41938ac89050ca43d67665c284d should be reverted, thus adding back mock, gcc and all end user packages
if conditions for steps associated to end user packages should be updated as in the rest of the file. mock and gcc steps should be double checked too.
vtk and itk should be added in a position suggested by the dependency graph (i.e., vtk at the same level of boost, mpi4py and occ, itk one level below that)

The link to the discussion above is dead, and in the meantime the error message has slightly changed
error parsing called workflow "fem-on-colab/fem-on-colab/.github/workflows/schedule_end_user.yml@main": job "test_ngsolve" calls workflow "fem-on-colab/fem-on-colab/.github/workflows/release.yml@main", but doing so would exceed the limit on called workflow depth of 1

gcc, vtk and itk have been added back. End user packages have not been added back. This seems satisfactory enough for now, since they have their own workflow anyway.


port logic.embedding.basic

From mathlib3port: e50b8c261b0a000b806ec0e1356b41945eda61f7

@kbuzzard Instructions page says to make a commit with mathlib3port version first so that one can easily see the diff between auto conversion and final version.

Thanks Yury for working on this when I was asleep! I've added the commit from the mathlib3port README. This has been marked "awaiting review" but I left several TODOs in the text which are still there.

@kbuzzard I see 1 TODO that is still there (about trans). I hope that someone will tell us what to do about it.

I've added the Trans instance. This was what I was told was the appropriate replacement for the @[trans] attribute when I ported logic.relation. I think this is ready for review now.

bros merge

Um, okay. :-)
bors merge


Add Material You Design

Please improve the ui to material you design language. It would look modern and consistent

Blocked by: https://github.com/flutter/flutter/issues/91605

Coming in the next release in a few days


downlink message sent without delay

Hi!
I've downloaded and installed Loraserver + lora-app-server 6 month ago and was able to launch it out of box. But I have problem with new version:  Loraserver sends Join accept packet immediately so Lorawan node not able to detect it as I assume it expects a delay between Uplink and downlink message. Not even sure if there is a problem with loraserver or with packet-forwarder (fresh new version 4.0.1). TTN handle Join request and sends responce after 5 sec delay and end node able to receive Join accept.
Some info can be found here:
https://forum.loraserver.io/t/end-node-doesnt-see-lora-app-servers-join-response/2883/3
Here I got advice to edit some config parameter but not sure that this is proper way. As I understand delay between uplink and downlink message should be implemented out of box as it was with older version of loraserver.
What is the proper way of handling delays between uplink and downlink messages as it it required for lorawan protocol?

Looking at the linked topic, it seems the issue has been resolved and was caused by a mis-configured LoRaWAN version (1.0 vs 1.1).


During the create and refine process, intermediate responses are not saved

Hi @jerryjliu. We talked on Discord about this and I figured I'd open an issue in order to track it and possibly recruit help, since I don't think I'm quite familiar enough with your code to tackle it myself yet.
Using ChatGPT to query a document index, I find that I sometimes get a response such as "Return the original answer. The new context is not relevant to the question ..." It would be helpful to store the responses that are being generated during the create and refine process, in case they are more useful than the final response.
As a workaround, I'm using a custom StringIO logger to grab the debug output, and then filtering each line looking for "Initial response" and "Refined response." In order to make this work, I modified "indices/response/builder.py" to remove newlines which are sometimes being added in front of or within the responses.
https://github.com/jerryjliu/llama_index/compare/main...Martok88:gpt_index:main#diff-8583bac0dcc8a7df7316fa7cc4b57e087845f8ccc9d02f5272ebf2d07d345b24
# Additional imports needed
import logging
from io import StringIO

# Configure custom string IO logger
logging.basicConfig()
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
log_capture_string = StringIO()
log_handler = logging.StreamHandler(log_capture_string)
log_handler.setLevel(logging.DEBUG)
logger.addHandler(log_handler)

# Suppress debug messages from imported modules (Optional, makes console output more readable)
gpt_index_logger = logging.getLogger("gpt_index")
gpt_index_logger.setLevel(logging.INFO)
openai_logger = logging.getLogger("openai")
openai_logger.setLevel(logging.INFO)
urllib3_logger = logging.getLogger("urllib3")
urllib3_logger.setLevel(logging.INFO)

# ... Query as usual, then ...
# Loop over debug messages to get initial and refined responses
responselist = []
log_contents = log_capture_string.getvalue()
log_capture_string.truncate(0)
for line in log_contents.splitlines():
    if line[0:20] == "> Refined response: " or line[0:20] == "> Initial response: ":
        responselist.append(line[2:])
response.response = "\n\n".join(responselist)

Should be solved in #745 !


Accessing the collection resource from an offer fails, works in documentdb

Package Name: @azure/cosmos
Package Version: 3.5.3
Operating system: MacOS 10.15.2
[x] nodejs

version: 10.16.3

[ ] browser

name/version:

[x] typescript

version: 3.7.2

Is the bug related to documentation in

[ ] README.md
[ ] source code documentation
[ ] SDK API docs on https://docs.microsoft.com

Describe the bug
Accessing the collection resource from an offer fails in v3 of the @azure/cosmos.
Using<EMAIL_ADDRESS>is successful.
Use Case is setting the RUPS for a specific collection.
To Reproduce
Steps to reproduce the behavior:
Using @azure/cosmos v3

Get all offers
const newClient = new CosmosClient({endpoint: credentials.host, key: credentials.masterKey}); const offers: Offers = newClient.offers; const feedResponse = await offers.readAll().fetchAll();
Iterate over all offers
await asyncForEach(feedResponse.resources, async def => {      await readOffer(newRups, client, dbname, collname, def); })
Foreach offer read the referenced collection
const readOffer = async (offerDefinition) => {  const collectionConfig = parseCollectionUrlToConfig(oldOfferDefinition.resource);  containerResponse = await client          .database(collectionConfig.databaseName)          .container(collectionConfig.collectionName)          .read(); }
Reading fails
{  "code": "Unauthorized",  "message": "The input authorization token can't serve the request. Please check that the expected payload is built as per the protocol, and check the key being used. Server used the following payload to sign: 'get\ncolls\nqvddairhgd8=\nmon, 13 jan 2020 09:10:53 gmt\n\n'\r\nActivityId: 43f3ccbb-16f2-4467-95a3-6ba127219719, Microsoft.Azure.Documents.Common/2.7.0" }

Using documentdb

Get the client
Same as above

Iterate over all offers
Same as above

Foreach offer read the referenced collection (using<EMAIL_ADDRESS>const readOffer = async (offerDefinition) => {  const oldClient = new DocumentClient(credentials.host, {masterKey: credentials.masterKey}); const collectionResponse = await oldClient.readCollectionAsync(oldOfferDefinition.resource);

Get clear text name of collection by checking the response header 'x-ms-alt-content-path'
if (collectionResponse.headers['x-ms-alt-content-path'] !== 'dbs/' + dbName) return;  if (collectionResponse.resource.id !== collName) return;
Expected behavior
Using @azure/cosmos@3 client reading and obtaining the referenced collection to an offer should still work. Documentation of different behaviour should be provided otherwise.

Additional context
The way the resourceId is computed seems to have changed from v1 to v3.
While v1 computes a collection-resourceId of the form abcdefghij8=, v3 computes an id of the form dbs/AbcdeAA==/colls/abcdefghij8=.
I extracted the 'naked' HTTP request (using the corresponding http-header fields; 'authorization' in particular) that is issued by client v1 and client v3.
Request generated by v1 works as expected. Request generated by v3 fails accordingly.
If I generate the authorization header field using the short form of the ID the request succeedes.

@holgergp Thanks for the report. One of the main changes in v1 -> v2/v3 was using the user-provided id to reference databases/containers instead of _rid. Most of the change was internal except for Offers which still use _rid for some backend compatibility reasons.
I need to investigate more to find the best fix for this. In the meantime, you should be able to get the same info by matching offer.offerResourceId to collection._rid. Let me know if this works for your use case:
const { CosmosClient } = require('@azure/cosmos')

const client = new CosmosClient({
  endpoint: '... your endpoint ...',
  key: '... your key ...'
})

async function main() {
  const { resources: databases } = await client.databases.readAll().fetchAll()
  const containerResponses = await Promise.all(
    databases.map(database => {
      return client
        .database(database.id)
        .containers.readAll()
        .fetchAll()
    })
  )

  const containers = containerResponses
    .map(response => response.resources)
    .flat()
  const { resources: offers } = await client.offers.readAll().fetchAll()

  for (const offer of offers) {
    const database = databases.find(
      database => database._rid === offer.offerResourceId
    )
    const container = containers.find(
      container => container._rid === offer.offerResourceId
    )
    // Important! Offers can now be connected to a database or a container since we support shared throughput
    if (database) console.log('Database', database.id, '-> Offer', offer.id)
    if (container) console.log('Container', container.id, '-> Offer', offer.id)
  }
}

main().catch(e => {
  console.log(e)
})

Thanks! @southpolesteve for the quick reply!
I'll try the workaround today and post if this works. Looks promising!
Can I help document or fix this issue?

Thanks again @southpolesteve! I can confirm that your workaround is working.
Additionally this 'workaround' looks imho much nicer compared to my version using the responseHeader-field!
Maybe though it would be a good idea to provide this snippet in the examples section of the project https://github.com/Azure/azure-sdk-for-js/tree/master/sdk/cosmosdb/cosmos/samples.
Maybe also including setting of the rups/throughput itself.
This would have helped me a lot working with the API.
Should I provide a pull request?

That would be awesome! PRs definitely welcome. FYI it is also possible to set throughput on create for databases and containers. Updates must go through the offer API https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/cosmosdb/cosmos/CHANGELOG.md#set-throughput-as-part-of-dbcontainer-create-operation-220


Critical SEO Tag Implementation Issues: missing tags which effect Search Engine Visibility and Performance

Description
SEO tags play a crucial role in optimizing web pages for search engines and improving their visibility in Google rankings. However, some pages in the application are currently lacking essential SEO tags, which can hinder their performance in search engine results pages (SERPs).
examples-
Lack of Metadata
Missing Alt Attributes
Lack of Structured Data (Schema.org markup)
XML Sitemap many more
plz assign me this task under GSSOC-2023
Screenshots

Additional information
By adding important SEO tags such as meta title, meta description, and heading tags to our web pages, we can provide search engines with relevant information about the content and context of each page. These tags help search engines understand the purpose of our pages and match them with relevant user queries.
Benefits:
Improved Search Engine Visibility: Including SEO tags ensures that search engines can effectively crawl and index our web pages, leading to better visibility in search results.
Higher Click-Through Rates (CTR): Well-optimized meta titles and descriptions can attract users' attention in search results, increasing the likelihood of them clicking on our pages.
Enhanced User Experience: SEO tags, such as heading tags, help structure the content on our pages, making it more organized and user-friendly. This improves the overall user experience and encourages visitors to spend more time on our site.
Targeted Keyword Optimization: By strategically incorporating relevant keywords in our SEO tags, we can align our content with user search intent and improve our rankings for specific queries.
To enhance the SEO performance and indexing of our pages, it is essential to implement these missing SEO tags across the application. This will help us rank higher in Google search results, attract more organic traffic, and ultimately improve the overall success of our website.

Assigned @Mrprayag077

@SurajPratap10, it has been quite a while since I made the changes. I kindly request you to create a pull request.


Zoom of entry cover (library) when switching between landscape and portrait

Steps to reproduce
https://github.com/LNReader/lnreader/assets/75091899/b4c3a288-f162-4e0f-a1d9-0f761e1d0d3b
Expected behavior
No zoom of the library covers when switching between landscape and portrait.
Actual behavior
Read title.
LNReader version
1.1.18
Android version
Android 8
Device
Huawei p8 lite 2017
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with a source, I should be opening an issue in the sources repository.
[X] I have updated the app to version 1.1.18.
[X] I will fill out all of the requested information in this form.

Not present in beta


The exception message does not compare right.

🐛 Bug Report

The exception message does not compare right.
To Reproduce
test('Expecting test fail', () => {
    expect(() => {
      throw 'Unknown action type dispatched: UnknownAction.';
    }).toThrow('Unknown action type dispatched: UnknownAction');
  });

Steps to reproduce the behavior:
Expected behavior
Expecting test fail

Link to repl or repo (highly encouraged)

envinfo

System:
    OS: macOS 10.15.5
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  Binaries:
    Node: 14.1.0 - /usr/local/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.14.5 - /usr/local/bin/npm

According to jest documentation:

You can provide an optional argument to test that a specific error is thrown:
string: error message includes the substring
error object: error message is equal to the message property of the object

So, in order to check the exactly message, an error object should be used instead.
Like:
test('Expecting test fail', () => {
  expect(() => {
    throw 'Unknown action type dispatched: UnknownAction.';
  }).toThrow(new Error('Unknown action type dispatched: UnknownAction'));
});

You right. Thanks a lot.


Report error for incomplete translators comments

Is your feature request related to a problem?
// translators: %1$s: world
_e( 'hello %1$s and %2$s', 'demo' );

=> translators comment is missing %2$s
Also give an error, if the translators comment has more placeholders than the actual string.
Describe the solution you'd like
Error: "Mismatch between placeholders in translators comment and the actual string. Please make sure you use the same number and type of placeholders in both"

For context, this was originally filed at https://github.com/wp-cli/i18n-command/issues/279
I18nSniff:: check_for_translator_comment() currently only checks for the presence of a translators comment if one of the text strings contains a placeholder, but it does not check whether the placeholders are adequately described.
For the case above, WordPress core uses the following style:
// Incorrect:

// translators: %1$s: John
_e( 'hello %1$s and %2$s', 'demo' );

// Incorrect:

// translators: %1$s: John %2$s: Jane
_e( 'hello %1$s and %2$s', 'demo' );

// Correct:

// translators: 1: John. 2: Jane.
_e( 'hello %1$s and %2$s', 'demo' );

Enforcing a specific style (1: John. 2: Jane. vs %1$s: John %2$s: Jane) could be considered to be out of scope, but I think it's well established in the WordPress ecosystem and 1: John. 2: Jane. is the format most commonly encountered by translators.

Enforcing a specific style (1: John. 2: Jane. vs %1$s: John %2$s: Jane) could be considered to be out of scope, but I think it's well established in the WordPress ecosystem and 1: John. 2: Jane. is the format most commonly encountered by translators.

I would stick with the more common standard of %s/%1$s,... as this is used outside of WP too, whereas the 1:, 2: is more of a WP thing, that is not adhered to by most of WP plugins either (most use %1$s in translators comments)

I have a feeling this is basically a duplicate of #1229. Please confirm and if so, let's keep the conversation in the original ticket.

It does sound like a duplicate indeed 👍

@swissspidy Thanks for confirming. I will close this issue in that case, though with the issue mention, they are now linked together so whenever this gets solved all information is available.


These credentials do not match our records.

Laravel Version: 5.8
Adldap2-Laravel Version: 6.0
PHP Version: 7.1
LDAP Type: ActiveDirectory

Description:
Hi, i try to reproduce every single step of the documentation(https://adldap2.github.io/Adldap2-Laravel/#/usage) without any tweak, but i think i missed something.
I'm trying to configure it to work with Ms active directory, using the userprincipalName.
My problem is when I try to login with a username and password (email address), I get the following error:

These credentials do not match our records

The logging is set to true and this log give me the next information:
[2019-03-24 22:48:06] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Binding - Username:
[2019-03-24 22:48:07] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Bound - Username:
As you can see, the Username is empty and doesn't look like correct.
I have also searched related issues but none of them seem to pertain to my situation.
Thanks for your help
ldap_auth.php
`
return [
'connection' => env('LDAP_CONNECTION', 'default'),

'provider' => Adldap\Laravel\Auth\DatabaseUserProvider::class,

'model' => App\User::class,

'rules' => [

    // Denys deleted users from authenticating.

    Adldap\Laravel\Validation\Rules\DenyTrashed::class,

    // Allows only manually imported users to authenticate.

    // Adldap\Laravel\Validation\Rules\OnlyImported::class,

],

'scopes' => [

    // Only allows users with a user principal name to authenticate.
    // Suitable when using ActiveDirectory.
    // Adldap\Laravel\Scopes\UpnScope::class,

    // Only allows users with a uid to authenticate.
    // Suitable when using OpenLDAP.
    // Adldap\Laravel\Scopes\UidScope::class,

],

'identifiers' => [

    'ldap' => [

        'locate_users_by' => 'userprincipalname',

        'bind_users_by' => 'distinguishedname',

    ],

    'database' => [

        'guid_column' => 'objectguid',

        'username_column' => 'email',

    ],

    'windows' => [

        'locate_users_by' => 'samaccountname',

        'server_key' => 'AUTH_USER',

    ],

],

'passwords' => [

    'sync' => env('LDAP_PASSWORD_SYNC', false),

    'column' => 'password',

],
'login_fallback' => env('LDAP_LOGIN_FALLBACK', false),

'sync_attributes' => [

    'email' => 'userprincipalname',

    'name' => 'cn',

],

'logging' => [

    'enabled' => env('LDAP_LOGGING', true),

    'events' => [

        \Adldap\Laravel\Events\Importing::class => \Adldap\Laravel\Listeners\LogImport::class,
        \Adldap\Laravel\Events\Synchronized::class => \Adldap\Laravel\Listeners\LogSynchronized::class,
        \Adldap\Laravel\Events\Synchronizing::class => \Adldap\Laravel\Listeners\LogSynchronizing::class,
        \Adldap\Laravel\Events\Authenticated::class => \Adldap\Laravel\Listeners\LogAuthenticated::class,
        \Adldap\Laravel\Events\Authenticating::class => \Adldap\Laravel\Listeners\LogAuthentication::class,
        \Adldap\Laravel\Events\AuthenticationFailed::class => \Adldap\Laravel\Listeners\LogAuthenticationFailure::class,
        \Adldap\Laravel\Events\AuthenticationRejected::class => \Adldap\Laravel\Listeners\LogAuthenticationRejection::class,
        \Adldap\Laravel\Events\AuthenticationSuccessful::class => \Adldap\Laravel\Listeners\LogAuthenticationSuccess::class,
        \Adldap\Laravel\Events\DiscoveredWithCredentials::class => \Adldap\Laravel\Listeners\LogDiscovery::class,
        \Adldap\Laravel\Events\AuthenticatedWithWindows::class => \Adldap\Laravel\Listeners\LogWindowsAuth::class,
        \Adldap\Laravel\Events\AuthenticatedModelTrashed::class => \Adldap\Laravel\Listeners\LogTrashedModel::class,

    ],
],

];
`
ldap.php
`
return [
'logging' => env('LDAP_LOGGING', true),

'connections' => [

    'default' => [

        'auto_connect' => env('LDAP_AUTO_CONNECT', true),

        'connection' => Adldap\Connections\Ldap::class,

        'settings' => [

            'schema' => Adldap\Schemas\ActiveDirectory::class,

            'account_prefix' => env('LDAP_ACCOUNT_PREFIX', ''),

            'account_suffix' => env('LDAP_ACCOUNT_SUFFIX', ''),

            'hosts' => explode(' ', env('LDAP_HOSTS', '<IP_ADDRESS>')),

            'port' => env('LDAP_PORT', 389),

            'timeout' => env('LDAP_TIMEOUT', 5),

            'base_dn' => env('LDAP_BASE_DN', 'OU=--,OU=--,OU=--,DC=--,DC=corp,DC=--,DC=com'),

            'username' => env('user@...com'),
            'password' => env('password'),

            'follow_referrals' => false,

            'use_ssl' => env('LDAP_USE_SSL', false),
            'use_tls' => env('LDAP_USE_TLS', false),

        ],

    ],

],

];
`
auth.php
`
return [
'defaults' => [
    'guard' => 'web',
    'passwords' => 'users',
],

'guards' => [
    'web' => [
        'driver' => 'session',
        'provider' => 'users',
    ],

    'api' => [
        'driver' => 'token',
        'provider' => 'users',
        'hash' => false,
    ],
],

'providers' => [
    'users' => [
        'driver' => 'ldap',
        'model' => App\User::class,
    ],

    // 'users' => [
    //     'driver' => 'database',
    //     'table' => 'users',
    // ],
],

'passwords' => [
    'users' => [
        'provider' => 'users',
        'table' => 'password_resets',
        'expire' => 60,
    ],
],

];
`
loginController(default)
`
namespace App\Http\Controllers\Auth;
use App\Http\Controllers\Controller;
use Illuminate\Foundation\Auth\AuthenticatesUsers;
class LoginController extends Controller
{
use AuthenticatesUsers;

/**
 * Where to redirect users after login.
 *
 * @var string
 */
protected $redirectTo = '/home';

/**
 * Create a new controller instance.
 *
 * @return void
 */
public function __construct()
{
    $this->middleware('guest')->except('logout');
}

}
`

Steps To Reproduce:

I forgot to mention, I sure about the ldap information 'cause I use it in another application. Thanks!

Hi @mgrossenbacher86,
Strange, your logs are saying that it successfully bound to your server with an empty username?
[2019-03-24 22:48:06] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection: - Operation: Adldap\Auth\Events\Binding - Username:
[2019-03-24 22:48:07] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection: - Operation: Adldap\Auth\Events\Bound - Username: <-- Here

Can you try clearing your config cache using php artisan config:clear and try again?
Also, your base_dn seems pretty lengthy. Is the user you're authenticating with contained in the OU you've given?
 'base_dn' => env('LDAP_BASE_DN', 'OU=--,OU=--,OU=--,DC=--,DC=corp,DC=--,DC=com'),

Hi Steve, thanks for your help.
I clear the cache and set a small path of the base dn, to set it more general. My user, it is inside of that base (env('LDAP_BASE_DN', 'OU=--,OU=--,OU=--,DC=--,DC=corp,DC=--,DC=com'),).
The log is showing the same error:
[2019-03-25 16:32:56] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Binding - Username:
[2019-03-25 16:32:56] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Bound - Username:
Do you have some suggestions? because I really lose here
Thanks!

No problem @mgrossenbacher86! This is definitely strange indeed.
Do you see other logs by attempting a login? You should see about 14 log entries in total when logging is enabled:
[2019-03-25 12:39:07] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Binding - Username<EMAIL_ADDRESS>[2019-03-25 12:39:07] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Bound - Username<EMAIL_ADDRESS>[2019-03-25 12:39:07] local.INFO: User 'Bauman, Steve' has been successfully found for authentication.
[2019-03-25 12:39:08] local.INFO: User 'Bauman, Steve' is being synchronized.
[2019-03-25 12:39:08] local.INFO: User 'Bauman, Steve' has been successfully synchronized.
[2019-03-25 12:39:08] local.INFO: User 'Bauman, Steve' is authenticating with username: 'CN=Bauman\, Steve,OU=Users,DC=corp,DC=acme,DC=org'
[2019-03-25 12:39:08] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Attempting - Username: CN=Bauman\, Steve,OU=Users,DC=corp,DC=acme,DC=org
[2019-03-25 12:39:08] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Binding - Username: CN=Bauman\, Steve,OU=Users,DC=corp,DC=acme,DC=org
[2019-03-25 12:39:08] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Bound - Username: CN=Bauman\, Steve,OU=Users,DC=corp,DC=acme,DC=org
[2019-03-25 12:39:08] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Passed - Username: CN=Bauman\, Steve,OU=Users,DC=corp,DC=acme,DC=org
[2019-03-25 12:39:08] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Binding - Username<EMAIL_ADDRESS>[2019-03-25 12:39:08] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Bound - Username<EMAIL_ADDRESS>[2019-03-25 12:39:08] local.INFO: User 'Bauman, Steve' has successfully passed LDAP authentication.
[2019-03-25 12:39:08] local.INFO: User 'Bauman, Steve' has been successfully logged in.

No, actually this is all:
[2019-03-25 16:31:02] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Binding - Username:
[2019-03-25 16:31:02] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Bound - Username:
[2019-03-25 16:32:56] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Binding - Username:
[2019-03-25 16:32:56] local.INFO: LDAP (ldap://<IP_ADDRESS>:389) - Connection:  - Operation: Adldap\Auth\Events\Bound - Username:
Do you know another test to check if in that case something else is showing in the log?

Looks like you're not actually authenticating with the Adldap2 auth driver (ldap).
Have you modified the LoginController that Laravel publishes when running php artisan auth:make command?

This is the loginController from controller->auth
`class LoginController extends Controller
{
/*
|--------------------------------------------------------------------------
| Login Controller
|--------------------------------------------------------------------------
|
| This controller handles authenticating users for the application and
| redirecting them to your home screen. The controller uses a trait
| to conveniently provide its functionality to your applications.
|
*/
use AuthenticatesUsers;

/**
 * Where to redirect users after login.
 *
 * @var string
 */
protected $redirectTo = '/home';

/**
 * Create a new controller instance.
 *
 * @return void
 */
public function __construct()
{
    $this->middleware('guest')->except('logout');
}

}
`

Hmmm, I would first try changing your base_dn to the following:
 'base_dn' => env('LDAP_BASE_DN', 'DC=corp,DC=--,DC=com'),

Then try logging in again normally and see if you see additional log entries.

I'm having the same issue on Laravel 5.8 although my username is being populated. It just seems to give up after bound. Username and IP starred out for security reasons.
[2019-04-01 16:42:27] local.INFO: LDAP (ldap://******) - Connection:  - Operation: Adldap\Auth\Events\Binding - Username: ******
[2019-04-01 16:42:27] local.INFO: LDAP (ldap://******) - Connection:  - Operation: Adldap\Auth\Events\Bound - Username: ******

Hi @shoeyn, are those all the log events shown? You should see many more events when logging into your app (shown in this comment).

Literally just managed to resolve my issue, the difference was I am using username compared to email and had to change:
'identifiers' => [
    'ldap' => [
        'locate_users_by' => 'userprincipalname',

to
'identifiers' => [
    'ldap' => [
        'locate_users_by' => 'samaccountname',

Wondering whether there's a field mixup for @mgrossenbacher86

Great to hear @shoeyn! Glad you were able to resolve your issue. This is kind of the double-edge sword of being configuration heavy.
Due to inactivity from the OP, I'm closing this one out.


fix: handle more errors with mailbox listing

Proposed change

Fix bool error message in config flow.
Handle more errors with mailbox listing.
Type of change

[ ] Dependency upgrade
[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (which adds functionality)
[ ] Breaking change (fix/feature causing existing functionality to break)
[ ] Code quality improvements to existing code or addition of tests
[ ] Documentation update
[ ] Adds a new shipper
[ ] Update existing shipper

Additional information

This PR fixes or closes issue: fixes #934

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 25.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 94.08%. Comparing base (8608d22) to head (660c103).
Report is 13 commits behind head on dev.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #935      +/-   ##
==========================================
- Coverage   95.07%   94.08%   -1.00%
==========================================
  Files           8        8
  Lines        1482     1538      +56
==========================================
+ Hits         1409     1447      +38
- Misses         73       91      +18

Files
Coverage Δ

custom_components/mail_and_packages/config_flow.py
97.25% <25.00%> (-2.75%)
:arrow_down:

... and 2 files with indirect coverage changes


Suddenly unable to search certain directories since 191

Some folders/projects, even in safe mode,  just search forever and ever. Other folders work just fine since
Started happening with 0.191
What can I provide for more info?

Are any errors displaying in the dev tools? cmd-alt-I

ENOENT: no such file or directory, stat '/Users/alexwait/chirpify/ingest/.env' Error: ENOENT: no such file or directory, stat '/Users/alexwait/chirpify/ingest/.env'

Looks like its failing to search symlinked files.
lrwxr-xr-x   1 alexwait  staff      26 Dec 18 14:11 .env -> /etc/profile.d/variables.sh

Okay, thanks for the details, this looks like a duplicate of https://github.com/atom/scandal/issues/22 will publish a new Atom release very soon with the fix.

Thanks so much!

0.192 is out now


Change the order for the list of regressions

Coverage remained the same at 46.462% when pulling a64aa943b1fa22c6b8e73f1d9647d993d9765fe6 on sylvestre:bug into d65263828acc20dcce58b5160abd23646b72681d on mozilla:master.


chore: refactor exports

Description

Move global declarations to index.ts
Ensure all type definitions make it into /dist
Fix return type of swup.getFragmentVisit() (hopefully)

Checks

[x] The PR is submitted to the master branch
[x] The code was linted before pushing (npm run lint)

I'll just merge an release. It's the only way we can know for sure.


Extract XHR stuff from the project

Probably could do with free-ifying Affjax itself, or providing a "driver" by some means, so that XHR and node stuff can be supported without resorting to the xhr2 hacks too.

Do we need to Freeify it? Couldn't we just have all the functions build AjaxRequest configs, and then you can have different backends provide a way to interpret the AjaxRequest? Seems like Free might add an unnecessary layer since the request config has all the information you need.

Yeah, if we have a config that works too. It doesn't work so well with function like get however... could also be something class based perhaps.
I just meant some way of separating the request mechanism from the higher level interface, in whatever form that takes. There is purescript-affjax-algebra already, so a free version isn't something that would need to live in here.

I just meant we already have a config with AffjaxRequest. All the helper functions just modify defaultRequest, and then call affjax. Why do we need them to also call affjax? That's the only thing that ties it to an implementation.

It's the same thing with affjax-algebra https://github.com/slamdata/purescript-affjax-algebra/blob/master/src/Network/HTTP/AffjaxF.purs#L41 Why bake in the implementation instead of just building requests and having the user call run for the appropriate driver.

@garyb, @natefaubion I've just encountered problem related to this xhr check under node with webpack (we are bundling our cli tools into self contained scripts under node) so I want to ask:

Do you still plan to abstract over _ajax?
Do you have any API proposition which I can try to implement to move away _ajax driver?

@natefaubion If I understand your suggestion correctly you opt for changing get, post, put etc. into Request constructors. What about functions like request and retry?
@garyb What do you think about this direction?

That sounds like a reasonable approach to me. request is essentially the interpreter for Request, so would be where you'd switch it out to avoid XHR, right?
retry I don't really think belongs in this library at all, since it could be made a little more general and then work for any Aff - I almost removed it in the last update, but thought I'd avoid breaking things any further than was already happening. :wink:

Oh yeah, thinking about it, the reason I didn't make this change to free it from XHR in the last release is that I wasn't sure how useful Request is as an abstraction - whether we need to reconsider the options / parameters available to make it more suitable for non-XHR requests, that sort of thing.

request is essentially the interpreter for Request, so would be where you'd switch it out to avoid XHR, right?

Ok. On the other hand request is also where some response parsing "magic" happens.

Do we want to move this whole request logic to outside interpreters?

What kind of contrib do we want to provide - something like purescript-affjax-node (with let say Affjax.Node module) and purescript-affjax-web?

I wasn't sure how useful Request is as an abstraction (...)

It is hard to tell. Maybe we can leave it as it is till we find any additional scenarios / requirements which we have to cover.

Do we want to move this whole request logic to outside interpreters?

If there's useful stuff that can be shared between request interpreters then we can provide it from this library for sure.

something like purescript-affjax-node (with let say Affjax.Node module) and purescript-affjax-web?

Yeah, I think so - that way we should avoid any possible issues with bundling things.

Apologies for barging in on a discussion I only half understand, with an issue that probably just obliquely touches the subject at hand. I ran into a problem with XHR2 as it doesn't support cookies which makes it impossible to use for Cookie authentication. There is a library based on XHR2 that does support cookies: https://github.com/souldreamer/xhr2-cookies.git. It solves my problem, so may be useful to others, too.


Set Up the Manage Rule Projects Application

Tutorial URL: https://developers.sap.com/tutorials/cp-cf-businessrules03-setup-mrp.html
Please specify the step you are referring to
Can you please provide steps for Business Application Studio?
Thanks

You do not need Business Application studio for modelling business rules. Business rules have a standalone application called "Manage Rules Application" which is used to model, manage and deploy rules.

Thank you for your feedback. The issue seems to be resolved, so I am closing the issue.
If you still have questions, feel free to reopen the issue.