Implement the BW6_761 curve.

@yelhousni et al. have found a curve, BW6_761 which, like SW6, has scalar field = Bls12_377::Fq, but which has better efficiency properties. It would be great to have an implementation of this curve in our library to enable more efficient recursion.
The implementation consists primarily of the following steps:

Constructing the field, which just requires instantiating the appropriate constants for Fq and Fq6.
Writing code for pairings.

An excellent starting point is the implementation here, and the paper.
Closes #3

cc @howardwu @kobigurk @gakonst this is probably relevant to y'all

Hi, I'd like to take this on. However, this is the first elliptic curve implementation I will undertake, although not the first crypto implementation. I have an idea of what I am doing, but I will have to read the paper carefully and may need some ocassional help.
Before that, I have a long list of questions/ideas that go beyond the scope of this issue.

Why the lack of assembly/SIMD in Zexe? This can be implemented at the field/models level, or lower, at the bigint ops level.
Zexe uses u64 by default. Is this a problem/suboptimal when porting to mobile/WASM/GPU which have 32-bit words?

Before I begin the next section, I would like to first say my priority for the next two weeks is to implement BW6-761. Regardless, when I find the time/reason:
Another project I am interested in is adding some support for GPU through wgpu-rs or some variant, although the WebGPU project is still in development. In particular, I am interesting in adding support for a highly parallel version of multiexponentiation. This could be very good for dedicated provers or personal computers.
I am also interested in adding support for the PLONK proving system, as there have been some exciting and interesting developments there recently. For instance, adding RAM for branching programs. Arithmetisation is not R1CS however, but similar to Groth16 permutation arguments. Since Groth16 is supported here, I assume this is fine. I think one exciting thing this would enable alongside the single-layer proof composition are "dark contracts". I would be interested to discuss how this should be implemented.
My enthusiasm stems mainly from the fact that I was starting to write my own rust crypto library, salsa, but zexe has many of the things I would like for the time being. There is a nice clean structure. Things that are of secondary interest to crypto for which I may want to fork zexe for my own purposes are EC primality testing and factoring.
Ideas I had for the (distant and ideal) future are writing a DSL in Rust that can compose sMPC (perhaps incl. garbled circuits), homomorphic encryption, ZK building blocks and generate all the necessary networking and data-passing between different components and doing optimisation (probably BayesOpt) in order to optimise the balance of different components to a particular hardware/communication setup. Pysyft, targeted at private machine learning, seems to think this DSL should be pytorch. I'm not so sure for now.
The DSL should include annotations that tell the system what kind of security is required for different parts of the computation, data ownership, differential privacy etc. Although, it has yet to be seen how big of an overlap there will be between privacy-preserving computation and SNARKs.
Another idea I had was to bring in a bit of functional verification, in the spirit of Microsoft's Everest Project or what AWS is doing with Galois Software's tools. In this case we might use something like Prusti.
Also I was wondering if it might be worth writing some macros to implement arithmetic for pseudo mersenne and generalised mersenne primes, which are more efficient than for general prime fields. I believe some BN curves use these, but I am not sure. This suggestion, however, is not in line with Zexe's strategy to write explicit implementations for given no. of limbs. May I know why this cannot be achieved with macros?
If you think any of these projects are interesting, I can start issues for them. However, I would like to prove myself a worthy contributor first by completing the BW6-761 implementation succesfully.

To do:

[ ] Parameters
[ ] G1 (omega) & G2 (lambda) GLV scalar multiplication (patent till sept 2020). Other endomorphism-based optimisations protected by same suite of patents: hash to point, subgroup check
[ ] G2 point compression (from twist of deg 6 inducing isomorphism)
[ ] G1 & G2 subgroup checks
[ ] Tests (I wonder if these can be automatically generated for future curves by running a script taking a given curve's params and running a sage script. Curve specific tests can be added on manually.)
[ ] Other curves such as BN curves with ratio ~ 1 for more efficient pairing-friendly chains.

Hi @jon-chuang, just wanted to see what the status of this issue is? Having BW6 in zexe is something we're interested in too and noticed that you've done a good deal of thinking around this and seems to have started working on it. Would be very helpful

Hi @ChaitanyaKonda , unfortunately, I have not begun work on it and will not be able to for the foreseeable future.
Feel free to work on it :) having your contribution will be great!

So I finished implementing and testing BW6-761 in zexe: https://github.com/yelhousni/zexe . There are about 10 branches testing different configurations with respect to the twist type (M or D), the twist degree (quadratic or sextic), the doubling/addition formulas (ABLR or AKLGL), the pairing type (ate and optimal ate) and the coordinates (affine vs projective). The fastest implem so far (which is the one described in the paper) is in the branch youssef/BW6-761-Fq-AKLGL-2ML-D; it implements an optimal ate pairing in ABLR projective coordinates with a sextic M-twist (G2 over Fq, 2 small Miller loops (Alg.5) and a lattice-based optimized final exp (Alg.6)).
The benchmarking shows that BW6-761 is 12X faster for a full pairing and ~22X faster for a Groth16 proof verification (~4 Miller loops and 1 final exp). This is surprising but mainly due to the choice of affine coordinates in SW6. In fact, projective coordinates are always faster for a pairing except in some cases (for instance SW6) as suggested in LautMontNae10 when the inverse is optimized in tower extensions, which is not yet the case for SW6 implementation I guess #20.
full pairing

Miller loop

final exp

There are few optimizations to do further:

[ ] use 2-NAF in the Miller loop
[ ] instantiate the second Miller loop by the first (optimization described in Alg.5)
[ ] use of endomorphism

This is great work @yelhousni ! If you’d look to open PR from that branch then that would be great and we can merge this version of the code already, and the optimizations as followups :)


Use un-url-encoded blobName value, when calculate SAS on the check

Fixes #415
When Azure SDK works with blob and its name contains symbols, which should be encoded in url, signature calculation performed from the un-url-encoded blobName value. In order to SAS check in Azure passes, we need to calculate SAS under the hood in the same way.

Thanks @iBavtovich! A test case failed the pipeline, can you help check?
https://dev.azure.com/azure/Azurite/_build/results?buildId=16293&view=logs&jobId=5c249ced-33d3-5644-8846-374752928f2b&j=5c249ced-33d3-5644-8846-374752928f2b&t=da461190-4564-5ab0-1aee-1dde69759da2

I double checked the problem and found that the request url was url-encoded twice: 1 time by Azure SDK and second by the http client library I used (which is wrong). But on the Azurite side it was decoded once (as should be)  in blobStorageContextMiddleware middleware. So there is no need to decode blobName again in BlobSASAuthenticator.


Support imagePullSecrets in ServingRuntime

Is your feature request related to a problem? If so, please describe.

Most of our use cases are using custom servingRuntime which are built by the users. In this case, the servingRuntime image mostly will be stored in the user's private image registry.
However, the current ServingRuntime CRD doesn't have the imagePullSecrets, so we are not able to provide it to the user.
Describe your proposed solution

support imagePullSecrets under ServingRuntime.spec
Describe alternatives you have considered

Similiar to InferenceService CRD, ServingRuntime should be able to support most of the Pod Spec.
Additional context

@lizzzcai you can provide imagePullSecrets via the service account used for the deployment pods, which can be set via the serviceAccountName configuration parameter (see here). Is that sufficient or you need to be able to set different secrets per serving runtime?

@lizzzcai you can provide imagePullSecrets via the service account used for the deployment pods, which can be set via the serviceAccountName configuration parameter (see here). Is that sufficient or you need to be able to set different secrets per serving runtime?

Hi @njhill , I tested this method but found that it is a system-wide configuration on the modelmesh-serving namespace. Hoever in our case, we are using user-namespace and each namespace will have multiple imagePullSecrets per namespace. So either this model-serving-config can be applied and overwrite in user namespace or enable the full PodSpec in servingruntime will be helpful.  The current workaround for us is to use webhook to inject it, but it will be good to have a long term solution from kserve side.

Hi @njhill , I have created a PR for this. I think it makes sense to add it as it is very common to use images from private repositories, and it is similar to volumes. What do you think?


Feature Request: Support Grafana Alerting

Hi folks! Are there any plans for this data source to support Grafana Alerting?

Grafana Alerting Data Source Compatibility

This feature will not be implemented. If needed, you can use log-based metrics + a monitoring data source for alerting.
Updated the README at: https://github.com/GoogleCloudPlatform/cloud-logging-data-source-plugin#alerting


Support native auth on mobile devices

Feature request
Is your feature request related to a problem? Please describe.
Currently Supabase support only web oAuth which is mainly used for web applications and thus provide no UX to mobile users.
Describe the solution you'd like
Change how the SDK handles auth on mobile devices and this could be achieved by such code example (pseudo code)
consider having such an abstract class
abstract class AuthProvider {
  Provider get provider;
}

then Supabase supported third party auth vendors could be implemented in such way
class GoogleProvider implements AuthProvider {
  final String accessToken;
  final String idToken;

  const GoogleProvider({
    required this.accessToken,
    required this.idToken,
  });

  @override
  Provider get provider => Provider.google;
}

then simply we can let the user uses Google lib to authenticate the user on a mobile flow using
 final user = await _googleSignIn.signIn();
 final googleAuth = await user.authentication;

_supabase.client.signInProvider(GoogleProvider(accessToken: googleAuth.accessToken,
                                               idToken: googleAuth.idToken));

then simply we could call a function in Supabase SDk that could handle this for example:
Future<GotrueSessionResponse> signInProvider({ required AuthProvider provider}) async {
    String accessToken, idToken;
    if (provider is GoogleProvider) {
       accessToken = provider.accessToken;
       idToken = provider.idToken;
    } else if ( provider is Facebook ) {
       accessToken = provider.accessToken;
   }
   final response = await api.signWithProvider(provider.provider, .accessToken , idToken);
    return GotrueSessionResponse(error: response.error);
  }

with that Supabase shall support native auth instead of web focused auth system.
Describe alternatives you've considered

current auth which is almost broken on ios due to redirecting isn't working and user has to manually click done on safari to return to the app.

Additional context
I would definitely like to help here if me jumping in gonna make this FR implemented faster.
Cheers.

Thanks for opening this issue. This feature is heavily requested feature, and the team is well aware of it! Currently, we need this issue to be solved on the server side in order to implement this on the client side.

Thanks for opening this issue. This feature is heavily requested feature, and the team is well aware of it! Currently, we need this issue to be solved on the server side in order to implement this on the client side.

Hello, the mentioned issue seems to be solved, any update on native auth on mobile devices??

@SushilGhorasaini1 we're waiting on https://github.com/supabase/gotrue-js/issues/169

What makes gotrue-dart reliant on the gotrue-js?
@bdlukaa

#61  still wont work unfortunately as nonce doesn't exist in most jwts on mobile. ex: google_sign_in id_token doesn't have nonce so we will always receive errors from backend with missing nonce in id_token

Hmm. I'll take a look!

Hmm. I'll take a look!
Wow that's really annoying that they don't return nonce.
I actually have a little while before I need this feature and I don't have the bandwidth to implement it now. You can go ahead and start if you'd like

This has to be changed in the backend and not in our end. Kang is informed & he created an issue for it in the backend repo, however this wont be fixed until after April afaik.

Ah I see. Do you happen to have a link to the issue?

ex: google_sign_in id_token doesn't have nonce

we can generate our own nonce, like in https://firebase.flutter.dev/docs/auth/social#apple

ex: google_sign_in id_token doesn't have nonce

we can generate our own nonce, like in https://firebase.flutter.dev/docs/auth/social#apple

Yes, you can, but how backend going to verify this nonce from the jwt token if it doesn't exist within?
Because that's how they do the verifying.

https://github.com/supabase/gotrue/issues/412

I made a PR to remove the requirement of nonce when it's not in the returned id_token
https://github.com/supabase/gotrue/pull/430

@DanMossa would that fix the google requirement?

It's kinda hard to tell if that's the only issue but I think it's the only thing stopping it right now.
There's no nonce embedded in some JWT tokens, and so that PR makes it so you don't need to send a nonce if and only if there's no nonce in the JWT

can confirm @DanMossa PR fixed the issue and now I can login with Google.

@k0shk0sh can you provide more info on how you did that?

Below did the trick for me :).
You get onAuthenticated after successful signIn.
     final user = await _googleSignIn.signIn();
     final googleAuth = await user?.authentication;
     if (googleAuth?.idToken != null) {
       final result = await _supabase.client.auth.signIn(
         oidc: OpenIDConnectCredentials(
           nonce: '',
           idToken: googleAuth.idToken!,
           provider: Provider.google,
         ),
       );
       if (result.user != null) return result.user!;
       return Future.error(result.error!.message);
     }

@k0shk0sh
If I'm not wrong, we should reopen this issue, correct?
https://github.com/supabase-community/gotrue-dart/pull/68#issuecomment-1114189925

https://github.com/supabase-community/supabase-flutter/issues/5 is already tracking that


MM-24810 Add e2e for reactions for archived channel posts

Summary
Add e2e tests for reactions of archived channel posts in center channel, RHS, and search
Ticket Link
https://mattermost.atlassian.net/browse/MM-28410

/update-branch

/update-branch


[0.9.0]: Make database serializable and hashable

Proposed changes
A breaking change to the database to make it serializable and hashable. This means that the data inserted into the database are now enforced to be primitive or non-primitive built-in types only, except for sets and frozensets. In essence, the data should be json serializable.
Therefore, a lot of unit tests are expected to fail, because the rounds need to be refactored to not store unserializable data to the db, and the corresponding properties in the synchronized data should now convert them back to the desired type.
Making the database hashable, will allow us to use the digest for the Tendermint app hash. This is a better alternative for the app hash because it guarantees that the state is the same for all the agents at the end of each round.
Fixes
n/a
Types of changes
What types of changes does your code introduce? (A breaking change is a fix or feature that would cause existing functionality and APIs to not work as expected.)
Put an x in the box that applies

[ ] Non-breaking fix (non-breaking change which fixes an issue)
[ ] Breaking fix (breaking change which fixes an issue)
[ ] Non-breaking feature (non-breaking change which adds functionality)
[x] Breaking feature (breaking change which adds functionality)
[ ] Refactor (non-breaking change which changes implementation)
[ ] Messy (mixture of the above - requires explanation!)

Checklist
Put an x in the boxes that apply.

[x] I have read the CONTRIBUTING doc
[x] I am making a pull request against the main branch (left side). Also you should start your branch off our main.
[x] Lint and unit tests pass locally with my changes
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have locally run services that could be impacted and they do not present failures derived from my changes

Further comments
n/a

Please fill the coverage gaps for abstract_round_abci: https://github.com/valory-xyz/open-autonomy/actions/runs/4017500998/jobs/6902151065

Please fill the coverage gaps for abstract_round_abci: https://github.com/valory-xyz/open-autonomy/actions/runs/4017500998/jobs/6902151065

After all, this is not a merge error. Our generators remove these files as they are not relevant anymore. However, our workflow should have detected this, though allows adding the files back (6278dd3).


Large Dexie Database on Ionic/Cordova App Requires App Restart Before Table Appears

Hi, first of all... this library is amazing, and very well documented. Well done to the author.
Anyway, onto my issue.
I am creating an app in Ionic where first time users of the app must sync their app, so that they can get offline search using indexeddb/dexie.
The syncing process downloads roughly 20,000 records into the dexie database.
The problem:
In my ionic/cordova app, after successfully downloading all 20,000 records the table has been created. However, there are no records in the table. Now here is what is weird... if I restart the app, all the records are there successfully!
I verify this by using chrome://inspect to connect to the phone. When I go to the indexeddb tool in developer tools, the table is successfully created, but there are no records.
IF I only download.. say 300 records in to the database, there is no issue. The issue presents itself when the database is "large".
After restarting the app with 20,000 records ... no issues with speed and performance. It all works great.
Furthermore, the issue is not reproduced in the browser, only on an ionic app that loads on the phone with cordova.
Code where I add to database from recursive GET function. Essentially I'm using .put
  fetch(token?) {
    this.api.getCrags(token)
      .subscribe((resp) => {
        this.db.table('table').put(resp.data, resp.key);
        if (resp.nextPageToken) {
          this.fetch(resp.nextPageToken);
        } else {
          // complete
        }
      });
  }

I'm glad to hear you like dexie. Thanks!
I believe the problem could be tied to not awaiting the promises returned by table.put(). It is hard for the application to know when the sync is done as your fetch(token?) method doesn't return neither a Promise or an Observable.
As your data size can consist of 20,000 records, I would strongly advice to use bulkPut() rather than put(). It would boost performance significantly and consume less memory.
Now, it seems you are retrieving data records one by one (or maybe chunk by chunk, but consuming them as an one-by-one). Maybe need to change API to retrieve chunks of records instead and use bulkPut to store them.
Also let your app know when the import is done, by:

Handle the promise returned from bulkPut()
Return Promise or Observable from the fetch() method.

As I have easier to think in a Promise base way, I give you a promise-base example. Can be done equivalent using Observables if you'd prefer.

class MyApi {
  async getCrags(token?) {
    // Dummy return value here (just to visualize to return chunks istead of individual items)
    return {
      chunk: [{value: 'foo', key: 1}, {value: 'bar', key: 2}],
      nextPageToken: null
    };
  }
}

class MyService {
  api: MyApi;
  async fetch(token?) {
    const resp = await this.api.getCrags(token);
    const table = this.db.table('table');
    const values = resp.chunk.map(d => d.value);
    const keys = resp.chunk.map(d => d.key);
    await table.bulkPut(values, keys);
    if (resp.nextPageToken) {
      await this.fetch(resp.nextPageToken);
    }
  }
}

This way, MyService.fetch() will return a Promise<void> that only resolves when all is imported.
Also, if anything goes wrong anywhere, the resulting promise will reject and you may act upon that too.

Hi @dfahlander i will check this out as soon as possible, thanks.


macOS 12 won't actually restart and install the update

Hey,
Love the new script and Config Profile. I noticed that my computers get to last stage of "Installing updates..." hud which then it should restart... but it just doesn't? I know caching is no longer part of the process, so maybe this is because it is caching it which can take 20-30 min on its own?
Not sure if this process is supposed to take a while or if it's me/our devices.

Yeah, I suspect this is due to the very large size of newer macOS updates. If you open System Preferences > Software Update and attempt to run the update in progress, it will show you how far it's gone and estimate download and run times, could always provide that as guidance to folks waiting a long time. But I've definitely seen the update take this long when run manually, and the script unfortunately doesn't provide as much visibility on progress as the native UI.

I'm seeing this, too. I installed the unmodified pkg on a test-machine. The flow of the script works excellent but it hangs on downloading / installing the update. (Intel / Monterey). Let it run for hours. Funny thing is, when I execute the script with sudo I can see the update downloading.
Same behavior when running only the script from Jamf as a policy. If it's started at recurring checkin, the updates hang / don't get downloaded.

I am experiencing the same behavior. Tested on multiple intel Monterey machines

IMO it might be helpful to show the output of the softwareupdate command that's being run (eg you can test by running sudo softwareupdate --install --all --restart)
One example I'm seeing:
Software Update Tool
Finding available software
Downloading macOS Monterey 12.2
Downloading: 87.95%
Failed to download & prepare update: Error Domain=SUOSUErrorDomain Code=201 "Download failed" UserInfo={NSLocalizedRecoverySuggestion=An error occurred while downloading the selected updates. Please check your internet connection and try again., NSLocalizedDescription=Download failed, NSUnderlyingError=0x600001bd04b0 {Error Domain=SUOSUErrorDomain Code=201 "Download failed" UserInfo=
{NSLocalizedDescription=Download failed, NSLocalizedRecoverySuggestion=An error occurred while downloading the selected updates. Please check your internet connection and try again.}
}}

Here is what happens when I try to run that command. It asks for a password, I enter it, and it just hangs endlessly after entering the password and hitting enter.

Intel mac now does find the update but still stuck with the same issue, hanging after password entry.

Wiped both my M1 and Intel Monterey testing machines, installed 12.2 on them (not 12.2.1) and started from scratch with some new testing last night.
Results:
M1: Working as intended
Intel: Still does not work, even leaving the machine on overnight. Same problem as detailed in the above posts remains with it hanging on installing updates (or the password prompts if doing the manual trigger from terminal)

Thanks for helping test this out! Software update instability seems to be a pretty widespread issue with recent macOS Big Sur and macOS Monterey updates, I've seen it discussed all over MacAdmins Slack. I'd recommend reaching out to Apple Enterprise Support to see if relief might come in a future patch.

I already have, snippet below
Subject: Re: [101616806379] SoftwareUpdateD process is inconsistant
Hello,
When you attempted to check for software updates, I see the following error in the log archive:
2022-02-09 14:26:07.091567+0000 0x33C8F Error 0x0 14904 softwareupdated: (SoftwareUpdateMacController) [com.apple.SoftwareUpdateMacController:ScanManager] [ScanFailed] Scan failed with majorError: Error Domain=SUMacControllerError Code=7509 “[SUMacControllerErrorAccessLost=7509] Access control was lost to another client” UserInfo={NSDebugDescription=[SUMacControllerErrorAccessLost=7509] Access control was lost to another client, NSLocalizedDescription=The current software update operation was cancelled due to another process starting a new operation. Please try again later.}
This indicates to me that the softwareupdate binary is being called too frequently.
For testing next steps, I recommend disabling any scripted calls for softwareupdate, disabling “Collect Available software updates” in Jamf’s Inventory Collection, and disabling any other policy in Jamf that might trigger a scripted software update operation. Once completed, run the following kickstart command and perform a single software update scan, using either Terminal or System Preferences:
sudo /bin/launchctl kickstart -k system/com.apple.softwareupdated
Finally, I want to position that Apple recommends using the AvailableOSUpdates command to check for updates via MDM. You may want to speak with Jamf about this command or how they’ve implemented it. Below is a link to our developer documentation on this feature:
https://developer.apple.com/documentation/devicemanagement/list_the_available_os_updates

Hi, sorry new at using your tool but I am still having the same issue with my test Intel system as it hangs as well. Not sure if caught everything posted earlier but seems like there isn't quite a solution yet for Intel Macs?
Here's the log stream off my latest attempt:
~ % log stream --style syslog --predicate 'senderImagePath ENDSWITH "logger"'
Filtering the log data using "senderImagePath ENDSWITH "logger""
Timestamp                       (process)[PID]
2022-02-22 23:07:29.338244-0800  localhost logger[3396]: Starting Install or Defer.sh. Performing validation and error checking...
2022-02-22 23:07:29.450387-0800  localhost logger[3396]: Connection to swscan.apple.com port 443 [tcp/https] succeeded!
2022-02-22 23:07:29.563756-0800  localhost logger[3396]: Validation and error checking passed. Starting main process...
2022-02-22 23:07:29.564310-0800  localhost logger[3396]: Install button label undefined by administrator. Using default value.
2022-02-22 23:07:29.564525-0800  localhost logger[3396]: Install button label: Install
2022-02-22 23:07:29.564785-0800  localhost logger[3396]: Defer button label undefined by administrator. Using default value.
2022-02-22 23:07:29.565199-0800  localhost logger[3396]: Defer button label: Defer
2022-02-22 23:07:29.565265-0800  localhost logger[3396]: Maximum deferral time preference undefined by administrator, or not set to a positive integer. Using default value.
2022-02-22 23:07:29.569386-0800  localhost logger[3396]: Maximum deferral time: 03d:00h:00m:00s
2022-02-22 23:07:29.569536-0800  localhost logger[3396]: Messaging logo undefined by admininstrator, or not found at specified path. Using default value.
2022-02-22 23:07:29.569617-0800  localhost logger[3396]: Messaging logo: /System/Library/PreferencePanes/SoftwareUpdate.prefPane/Contents/Resources/SoftwareUpdate.icns
2022-02-22 23:07:29.569669-0800  localhost logger[3396]: Support contact: IT
2022-02-22 23:07:29.592716-0800  localhost logger[3396]: Kickstarting com.apple.softwareupdated...
2022-02-22 23:08:34.683099-0800  localhost logger[3396]: Checking for pending system updates...
2022-02-22 23:08:47.795687-0800  localhost logger[3396]: Deferral deadline: 2022-02-25 23:08:47
2022-02-22 23:08:47.797527-0800  localhost logger[3396]: Time remaining: 03d:00h:00m:00s
2022-02-22 23:08:47.826882-0800  localhost logger[3396]: Prompting to install updates now or defer...
2022-02-22 23:09:33.742204-0800  localhost logger[3396]: User clicked Install after 00d:00h:00m:46s.
2022-02-22 23:09:33.772455-0800  localhost logger[3396]: Kickstarting com.apple.softwareupdated...
2022-02-22 23:09:33.904571-0800  localhost logger[3396]: 2022-02-22 23:09:33.904 jamfHelper[3642:16254] XType: com.apple.fonts is not accessible.
2022-02-22 23:09:33.904651-0800  localhost logger[3396]: 2022-02-22 23:09:33.904 jamfHelper[3642:16254] XType: XTFontStaticRegistry is enabled.
2022-02-22 23:10:38.851072-0800  localhost logger[3396]: Installing all Apple system updates...
2022-02-22 23:10:38.851312-0800  localhost logger[3396]: System will restart as soon as the update is finished. Cleanup tasks will run on a subsequent update check.
I even ran the kickstart command manually without traction or output.
Can you please guide me in the right direction?
Thank you!

Can someone please advise if possible?

Hey, we unfortunately don't have surefire fixes for this issue at the moment, since software update reliability seems to have tanked considerably with recent macOS Big Sur and Monterey updates on MDM-managed Macs (which is everyone using this script framework). If you haven't already, please reach out to Apple Enterprise Support so they are aware how much of an impact these issues have, and hopefully that will help bring faster solutions. In the meantime, I'd recommend updating to the latest version of Install or Defer (5.0.1 at the moment; 5.0.2 will probably drop in the next few days with a minor bug fix for folks running beta macOS releases) and seeing if the fixes I've applied there make your update runs more reliable.

Has anyone tried this on intel machines with the 12.3 beta yet that is out now? We really need a solution for our org soon as we have a huge number of machines.

The last few 12.3 beta releases have indicated that they specifically address some software update stability issues so fingers crossed that this release helps. Note that you'd probably need to update to 12.3 (hopefully 11.6.5 includes this fix too) to get the fix, meaning 12.2.1-to-12.3 may still encounter issues unless Apple has a way to improve update reliability before receiving the patch.

Looks like the official 12.3 dropped today.

Yep, and 11.6.5 which will hopefully bring relief for #54. 🤞🏻

Has anyone compared Install or Defer to Nudge? What are the major differences? I would prefer to use our tried and tested Install or Defer workflow which has worked great for us in the past, but I am not sure how long our organization is willing to wait without proper enforcement. From a quick read I don't think it has any issues with Monterey and Intel machines, but from what I gather it looks like Nudge makes the user initiate the update manually via Software Update on BOTH intel and m1, and just continues to hassle the user until they do it. As opposed to Install or Defer which seems to perform things in a more truly forced / automated fashion by utilizing softwareupdated for Intel users then requiring the manual clicks for M1 only (due to apple security limitations)

For what it's worth, I intend on modifying my fork of install-or-defer this
week so that the update button also pushes users to System Preferences >
Software Update. At least then, if the update fails, the user is notified
in a more transparent way (and they aren't stuck with a floating "Updates
are being installed" window"). Perhaps at the end of the deferral window,
if the user *still *hasn't updated, I'll enforce the update using the
softwareupdate command.
On Mon, Mar 14, 2022 at 4:20 PM romnation1 @.***> wrote:

Has anyone compared Install or Defer to Nudge? What are the major
differences? I would prefer to use our tried and tested Install or Defer
workflow which has worked great for us in the past, but I am not sure how
long our organization is willing to wait without proper enforcement.
From a quick read I don't think it has any issues with Monterey and Intel
machines, but from what I gather it looks like Nudge makes the user
initiate the update manually via Software Update on BOTH intel and m1, and
just continues to hassle the user until they do it. As opposed to Install
or Defer which seems to perform things in a more truly forced / automated
fashion by utilizing softwareupdated for Intel users then requiring the
manual clicks for M1 only (due to apple security limitations)
—
Reply to this email directly, view it on GitHub
https://github.com/mpanighetti/install-or-defer/issues/76#issuecomment-1067395388,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHRVCXIZHX2ACTL454QUWSLU77CSNANCNFSM5OFO6FZQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

Has anyone compared Install or Defer to Nudge? What are the major differences?

From what I understand, Nudge is a great tool, but it has no hard enforcement mechanisms (it runs a persistent alert to get folks to apply updates on their own, similar conceptually to the Apple Silicon workflow for this framework but built fully around that nudging workflow), so it won't specifically address your organization's concerns about enforced update timelines.

For what it's worth, I intend on modifying my fork of install-or-defer this week so that the update button also pushes users to System Preferences > Software Update. At least then, if the update fails, the user is notified in a more transparent way (and they aren't stuck with a floating "Updates are being installed" window"). Perhaps at the end of the deferral window, if the user *still *hasn't updated, I'll enforce the update using the softwareupdate command.

Pull requests are welcome if you have thoughts on improving the Install or Defer user experience. Let me know how your tests go! 😁

12.3.1 dropped yesterday. Has anyone had a chance to test and see if updates on Intel machines are working properly now?

I tested this morning on my 12.3 intel machine (attempting to update to 12.3.1) and am still having the same problem unfortunately.

Seeing this same issue on some of our Intel machines - Apple Silicon machines are working as intended. Thought it was something with softwareupdate hanging and tried running - sudo launchctl kickstart -k system/com.apple.softwareupdated but no luck on the machines in question.

TBH I think softwareupdate is a dead method to install updates :( Apple are shifting the process to MDM commands and I'm already testing a workflow of using Jamf APIs to trigger the update.

Apple is no longer supporting macOS Monterey beyond security patches, and they recommend moving to the current macOS release to receive bug fixes related to software updates. Going to close this out, but if you encounter similar issues in macOS Sonoma, please submit a new issue for tracking.


Applinks will not work until the server side file is missing.

#Problem with Applinks and Rapid-Tests
I really want to add a button on the booking page for adding a rapid test to the CWA. This is important if someone is booking the test online with its smartphone. He usually cannot print the code and scan it afterwards with the CWA.
But clicking on the link opens the Appstore and not the app.
Avoid duplicates

[x] Bug is not mentioned in the FAQ
[x] Bug is specific for iOS only, for general issues / questions that apply to iOS and Android please raise them in the documentation repository
[x] Bug is not already reported in another issue

Technical details

Device name: All
iOS version: 14.5.1
App version: 2.2.1

Describe the bug
The application seems to correctly define the following domain s.coronawarn.app as Accociated Domain. But unfortunately this will not work until there is no server-side file called apple-app-site-association accessible via https://s.coronawarn.app/apple-app-site-association. I'm not sure why this is missing. It might be on purpose (due to a bug in the app) or by mistake. I don't know, but it should be fixed asap as it breaks the user experience if you receive the QR-Code on your mobile device.
Steps to reproduce the issue
Please use a QR-Code and extract the URL out of it. Open the URL in the Browser.
Current behavior: The Appstore opens ...
Expected behaviour
CWA should open and import the rapid test data.
Possible Fix
Add the required file apple-app-site-association on the web-site that is used for s.coronawarn.app. That's it.
I don't know whether the app is capable of loading the data. But the code looks capable of doing so.
Additional context
Our test center does not allow bookings on site. We expect that bookings will be done by the costomer online. Therefore we do not provide the QR-Code when the user enters the test center. We provide the QR-Code via mail if the test result is uploaded to the CWA-Backend. Users of mobile phone currently have to print the QR-Code and scan it afterwards what is a big bummer.

@eilers Thanks for reporting. We will come back to you. Internal Tracking ID: EXPOSUREAPP-7379
Best wishes, DS

Corona-Warn-App Open Source Team

@eilers Did you take a look at https://github.com/corona-warn-app/cwa-event-landingpage/tree/main/src/well-known?
There is a file in there called apple-app-site-association.

@dsarkar Let's transfer to either documentation or event-landingpage repository since it affects all devices, not only iOS.

The file apple-app-site-association is located in the .well-known
The issue is Accociated Domain works only with Safari and not for exampel duckduckgo Browser.
Also if you open in tab the last link Safari will save this settings then you have to long press on the link and select "Öffnen in Corona-Warn"


Add Norwegian Bokmål

I hope I am not being rude, but I added Norwegian Bokmål to AnyPicker. Please add it if you want to.

Thank you  for contributing @pusle.


Cannot Install NewsPlease on fresh Ubuntu LTS

Mandatory

[X] I read the documentation (readme and wiki).
[X] I searched other issues (including closed issues) and could not find any to be related. If you find related issues post them below or directly add your issue to the most related one.

Describe the bug
cffi updates have broken the library? No error code printed, NewsPlease silently fails.
NewsPlease.from_url('any_url_here', timeout=30)
Will always return None.
To Reproduce
From a brand new Ubuntu 18.04.04 LTS install, if you install pip3, python3, and run pip3 install news-please, it installs with no issues.  However, above statement will always return None without an Exception thrown.
Further investigation shows that reinstalling news-please will cause a segmentation fault within pip3.
Error occurs upon installation of cffi, this is likely the broken dependency that causes this to fail.
Expected behavior
If cffi fails, would expect an exception being thrown rather than returning None for article contents.
Log
Collecting news-please
Collecting lxml>=3.3.5 (from news-please)
  Using cached https://files.pythonhosted.org/packages/dd/ba/a0e6866057fc0bbd17192925c1d63a3b85cf522965de9bc02364d08e5b84/lxml-4.5.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting bs4 (from news-please)
Collecting PyDispatcher>=2.0.5 (from news-please)
Collecting Scrapy>=1.1.0 (from news-please)
  Using cached https://files.pythonhosted.org/packages/3b/e4/69b87d7827abf03dea2ea984230d50f347b00a7a3897bc93f6ec3dafa494/Scrapy-1.8.0-py2.py3-none-any.whl
Collecting awscli>=1.11.117 (from news-please)
  Using cached https://files.pythonhosted.org/packages/b8/40/0d8bf5bbc9910ba5312b55f5962f76681387ecf296463fb9d96f6f68a516/awscli-1.18.3-py2.py3-none-any.whl
Collecting beautifulsoup4>=4.3.2 (from news-please)
  Using cached https://files.pythonhosted.org/packages/cb/a1/c698cf319e9cfed6b17376281bd0efc6bfc8465698f54170ef60a485ab5d/beautifulsoup4-4.8.2-py3-none-any.whl
Collecting PyMySQL>=0.7.9 (from news-please)
  Using cached https://files.pythonhosted.org/packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl
Collecting hjson>=1.5.8 (from news-please)
Collecting plac>=0.9.6 (from news-please)
  Using cached https://files.pythonhosted.org/packages/86/85/40b8f66c2dd8f4fd9f09d59b22720cffecf1331e788b8a0cab5bafb353d1/plac-1.1.3-py2.py3-none-any.whl
Collecting newspaper3k>=0.2.8 (from news-please)
  Using cached https://files.pythonhosted.org/packages/d7/b9/51afecb35bb61b188a4b44868001de348a0e8134b4dfa00ffc191567c4b9/newspaper3k-0.2.8-py3-none-any.whl
Collecting elasticsearch>=2.4 (from news-please)
  Using cached https://files.pythonhosted.org/packages/10/60/0c79dde3e81beffeed422599d9ac65419289095186d37a3201739d52a57d/elasticsearch-7.5.1-py2.py3-none-any.whl
Collecting warcio>=1.3.3 (from news-please)
  Using cached https://files.pythonhosted.org/packages/90/c4/86bc02bc3bc33c34ab24e52af8a1c34eb6e03e7cd5b3904057ebcea311da/warcio-1.7.1-py2.py3-none-any.whl
Collecting readability-lxml>=0.6.2 (from news-please)
Collecting python-dateutil>=2.4.0 (from news-please)
  Using cached https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl
Collecting six>=1.10.0 (from news-please)
  Using cached https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
Collecting hurry.filesize>=0.9 (from news-please)
Collecting langdetect>=1.0.7 (from news-please)
Collecting dotmap>=1.2.17 (from news-please)
  Using cached https://files.pythonhosted.org/packages/41/64/a63c863b674b3ce90af32632a1bec59e8b0d64c5afa9782ab7e5a5b6b33e/dotmap-1.3.13-py3-none-any.whl
Collecting ago>=0.0.9 (from news-please)
Collecting pyOpenSSL>=16.2.0 (from Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/9e/de/f8342b68fa9e981d348039954657bdf681b2ab93de27443be51865ffa310/pyOpenSSL-19.1.0-py2.py3-none-any.whl
Collecting cssselect>=0.9.1 (from Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/3b/d4/3b5c17f00cce85b9a1e6f91096e1cc8e8ede2e1be8e96b87ce1ed09e92c5/cssselect-1.1.0-py2.py3-none-any.whl
Collecting parsel>=1.5.0 (from Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/86/c8/fc5a2f9376066905dfcca334da2a25842aedfda142c0424722e7c497798b/parsel-1.5.2-py2.py3-none-any.whl
Collecting zope.interface>=4.1.3 (from Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/16/1c/d9e4d1e4eb9777ae675c5ac01290e70012498944d5e743bd2777d1096ad7/zope.interface-4.7.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting service-identity>=16.0.0 (from Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/e9/7c/2195b890023e098f9618d43ebc337d83c8b38d414326685339eb024db2f6/service_identity-18.1.0-py2.py3-none-any.whl
Collecting queuelib>=1.4.2 (from Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/4c/85/ae64e9145f39dd6d14f8af3fa809a270ef3729f3b90b3c0cf5aa242ab0d4/queuelib-1.5.0-py2.py3-none-any.whl
Collecting w3lib>=1.17.0 (from Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/6a/45/1ba17c50a0bb16bd950c9c2b92ec60d40c8ebda9f3371ae4230c437120b6/w3lib-1.21.0-py2.py3-none-any.whl
Collecting protego>=0.1.15 (from Scrapy>=1.1.0->news-please)
Collecting cryptography>=2.0 (from Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/45/73/d18a8884de8bffdcda475728008b5b13be7fbef40a2acc81a0d5d524175d/cryptography-2.8-cp34-abi3-manylinux1_x86_64.whl
Collecting Twisted>=17.9.0; python_version >= "3.5" (from Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/88/e2/0c21fadf0dff02d145db02f24a6ed2c24993e7242d138babbca41de2f5a2/Twisted-19.10.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting docutils<0.16,>=0.10 (from awscli>=1.11.117->news-please)
  Using cached https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl
Collecting colorama<0.4.4,>=0.2.5; python_version != "3.4" (from awscli>=1.11.117->news-please)
  Using cached https://files.pythonhosted.org/packages/c9/dc/45cdef1b4d119eb96316b3117e6d5708a08029992b2fee2c143c7a0a5cc5/colorama-0.4.3-py2.py3-none-any.whl
Collecting rsa<=3.5.0,>=3.1.2 (from awscli>=1.11.117->news-please)
  Using cached https://files.pythonhosted.org/packages/e1/ae/baedc9cb175552e95f3395c43055a6a5e125ae4d48a1d7a924baca83e92e/rsa-3.4.2-py2.py3-none-any.whl
Collecting s3transfer<0.4.0,>=0.3.0 (from awscli>=1.11.117->news-please)
  Using cached https://files.pythonhosted.org/packages/69/79/e6afb3d8b0b4e96cefbdc690f741d7dd24547ff1f94240c997a26fa908d3/s3transfer-0.3.3-py2.py3-none-any.whl
Collecting botocore==1.15.3 (from awscli>=1.11.117->news-please)
  Using cached https://files.pythonhosted.org/packages/6f/c7/4a7174b08e3cf644782641325db9bbe5fb5087c21d92486e5a7fd7ad4c2e/botocore-1.15.3-py2.py3-none-any.whl
Collecting PyYAML<5.3,>=3.10 (from awscli>=1.11.117->news-please)
Collecting soupsieve>=1.2 (from beautifulsoup4>=4.3.2->news-please)
  Using cached https://files.pythonhosted.org/packages/81/94/03c0f04471fc245d08d0a99f7946ac228ca98da4fa75796c507f61e688c2/soupsieve-1.9.5-py2.py3-none-any.whl
Collecting jieba3k>=0.35.1 (from newspaper3k>=0.2.8->news-please)
Collecting nltk>=3.2.1 (from newspaper3k>=0.2.8->news-please)
Collecting feedparser>=5.2.1 (from newspaper3k>=0.2.8->news-please)
Collecting Pillow>=3.3.0 (from newspaper3k>=0.2.8->news-please)
  Using cached https://files.pythonhosted.org/packages/19/5e/23dcc0ce3cc2abe92efd3cd61d764bee6ccdf1b667a1fb566f45dc249953/Pillow-7.0.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting tldextract>=2.0.1 (from newspaper3k>=0.2.8->news-please)
  Using cached https://files.pythonhosted.org/packages/fd/0e/9ab599d6e78f0340bb1d1e28ddeacb38c8bb7f91a1b0eae9a24e9603782f/tldextract-2.2.2-py2.py3-none-any.whl
Collecting tinysegmenter==0.3 (from newspaper3k>=0.2.8->news-please)
Collecting requests>=2.10.0 (from newspaper3k>=0.2.8->news-please)
  Using cached https://files.pythonhosted.org/packages/1a/70/1935c770cb3be6e3a8b78ced23d7e0f3b187f5cbfab4749523ed65d7c9b1/requests-2.23.0-py2.py3-none-any.whl
Collecting feedfinder2>=0.0.4 (from newspaper3k>=0.2.8->news-please)
Collecting urllib3>=1.21.1 (from elasticsearch>=2.4->news-please)
  Using cached https://files.pythonhosted.org/packages/e8/74/6e4f91745020f967d09332bb2b8b9b10090957334692eb88ea4afe91b77f/urllib3-1.25.8-py2.py3-none-any.whl
Collecting chardet (from readability-lxml>=0.6.2->news-please)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting setuptools (from hurry.filesize>=0.9->news-please)
  Using cached https://files.pythonhosted.org/packages/3d/72/1c1498c1e908e0562b1e1cd30012580baa7d33b5b0ffdbeb5fde2462cc71/setuptools-45.2.0-py3-none-any.whl
Collecting pyasn1 (from service-identity>=16.0.0->Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl
Collecting attrs>=16.0.0 (from service-identity>=16.0.0->Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/a2/db/4313ab3be961f7a763066401fb77f7748373b6094076ae2bda2806988af6/attrs-19.3.0-py2.py3-none-any.whl
Collecting pyasn1-modules (from service-identity>=16.0.0->Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/95/de/214830a981892a3e286c3794f41ae67a4495df1108c3da8a9f62159b9a9d/pyasn1_modules-0.2.8-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8 (from cryptography>=2.0->Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/f1/c7/72abda280893609e1ddfff90f8064568bd8bcb2c1770a9d5bb5edb2d1fea/cffi-1.14.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting constantly>=15.1 (from Twisted>=17.9.0; python_version >= "3.5"->Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/b9/65/48c1909d0c0aeae6c10213340ce682db01b48ea900a7d9fce7a7910ff318/constantly-15.1.0-py2.py3-none-any.whl
Collecting PyHamcrest>=1.9.0 (from Twisted>=17.9.0; python_version >= "3.5"->Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/ac/6c/a641af18e416e6501c10b03742387176626a1d48196100160df796f36632/PyHamcrest-2.0.0-py3-none-any.whl
Collecting Automat>=0.3.0 (from Twisted>=17.9.0; python_version >= "3.5"->Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/dd/83/5f6f3c1a562674d65efc320257bdc0873ec53147835aeef7762fe7585273/Automat-20.2.0-py2.py3-none-any.whl
Collecting incremental>=16.10.1 (from Twisted>=17.9.0; python_version >= "3.5"->Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/f5/1d/c98a587dc06e107115cf4a58b49de20b19222c83d75335a192052af4c4b7/incremental-17.5.0-py2.py3-none-any.whl
Collecting hyperlink>=17.1.1 (from Twisted>=17.9.0; python_version >= "3.5"->Scrapy>=1.1.0->news-please)
  Using cached https://files.pythonhosted.org/packages/7f/91/e916ca10a2de1cb7101a9b24da546fb90ee14629e23160086cf3361c4fb8/hyperlink-19.0.0-py2.py3-none-any.whl
Collecting jmespath<1.0.0,>=0.7.1 (from botocore==1.15.3->awscli>=1.11.117->news-please)
  Using cached https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting idna (from tldextract>=2.0.1->newspaper3k>=0.2.8->news-please)
  Using cached https://files.pythonhosted.org/packages/89/e3/afebe61c546d18fb1709a61bee788254b40e736cff7271c7de5de2dc4128/idna-2.9-py2.py3-none-any.whl
Collecting requests-file>=1.4 (from tldextract>=2.0.1->newspaper3k>=0.2.8->news-please)
  Using cached https://files.pythonhosted.org/packages/23/9c/6e63c23c39e53d3df41c77a3d05a49a42c4e1383a6d2a5e3233161b89dbf/requests_file-1.4.3-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests>=2.10.0->newspaper3k>=0.2.8->news-please)
  Using cached https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography>=2.0->Scrapy>=1.1.0->news-please)
Installing collected packages: lxml, soupsieve, beautifulsoup4, bs4, PyDispatcher, six, pycparser, cffi, cryptography, pyOpenSSL, cssselect, w3lib, parsel, setuptools, zope.interface, pyasn1, attrs, pyasn1-modules, service-identity, queuelib, protego, constantly, PyHamcrest, Automat, incremental, idna, hyperlink, Twisted, Scrapy, docutils, colorama, rsa, jmespath, urllib3, python-dateutil, botocore, s3transfer, PyYAML, awscli, PyMySQL, hjson, plac, jieba3k, nltk, feedparser, Pillow, certifi, chardet, requests, requests-file, tldextract, tinysegmenter, feedfinder2, newspaper3k, elasticsearch, warcio, readability-lxml, hurry.filesize, langdetect, dotmap, ago, news-please
Segmentation fault (core dumped)

Versions (please complete the following information):

OS: Ubuntu 18.04.04 (Brand new install)
Python Version 3.6
news-please Version >= 1.2.

Intent (optional; we'll use this info to prioritize upcoming tasks to work on)

[ ] personal
[ ] academic
[X] business
[ ] other
Some information on your project:

Confirmed that forcing
news-please==1.4.24
and
cffi==1.13.2
makes the library work as intended.

Thanks for the issue report. Do you mean that cffi could not be installed correctly and thus newsplease fails when running NewsPlease.from_url('any_url_here', timeout=30).


"Birda", "Berecuța", "Mânăstire", "Sângeorge" are listed in the-drop-down menu

Description:
The villages are listed in the drop-down menu alongside its corresponding county and commune, according to the law
Preconditions:
The website (https://peviitor-ro.github.io/ui.orase/) is up and runing. After every step delete the text you typed from the search bar.

Step 1
Write in the search bar "Birda".

Expected results
The location is listed in the drop-down menu as "Sat Birda, TIMIȘ(Birda) ".

Step 2
Write in the search bar "Berecuța".

Expected results
The location is listed in the drop-down menu as "Sat Berecuța, TIMIȘ(Birda) ".

Step 3
Write in the search bar "Mânăstire".

Expected results
The location is listed in the drop-down menu as "Sat Mânăstire, TIMIȘ(Birda) ".

Step 4
Write in the search bar "Sângeorge".

Expected results
The location is listed in the drop-down menu as "Sat Sângeorge, TIMIȘ(Birda) ".


585 user cache data reformatting

Here's to code that attempts to execute the construct query and write to the fcrepo format:
https://github.com/ucd-library/aggie-experts/blob/006cee75cbf1f468da9fdb67c4eb30c03aff9784/harvest/experts-client/lib/cache/index.js#L565

@rakunkel-ucd I've started to make progress.  This last check in produces pretty good experts and authorships. though there are a few  errors still (like the label), and I haven't added the grants yet.


Cannot log in with Github

As in title

Uncaught (in promise) FirebaseError: Missing or insufficient permissions.
    at new e (https://oasis.sh/_next/static/chunks/ec5c9526.b00002c8385ee69f830c.js:1:1774)
    at https://oasis.sh/_next/static/chunks/ec5c9526.b00002c8385ee69f830c.js:1:152171
    at Ze.<anonymous> (https://oasis.sh/_next/static/chunks/ec5c9526.b00002c8385ee69f830c.js:1:151384)
    at It (https://oasis.sh/_next/static/chunks/483978de5ae00bb23ca9ff891b37fa73fa4376d2.af1fa64117069ccb745f.js:1:10633)
    at Ct (https://oasis.sh/_next/static/chunks/483978de5ae00bb23ca9ff891b37fa73fa4376d2.af1fa64117069ccb745f.js:1:10372)
    at ni.EeOn.ni.sa (https://oasis.sh/_next/static/chunks/483978de5ae00bb23ca9ff891b37fa73fa4376d2.af1fa64117069ccb745f.js:1:39289)
    at xn (https://oasis.sh/_next/static/chunks/483978de5ae00bb23ca9ff891b37fa73fa4376d2.af1fa64117069ccb745f.js:1:18899)
    at En (https://oasis.sh/_next/static/chunks/483978de5ae00bb23ca9ff891b37fa73fa4376d2.af1fa64117069ccb745f.js:1:16716)
    at dn.EeOn.o.Ca (https://oasis.sh/_next/static/chunks/483978de5ae00bb23ca9ff891b37fa73fa4376d2.af1fa64117069ccb745f.js:1:20957)
    at dn.EeOn.o.Xa (https://oasis.sh/_next/static/chunks/483978de5ae00bb23ca9ff891b37fa73fa4376d2.af1fa64117069ccb745f.js:1:20251)

Hey, this has been fixed. It was an issue with our security rules via Firebase, thanks for spotting this bug! However, even after logging in you won't see anything yet, as we are still working on the redesign and features, stay tuned!


Task Control Codes

I was using hugging faces  codet5-base  to try code generation and understanding tasks.
I  cannot find documentation anywhere that indicates how to use "task control codes" for different input types.
I am trying to do something like Figure 1 in the  original paper

Hi there, the codet5-base checkpoint is the one trained with the proposed identifier-aware denoising pretraining objective, while "task control codes" are only used for the multi-task finetuning. So you need to finetune CodeT5-base checkpoint on downstream tasks or directly use our released checkpoints.


Remove integer constraint

Refactored some files.
Changed Invoice class to accept float numbers.
Changed toJson method of RedirectionForm to return string like before.

according to the PR title (Remove integer constraint), this doesn't have atomic commits.
I close this PR. please fix it and then let me know by another PR.


Dependent resources

In my SSR app, I have a resource, categories, for retrieving a list of categories from the server. I also create another resource, items, for retrieving all items in a category by its id. In order to get the id of a category by its URL slug (which is a route param), my items resource has to depend on categories. I haven't found any way to make a resource either depend on or wait for another one. If I do something like:
let category_index = create_memo(cx, move |_| {
    categories.with(cx, |categories| {
        params.with(|params| {
            let slug = &params.as_ref().unwrap().category;

            categories
                .iter()
                .position(|category| slugify(&category.name) == *slug)
                .unwrap()
        })
    })
});

let items = create_resource(
    cx,
    move || {
        categories.with(cx, |categories| {
            category_index().map(|category| categories[category].id)
        })
    },
    move |id| async move { Some(search_items(cx, id??, None).await.unwrap()) },
);

I get an error about reading categories outside a <Suspense>. But if I move reading from categories from getter to fetcher, the items resource obviously runs once, doesn't get categories and doesn't ever refetch. Any thoughts on how to tackle this? Thanks in advance.

I get an error about reading categories outside a <Suspense>.

I think you get a warning, not an error, right? Does it work as expected? If so, we can work on how to resolve the warning so things work correctly. If not, do you get other hydrating warnings etc.?
It's semi-intentionally difficult for a resource to depend on another resource, as this kind of async-request waterfall is usually a bad idea. I'm assuming you're doing something on the server like reading this data from a database... In which case, why doesn't search_items just take the category slug as an argument instead of the ID? It's usuall better to do two reads from the database (to look up category info, then to look up items) on the server in a single request than to do it in two separate requests. But maybe I'm missing some context.

Sure it was a warning, but the things didn't work at all, otherwise I wouldn't even bother. I believe there was a hydration warning as well. I can create a minimal repro when I have some free time.
As for why I don't just pass slug to search_items: categories are actually global shared state that I pass down via context and use in many places, e.g. in my  component on every page. So in this case I can't quite agree that fetching categories in different server fns is a better approach. Also imagine having to display a <Select items=... current_index=...> component on that page, to which we pass the category list and current category index. So we could return categories from search_items alongside the items we need, but should we, actually? Not only does it overfetch, but also looks like a not so good design in general.

Here's a similar example that I just put together — this one is client-side only, but works perfectly as far as the interaction between the resources. I'll try to put together a client-server example a little later to see what's broken.

async fn categories() -> Vec<String> {
    TimeoutFuture::new(1_000).await;
    (0..10).map(|n| format!("post-{n}")).collect()
}

async fn post(id: usize) -> Option<String> {
    TimeoutFuture::new(2_000).await;
    if id < 10 {
        Some(format!("Post #{id}"))
    } else {
        None
    }
}

#[component]
pub fn App(cx: Scope) -> impl IntoView {
    let (id, set_id) = create_signal(cx, 0);
    let slug = create_memo(cx, move |_| format!("post-{}", id()));
    let categories = create_resource(cx, || (), |_| categories());
    let category_index = create_memo(cx, move |_| {
        categories.with(cx, |categories| {
            slug.with(|slug| {
                categories
                    .iter()
                    .position(|category| category == slug)
                    .unwrap()
            })
        })
    });

    let post = create_resource(
        cx,
        move || {
            categories.with(cx, |categories| {
                category_index()
            })
            .flatten()
        },
        move |id| async move { Some(post(id.unwrap_or(0)).await.unwrap()) },
    );

    view! { cx,

            <button on:click=move |_| set_id.update(|n| *n -= 1)>"-1"</button>
            "slug: "
            {slug}
            <button on:click=move |_| set_id.update(|n| *n += 1)>"+1"</button>

            {move || {
                categories
                    .with(
                        cx,
                        |categories| {
                            view! { cx,
{format!("{:#?}", categories)}
 }
                        },
                    )
            }}

            {move || {
                post
                    .with(
                        cx,
                        |post| {
                            view! { cx,
{format!("{:#?}", post)}
 }
                        },
                    )
            }}

    }
}

And actually thinking through it gives me a good idea: if you literally don't want to start reading the second resource until the first one is resolved, the second can be created in a component that's under a <Suspense/> that includes reading the first one. That's the way to express "defer doing this work until this resource is available."

So because I thought this was an interesting problem, I built a little example that does more or less what you're hoping for, I think. It's kind of a kludgy pattern, so I'm open to suggestions for APIs that would make stuff like this easier in the future, but here's a working example, in case it's helpful to you or to anyone else who reads this issue in the future!
use leptos::*;
 use async_timer::oneshot::{Oneshot, Timer};

#[server(Categories, "/api")]
async fn categories() -> Result<Vec<String>, ServerFnError> {
    Timer::new(std::time::Duration::from_secs(1)).await;
    Ok((0..10).map(|n| format!("post-{n}")).collect())
}

#[server(Post, "/api")]
async fn post(id: usize) -> Result<Option<String>, ServerFnError> {
    Timer::new(std::time::Duration::from_secs(2)).await;
    if id < 10 {
        Ok(Some(format!("Post #{id}")))
    } else {
        Ok(None)
    }
}

#[component]
pub fn App(cx: Scope) -> impl IntoView {
    let (id, set_id) = create_signal(cx, 0);
    let slug = create_memo(cx, move |_| format!("post-{}", id()));
    let categories = create_resource(cx, || (), |_| categories());

    let category_index = move |cx| create_memo(cx, move |_| {
        categories.with(cx, |categories| {
            slug.with(|slug| {
                categories
                    .as_ref()
                    .map(|categories| {
                        categories.iter()
                            .position(|category| category == slug)
                            .unwrap_or(0)
                    })
                    .unwrap_or(0)
            })
        })
    });

    view! { cx,

            <button on:click=move |_| set_id.update(|n| *n -= 1)>"-1"</button>
            "slug: "
            {slug}
            <button on:click=move |_| set_id.update(|n| *n += 1)>"+1"</button>

        <Suspense fallback=move || { view! { cx,
"Loading categories..."
 }}>

                {move || {
                    categories
                        .with(
                            cx,
                            |categories| {
                                view! { cx,
{format!("{:#?}", categories)}
 }
                            },
                        )
                }}

        </Suspense>
        <Post category_index=category_index(cx)/>
    }
}

#[component]
fn Post(cx: Scope, category_index: Memo<Option<usize>>) -> impl IntoView {
    let post = create_resource(
        cx,
        category_index,
        move |id| async move { Some(post(id.unwrap_or(0)).await.unwrap()) },
    );

    view! { cx,
        <Suspense fallback=move || view! { cx,
"Loading post..."
 }>
            {move || post.with(cx, |post| view! { cx,

{format!("{:#?}", post)}

            })}
        </Suspense>
    }
}


"rastertopwg" seems to produce output not liked by Brother MFC-J430W

Brother MFC-J430W is a printer which support AirPrint.
The system is Ubuntu 20.04, running cups-snap in parallel mode (although the problem also happens on system's cups so it's probably not the cause).
At one point, error_log says:
D [02/Jul/2021:01:43:19 +0700] [Job 3] Using following CUPS filter chain to convert input data to the APPLE Raster format: gziptoany pdftoraster rastertopwg

And after a while, it says:
D [02/Jul/2021:01:43:23 +0700] [Job 3] JOBSTATE: document-format-error

Which make the rastertopwg my suspect.
The full error_log is attached, along with $SNAP/var/spool (in a .tar.gz).

@peat-psuwit, and everyone else, could you please update cups-filters to the current GIT master state? There were bugs for printing to Raster printers via cups-browsed which should have been fixed now.
Main problem was that on Raster jobs the data was not sent with the correct color space and depth. In addition there were also problems with page geometry/margins. I have done several fixes in the last few weeks (last update yesterday). So please retry with the newest GIT snapshot.
@michaelrsweet, the calling of CUPS filters (now filter functions) by the implicitclass backend is to allow clusters of printers which use different input formats. The cluster CUPS queue generated by cups-browsed creates a PPD with application/vnd.cups-pdf as data format (emulation of PDF printer) and the backend, once it got told the destination printer's URI, converts the PDF into the printer's format.

@tillkamppeter Could you please advice on how I should test the git master of cups-filter? I was using Snap version of cups before, but according to snapcraft.yaml that doesn't use the git version of cups-filter.

@peat-psuwit how do you use the CUPS Snap? Did you simply grab it from the Snap Store or did you build it from source?

@peat-psuwit I have updated the CUPS Snap to cups-filters 2.x now. The Snap Store's build service is currently building it and in ~1 hour (most probably earlier) it will be available and an auto-update of it on your system should take place, but only if the CUPS Snap you have installed is from the Snap Store.
If you have built the Snap from source by yourself, you need to git pull and rebuild right now.
Please tell whether this solves your problem.

Ok, this is a weird one: with cups-snap version 596 from snap store, the printer now prints, but in black-and-white. And after a while the printer won't print anything until I restart both the printer and CUPS.
Will collect the log later, after I try to reproduce won't-print state.

Thanks for testing
Does the printer print in color if you explicitly request color when sending the job (to the lp command line add -o print-color-mode=color)? This would mean that cups-browsed does not set the defaults correctly when creating a print queue.
As the CUPS Snap is still under heavy development it is configured by default to do logging in debug mode. You find the logs in /var/snap/cups/current/var/log/. Most important is error_log containing the logging of both succeeding and failing jobs, especially also of the last succeeding and the first failing job. cups-browsed_log would be of help for the default setting problem.

If I added -o print-color-mode=color to the lp command line, then yes the printer would print in color. Unfortunately I can't reproduce the state where the printer won't print, but that I feel it's more like printer firmware's bug.
Just to be thorough, I've included all the log in /var/snap/cups/current/var/log/ in a zip file. The last 2 jobs are the same file printed with, follow by without, -o print-color-mode=color.
cups-snap_logs_2022-01-15.zip
Although I'm not sure now if this should continue in this issue. Feel free to close this issue, and I'll open a new one about the printer defaulting to black-and-white.

I have already found out what causes the problem of defaulting to monochrome. The PPD generated by cups-browsed does not only contain color and grayscale settings but also an "Auto" setting which is set to the default (resembling print-color-mode-supported and print-color-mode-default printer IPP attributes), but CUPS, at least with its most resent commit (note also that the CUPS Snap uses the GIT master of CUPS), does not support "Auto" as color mode and ends up considering this as monochrome. I have to check here whether the correct fix is making CUPS supporting "Auto" or cups-browsed not generating PPD files with "Auto" for the color mode.

@tillkamppeter The PPD shouldn't have an "auto" option - generally speaking, "auto" maps to "color" for color pages and "monochrome" for B&W pages, which is something that the PDF RIP can automatically detect and use the correct color/grayscale colorspace when "color" is the default (i.e. optimize for the most common usage and don't add a separate option to enable it).

I have fixed the color problem now. The mentioned commit does not only set the default to color if a color choice is present, but also ignore the auto choice in the print-color-mode-supported IPP attribute, only adding the other choices to ColorModel.

In another commit I will also take into account print-color-mode-default like in OpenPrinting/cups@98ac8653eb3 .

As now the color problem is solved (thanks @michaelrsweet) and the problem of printing ceasing to work after a certain amount of successful jobs not being reproducible I am closing this issue here.

OpenPrinting/cups@98ac865 is ported to the PPD generator of cups-filters now (commit 99d1cc62) and I have triggered a rebuild of the CUPS Snap, now the color mode defaults should be always correct.


KAFKA-6035: Avoid creating changelog topics for state stores that are directly piped to a sink topic

This PR aims to optimize topology by eliminating state stores that are directly piped to a sink topic.
There few things to note:

If topology sink operator is toStream, then we consider the operator just before toStream and check if it has state store.

Just removing the state store from storeToChangelogTopic can cause a problem in case of failure, especially with EoS. So, what we propose here is that if the sink operator contains state store, then i) we remove selected state stores from storeToChangelogTopic and put it into stateStoreToChangelogTopicOnlyForRestoring.  The first step will make sure that selected state stores will not write to changelog topic. The second step is only needed when the state store reads from changelog topic (say, in case of failure).

Closing this stale PR due to inactivity. We still want this improvement, but the PR would need to be updated to the latest code base.


Fix svcat create class error looks wierd

This PR is a

[ ] Feature Implementation
[X] Bug Fix
[ ] Documentation

What this PR does / why we need it:
Which issue(s) this PR fixes

Fixes #2396
Please leave this checklist in the PR comment so that maintainers can ensure a good PR.
Merge Checklist:

[ ] New feature

[ ] Tests
[ ] Documentation

[ ] SVCat CLI flag
[ ] Server Flag for config

[ ] Chart changes
[ ] removing a flag by marking deprecated and hiding to avoid
breaking the chart release and existing clients who provide a
flag that will get an error when they try to update

/ok-to-test

make verify fails with:
pkg/svcat/service-catalog/class.go:147:11: if block ends with a return statement, so drop this else and outdent its block

IMHO it's a stupid rule, but...

@carolynvs The unit tests were added. Can you please help review again?

@shouhong Can you also add tests for create class?

@luksa The create class tests were added. Can you please review?

/lgtm

/approve


Wrap /company-signup/ page in a feature flag

We are moving this page into wagtail! By default, this PR doesn't change anything. Enable the WAGTAIL_COMPANY_SIGNUP flag to disable the existing company-signup page, and allow a wagtail page with that URL to shine through.
Additions

WAGTAIL_COMPANY_SIGNUP

Changes

urls.py changes to accomodate the new flag

Todos

After we've switched this flag permanently , we should remove the 'selfregistration' app.

Checklist

[ ] PR has an informative and human-readable title
[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the development playbook
[ ] Passes all existing automated tests
[ ] Any change in functionality is tested
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged
[ ] Visually tested in supported browsers and devices
[ ] Project documentation has been updated
[ ] Reviewers requested with the Reviewer tool :arrow_right:

I'd like to proceed, even thought it doesn't hit the diff target for codecov-- I don't think there's a good way to test code in a block that depends on an optional app being in INSTALLED_APPS.


Range Value binding dos not change displayed range

Hey Onsen Team,
The rerender with a different value say from 10 to 50 does not change the displayed range.
<Range modifier="material"                                value={this.state.duration_percentage}                                onChange={(event) => setTimeout(() => { this.setState({duration_percentage: 10}) }, 200) }                                style={{ width: '100%' }}                         />
Expected behaviour: Range is displaying 10 percent
Instead: Range displays what was selected.
I also found no hint that it is programmed to hit _onChange in the underlying javascript implementation of the compknent.

Thanks for reporting @Mythli !
We will take a look at this. So it is not updating when the state changes? That's definitely a bug so we will look into it. Maybe the value prop is not working correctly.


is v3 ESM only?

Support plan

which support plan is this issue covered by? (e.g. Community, Sponsor, or
Enterprise): community
is this issue currently blocking your project? (yes/no): no
is this issue affecting a production system? (yes/no): no

Context

node version: 18
module (formidable) version: 2
environment (e.g. node, browser, native, OS): node
used with (i.e. popular names of modules):
any other relevant information:

What problem are you trying to solve?
I am currently on v2 and I wonder wether to update to v3 or not.
My project is still using commonjs, but it seems v3 is ESM only.

Can you tell me if it is ESM only?
If yes, can you make a clear statement about it in readme? for example https://www.npmjs.com/package/file-type wrote this:

This package is a ESM package. Your project needs to be ESM too. [Read more](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c).

Do you have a new or modified API suggestion to solve the problem?
n/a

Did you read the changelog ?

Yes https://github.com/node-formidable/formidable/blob/0179abfb75b586f3115298ee221674b5cb019ab3/CHANGELOG.md?plain=1#L80
I think @GrosSacASac could have said it immediately instead, since it's quite buried in the changelog, and most users don't read it to begin with. Just answering the question immediately would save everyone a lot of time, but that's just my opinion :)

Hi,
yes, I did read the changelog, but AFAIK the fact that you internally use ESM does not imply that you don't built with commonJS support (sorry i'am not sure about it).
At least I would say it is not quite obvious, and it is not a feature, but a breaking change...

Is it a good idea to make it work both with commonjs and esmodules ?

https://github.com/node-formidable/formidable/pull/944

Published<EMAIL_ADDRESS>

Thank you!
I guess I have to move on 3.5+ now ;-)


tweepy.errors.Unauthorized: 401 Unauthorized 89 - Invalid or expired token.

I have an API version2 of the following keys:
consumer_key,consumer_secret_key,access_token,access_token_secret
and I'm having this error:
tweepy.errors.Unauthorized: 401 Unauthorized
89 - Invalid or expired token.
I actually made system time to UTF. And I don't know why this is happening!
Do I have to do something more for the time?
Could you please help me?
The following code is what I'm currently working on
auth = tweepy.OAuth1UserHandler(consumer_key,consumer_secret_key)
auth.set_access_token(access_token,access_token_secret)
api = tweepy.API(auth)
print(api)
def twitter_search(outfilename, keyword):
result = []
for i in range(1,4):
    tweets = api.search_tweets(keyword)
    result = []
    for tweet in tweets:
        result.append([tweet.created_at, tweet.id_str, tweet.text, tweet.favorite_count,tweet.retweet_count])
    result = sorted(result, key=lambda x:-x[-2])
df = pd.DataFrame(result, columns = ['time','id','text','likey','retweet'])

print(len(result))
print(df)

#df to csv
df.to_csv(outfilename, encoding = 'utf-8-sig')

if name=="main":
outfilename = 'tweet_search.csv'
keyword = '~~~~'
twitter_search(outfilename, keyword)

Double check that your credentials are correct. Regenerate them if necessary.
For code block usage, see https://docs.github.com/en/github/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks.

I actually regenerated several times already but it still gives me the same error.
But using Bearer token was fine. It worked!
And it says I already pulled about 10,000 tweets even though I wasn't able to do it even once...
Is that because of the bearer token that I used before?
Does selecting Read and write and Direct message from User authentication settings matters? I chose that...
Not the Read and write one.

It's almost certainly an issue with the actual credentials or how you're passing them.
Bearer tokens are used with Twitter API v2 and shouldn't be relevant to API, Tweepy's interface for Twitter API v1.1.
I'm not sure where you're seeing that, but that shouldn't be relevant to this issue.
No, the error indicates your credentials are invalid.

Then should I use API v1.1 consumer_key,consumer_secret_key,access_token,access_token_secret?

API, Tweepy's interface for Twitter API v1.1, requires those credentials to authenticate with user context, yes.

I actually used Tweepy's interface for twitter API v1.1 but it always says to elevate, what should I do with that?

Assuming you mean it says Tweepy API v1.1 requires Elevated access, this is true, as the error you're likely seeing indicates.
https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api
That's unrelated to this issue though.


SparseTensor multiplication in tf.keras.layers.Dense

System information

TensorFlow version (you are using): 1.12.0
Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.
In keras.layers.Dense, tf.sparse_tensor_dense_matmul is called for SparseTensor as this.
But in tf.keras, only tf.tensordot (which can not treats SparseTensor) is called as this.
Do you have any plan to treat SparseTensor in tf.keras?
Will this change the current api? How?
We can fix it simply using tf.sparse_tensor_dense_matmul.
If needed, I can create PR.

I believe we have this feature now in TF tf.sparse.sparse_dense_matmul. Please reopen if that's not the case. Thanks!


Forecast does not work with Mobile API

System Health details
.
Checklist

[X] I have filled out the issue template to the best of my ability.
[X] This issue only contains 1 issue (if you have multiple issues, open one issue for each issue).
[X] This issue is not a duplicate issue of any previous issues..

Describe the issue
The specific piece of code is here: https://github.com/ciejer/metservice-weather/blob/7271cf16d9055e7565eb5c3ef4d9375fb133c2e6/custom_components/metservice_weather/coordinator.py#L219
Reproduction steps

Use Mobile API
Check logs / entities
Forecast is not unavailable due to an error

Debug logs
Logger: homeassistant.components.websocket_api.http.connection
Source: custom_components/metservice_weather/coordinator.py:219
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 09:57:40 (5 occurrences)
Last logged: 10:17:07

[546830180032] Error handling message: Unknown error (unknown_error) Cody Cooper from <IP_ADDRESS> (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36 Edg/<IP_ADDRESS>)
[546728372160] Error handling message: Unknown error (unknown_error) Cody Cooper from <IP_ADDRESS> (Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36 Edg/<IP_ADDRESS>)
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/decorators.py", line 26, in _handle_async_response
    await func(hass, connection, msg)
  File "/usr/src/homeassistant/homeassistant/components/weather/websocket_api.py", line 102, in ws_subscribe_forecast
    await entity.async_update_listeners({forecast_type})
  File "/usr/src/homeassistant/homeassistant/components/weather/__init__.py", line 1072, in async_update_listeners
    native_forecast_list: list[Forecast] | None = await getattr(
                                                  ^^^^^^^^^^^^^^
  File "/config/custom_components/metservice_weather/weather.py", line 155, in async_forecast_daily
    return self.forecast_daily
           ^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/metservice_weather/weather.py", line 211, in forecast_daily
    num_days = self.coordinator.get_forecast_daily("", 0)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/metservice_weather/coordinator.py", line 219, in get_forecast_daily
    all_days = self.data[RESULTS_FORECAST_DAILY]["layout"]["primary"]["slots"][
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
KeyError: 'layout'

### Diagnostics dump

_No response_

Thanks for your legendary work @codyc1515!
v0.3.0 resolves this, including shipping your contributions; note you'll need to recreate the device in 'integrations' as this fix changes what is saved by the initial config flow.


LastInsertId - user SCOPE_IDENTITY, not @@IDENTITY

@@IDENTITY can return unexpected results if an insert has side-effects which create identity values in other tables, such as firing an AFTER INSERT triggers.
Using SCOPE_IDENTITY() instead resolves this problem.
MS documentation
(This was previously discussed under #133 - the user who raised it closed the issue because SCOPE_IDENTITY went out of scope before it was called. However, @@IDENTITY and SCOPE_IDENTITY both go out of scope when a session ends, so from that perspective, this change has no functional impact.)

Can you add a test without transaction?
Please see #133

I like this change.
There are two cases that tests are needed:

since another batch is used to retrieve identity, the scope_identity system function might return null when executed without a transaction (the driver uses the same connection but a different batch);
the statements are prepared using a stored procedure when they have parameters, the scope_identity might return null when using parameters.

@dimdin - There's not quite enough information in #133 to explain why the user found that LastInsertId() didn't work as expected when used with SCOPE_IDENTITY.
@@IDENTITY and SCOPE_IDENTITY are both limited to the current connection, and will return NULL (or not) in the same way in this scenario - the only difference between them is the scope from which the identity value will be drawn.
Either approach is vulnerable to problems if LastInsertId() is run on a different connection to the one where the insert took place - I've seen this cause problems in systems where where connections are pooled.
I've added a test without a transaction as you have requested.
Please could you elaborate a bit on the scenario for the second test you requested? Are you taking about the execution of any prepared statement, or the execution of a prepared stored procedure statement specifically? i.e. (pseudocode):
statement := prepare "any_sql_statement ?, ?"
execute (statement,parameters)
LastInsertId()

or specifically
statement := prepare "exec stored_procedure ?,?"
execute (statement,parameters)
LastInsertId()

The behaviour in either context will depend on whether LastInsertId() is executed on the same connection as the prepared statement.

Thank you @edharper01
The driver is using sp_executesql to execute SQL statements with parameters (either prepared or directly executed).
To test this case just use an insert statement with parameters i.e.
Exec("insert into foo (baz) values (?)", 1)

@dimdin thanks for the clarification.
Dynamic SQL commands are executed in a different scope to the session that executes them, so when sp_executesql is used to insert to a table with an identity, SCOPE_IDENTITY will be NULL in the executing session.
@@IDENTITY will return a value, but in cases where the insert causes other tables to be updated (through replication, or trigger - as in the test cases I've created for this PR), the value of @@IDENTITY will be unexpected - it will not be the new identity inserted into the table which is the subject of the insert statement.
I can't see any solutions to this issue which are both risk-free and always correct.
The simplest fix would be to change the statement executed by LastInsertId to
select cast(ISNULL(SCOPE_IDENTITY(),@@IDENTITY) as bigint)

This would leave the results of LastInsertId no less accurate than they are now, and in some cases more accurate - but since I guess that the vast majority of insert statements executed by users of this module have parameters, the benefit over using @@IDENTITY alone would be limited.
The only completely accurate fix for parameterised statements would be to do more or less what is outlined in the last message on #133 and execute SCOPE_IDENTITY within the scope of the statement passed to sp_executesql, returning the value as a result set to be cached as a property of the current connection and returned by LastInsertId. This seems risky, because of the difficulty of guaranteeing that appending to the SQL statement and returning additional result sets won't have unexpected side effects for existing users' code.
If output parameters were supported by the package this would offer a slightly lower-risk method of implementing this approach, but in any case, this is almost certainly beyond my ability to implement safely.
I'm tempted to suggest making a breaking change and removing LastInsertId altogether, but I suspect this would be an unpopular move.
What's the preferred option of the project maintainers?

The Microsoft PHP driver (which has a similar lastInsertID) uses @@IDENTITY https://github.com/Microsoft/msphpsql/blob/a02691e277d5804c3d8b28fcdfee31e6fd2df8ab/source/pdo_sqlsrv/pdo_dbh.cpp#L31
Go postgresql driver github.com/lib/pq simply doesn't support LastInsertID (Postgres has the same issue as MS SQL Server).
I'm against using select cast(ISNULL(SCOPE_IDENTITY(),@@IDENTITY) as bigint) because SCOPE_IDENTITY is intrinsically limited to the scope of the batch and we are obtaining this value by running a new batch.
https://docs.microsoft.com/en-us/sql/t-sql/functions/scope-identity-transact-sql
Returns the last identity value inserted into an identity column in the same scope. A scope is a module: a stored procedure, trigger, function, or batch.
I'm okay either leaving it as @@IDENTITY and adding documentation in both a package comment and README of the limitations of this method, or removing it and documenting that it is gone and why in both a package comment and README. In either case we should document alternatives such as either OUTPUT clause or running select scope_identity(); within the same batch.
I would argue for keeping it for compatibility and it being often correct. I would argue against it as it is never the "correct" answer due to the extra round trip (performance) and scope issue.


Receive Null in the successCallback on iOS

When I call the launch method in iOS, I get null in the success callback response. Is this an intended behaviour or a bug?

Hello, @maxfloden
Can we chat via skype?
My Skype id is Stanko_Marchovic_910423.
Thanks

Was there any solution for this issue, because I am also struggling with it? Thank you in advance!


updated kubeflow installation script so that the helm command is idempotent

Which issue is resolved by this Pull Request:
running make deploy-kubeflow sometimes fails due to various reasons. After fixing whatever issue that cause the failure, I am unable to run the script again.
Error: INSTALLATION FAILED: cannot re-use a name that is still in use
Traceback (most recent call last):
  File "utils/kubeflow_installation.py", line 256, in <module>
    install_kubeflow(
  File "utils/kubeflow_installation.py", line 79, in install_kubeflow
    build_component(
  File "utils/kubeflow_installation.py", line 132, in build_component
    install_helm(component_name, installation_path, namespace)
  File "/Volumes/workplace/kubeflow/kubeflow-manifests/tests/e2e/utils/utils.py", line 222, in install_helm
    assert install_retcode == 0
AssertionError
make: *** [deploy-kubeflow] Error 1

Description of your changes:
Updated the helm command from helm install to helm upgrade --install, which would run an install if a release by this name doesn't already exist.
https://helm.sh/docs/helm/helm_upgrade/
Testing:

[N.A] Unit tests pass
[N.A] e2e tests pass
Details about new tests (If this PR adds a new feature)
Details about any manual tests performed

Manually tested by running deploy-kubeflow. Observed that installation is successful without Error: INSTALLATION FAILED: cannot re-use a name that is still in use.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Thanks you for your contribution


Fix incorrect request body when intercepting fetch with getMiniflareFetchMock

Hey there!
I am using miniflare with jest to test a Workers library. I need to write a simple unit test that intercepts a request body and returns it in response.
Example (full code here):
fetchMock.get('...')
  .intercept({
    method: () => true,
    path: () => true,
  })
  .reply(200, (opts) => {
    return opts;
  });

The issue I'm encountering is that opts.body doesn't contain request body string, as expected per https://undici.nodejs.org/#/docs/best-practices/mocking-request?id=reply-with-data-based-on-request , but AsyncGenerator.
I first thought that this was an issue in undici but after discussion with their team (https://github.com/nodejs/undici/issues/1756) it seems that the solution is to expose isMockActive property on Dispatcher for test utilities on MockAgent to work properly.
This PR adds a new MiniflareDispatcherMock class that extends the base MiniflareDispatcher with isMockActive getter. I decided to not add the getter directly to MiniflareDispatcher, because it's a testing function that doesn't exist on real dispatchers.
This class is used on fetch polyfill to instantiate dispatcher when bound with MockAgent.

Well done @robertcepa this is something we have been trying to work around here. Thank you for the PR.
Hey @mrbbot, we have been trying to get this working here since mid-last year and have just noticed someone has already fixed it. Do you have any idea when this will be released? Thank you.

Hi! I'm also invested in seeing this released :) do we have a timeline?

@mrbbot was talking about this PR today with the team. Is there a plan for a wide release? The automated testing we are trying to implement needs this.

Can we get any updates on this one, please? We really want to be able to test our API. 😁

Just stumbled across this PR when trying to figure out how to unit test my worker. Do you know when this will be added to a release @mrbbot?


Release v1.4.1

@JuliaRegistrator register
Release notes:
What's Changed

Fix init_rme by @Zapiano in https://github.com/open-AIMS/ReefModEngine.jl/pull/24

New Contributors

@Zapiano made their first contribution in https://github.com/open-AIMS/ReefModEngine.jl/pull/24

Full Changelog: https://github.com/open-AIMS/ReefModEngine.jl/compare/v1.4.0...v1.4.1

Registration pull request created: JuliaRegistries/General/116941
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v1.4.1 -m "<description of version>" 3ffddb86d2de8d3844fca936ad41aa4d5bff45ad
git push origin v1.4.1


Where is the random button???

What's up bro!
It's time to implement the random button in this project.
This button should print a random card on the screen each time it is clicked.

Hi boss!
I'm working on it...


How to change the version of ChatGPT?

I have use the paid version API and when I change the version it does not change based on the ChatGPT respond.

Thank you for any of the support.

Hey @navotera this issue you are experiencing I believe is not really because of a problem with the UI, this is the normal behaviour when using the model called gpt-4.
There are many questions online about that. For example: gpt-4-through-api-says-its-gpt-3 , anyone-getting-gpt-3-responses-when-calling-api-for-gpt-4-model, api-model-set-to-gpt-4-but-seems-to-respond-as-gpt-3
To see the expected result (the model saying it is based on chat GPT 4) use the model: gpt-4-turbo for example.
Showing this behaviour locally:
Model gpt-4:

Model gpt-4-turbo:

Thank you for your response.
It should be noted in the readme to avoid such confusion.

@navotera LLMs are generally not introspective, because their own parameters (model name, cutoff date, etc.) are not generally present in training data. there are many such tripping hazards for new users of LLMs, and i think this kind of knowledge is unfortunately out of scope for this repo. they would be better suited to tutorials or cheatsheets providing general information.


Howto: paramValues for dynamic route src/routes/[uid]/+page.svelte

I have a top level dynamic route in my Sveltekit app: src/routes/[uid]/+page.svelte
uid is a database uid of 28 alphanumeric characters such as kthMdsJHyxbGQVb30YzZK1yDJ5j1 or HLQBWSwvpKf93PwFhye3xhd61063. The point being, those are unknown beforehand so I would need something in paramValues to match those e.g. a regex such as /^[a-zA-Z0-9]{28}$/.
How do I do that i.e. turn the below trial code with two hardcoded UIDs into something that works for any uid possible?
    paramValues: {
      '/[uid]': ['kthMdsJHyxbGQVb30YzZK1yDJ5j1', 'HLQBWSwvpKf93PwFhye3xhd61063']

    }

Are you running into an error or is this a more general question? My interpretation is there's just confusion on how to use super sitemap:

Within super-sitemap's config, paramValues is not a regex, it is the data itself that should be used as param values, so super sitemap is given the required data that it needs to generate a list of paths for a dynamic route. For example, an array containing a list of blog post titles. Or in your case, an array containing the list of current UIDs. See example: https://github.com/jasongitmail/super-sitemap?tab=readme-ov-file#the-everything-example
regexs are only used in super sitemap's config for excludeRoutePatterns, which allows you to exclude certain routes so that paths are not created for them.
The data itself (i.e. your valid UIDs) needs to exist somewhere in order for a list of paths to be generated that including this data. In this case, presumably your database includes the UIDs for valild records; you'd simply fetch them (see link earlier) and then provide that as an array of uid's as the paramValues to be used. You can fetch them however you'd like, the only thing supersitemap cares about is that give it an array of values to be used as your UIDs.

// This would actually be a request to your database
const arrayOfUIDsFromDB = ['kthMdsJHyxbGQVb30YzZK1yDJ5j1', 'HLQBWSwvpKf93PwFhye3xhd61063', ...]

 ...
 paramValues: {
      '/[uid]': arrayOfValuesFromDB
    }
 ...

If values of these change over time, do not prerender this route with Svelte Kit. Instead, let the sitemap route be generated on an as-needed basis with the most recent data available in your database. But I'd recommend you enable caching headers on your response, so that crawlers won't generate a new sitemap more than every x seconds, like say 60sec, but you could use something longer too.

    headers: {
      'custom-header': 'foo',
      'cache-control: 'max-age=0, s-maxage=60'
    },


Add guidance banner to introduction page

What is the context of this PR?
Describe what you have changed and why, link to other PRs or Issues as appropriate.
How to review
Describe the steps required to test the changes (include screenshots if appropriate).
Checklist

[ ] New static content marked up for translation
[ ] Newly defined schema content included in eq-translations repo

 Banner on introduction page (3 SP)

Validator branch needs changing.


Setup Java Maven Projects for Both Client and Server

Set Up Java Maven Projects for Both CLIENT and SERVER
Initialize Java Maven projects for both the CLIENT and SERVER applications.
Repository:
Server
Client
Actionable Tasks:
For SERVER:

Create a new Maven project in Misight-Server.
Set up the pom.xml with the necessary dependencies for Spring Boot, JPA, MySQL, and testing libraries (JUnit, Mockito).
Organize the directory structure:
src/main/java for the main application code.
src/main/resources for application properties and configuration files.
src/test/java for unit tests.

For CLIENT:

Create a new Maven project in Misight-Client.
Set up the pom.xml with necessary dependencies (e.g., Apache HttpClient for making API requests, JUnit for testing).
Organize the directory structure:
src/main/java for the main application code.
src/test/java for unit tests.

Completion Criteria:
Both Maven projects are initialized, the directory structures are set up correctly, and the pom.xml files are configured with required dependencies.

This is confirmed compplete


The events in bugsnag record the native stacktrace, not the stacktrace within Flutter

For example
NoSuchMethodError: The getter 'services' was called on null. Receiver: null Tried calling: servicesfunction signature specialization <Arg[2] = Owned To Guaranteed> of function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @escaping @callee_unowned @convention(block) (@unowned Swift.Optional<Swift.AnyObject>) -> () to @escaping @callee_guaranteed (@in_guaranteed Swift.Optional<Any>) -> (), Argument Types : [@escaping @callee_unowned @convention(block) (@unowned Swift.AnyObject?) -> ()]> of bugsnag_crashlytics.SwiftBugsnagCrashlyticsPlugin.handle(_: __C.FlutterMethodCall, result: (Any?) -> ()) -> ()

The exception message is the only useful piece of information, the stacktrace treats the Flutter code as an opaque blob. The Sentry plugin does not have this issue.

any way to symbolicate the logs? or have any info about the current window, ecc?