move github actions to micromamba action

Use micromamba setup isntead of pip install for speedup (e.g. similar to weldx setup)
      - uses: CagtayFabry/pydeps2env@main
        with:
          file: 'pyproject.toml'
          channels: 'conda-forge defaults'
          extras: 'test vis media'
          setup_requires: 'include'

      - name: Setup Conda Environment
        uses<EMAIL_ADDRESS>        with:
          environment-file: ./environment.yml
          environment-name: weldx
          init-shell: >-
            bash
            powershell
          cache-environment: true
          create-args: >-
            python=${{ matrix.py }}
            wheel
            pip

pip will always be faster than anything conda based ;) But feel free to open a PR if you prefer this solution.

pip will always be faster than anything conda based ;) But feel free to open a PR if you prefer this solution.

I guess the cache action would have to monitor pyproject.toml in that case, can you add change this in #107?


If a field value is purely numeric, re.sub will throw and exception.

Nice code, thanks for all this work :)

Thanks for your fix! Sorry it took so long to get to it. I integrated it into the development branch. It will go into Pypi next time I make a release.


Missing from lean 4: attributes

The following lean 3 builtin attributes have no lean 4 analogues:

@[refl]
@[symm]
@[trans]
@[subst]
@[intro]
@[elab_as_eliminator]
@[pp_nodot]
@[pp_using_anonymous_constructor]
@[algebra]
@[ematch]
@[ematch_lhs]
@[rsimp]
@[no_rsimp]
@[hole_command]
@[user_attribute]
@[user_command]
@[user_notation]

There are also a number of additional attributes in mathlib that lean 4 does not support. This is blocked on the ability to create custom attributes in lean 4.

Most of these attributes are obsolete anyhow.

@[refl] & @[trans]:  These are still interesting, but mainly for calc and reflexivity/transitivity tactics.  A related but potentially deferrable question is whether they should stay attributes, or become typeclasses.
@[ematch]. The real question here is what do with proofs that use cc?  Reimplement congruence closure in Lean 4, or refactor all the proofs?

The rest can be thrown away as far as I'm concerned.

I think pp_using_anonymous_constructor could still be useful in lean 4 with the same meaning, and elab_as_eliminator probably warrants its own issue (I haven't got to elaboration of translated lean 4 files yet but I know this is going to be a major issue eventually).
I'll open up some more issues for individual tactics at some point, if someone else doesn't get to them first.


Don't change ElementCollection type on toArray

Proposed changes
Don't change ElementCollection type on toArray
Fixed https://github.com/selenide/selenide/issues/844
Checklist

[ ] Checkstyle and unit tests pass locally with my changes by running gradle check chrome htmlunit command
[x] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

@borisosipov what problem are you actually trying to solve?

@asolntsev this issue https://github.com/selenide/selenide/issues/844


Remove allow pragmas for clippy::missing_panics_docs

This false positive was addressed in rust-lang/rust-clippy#6996.
This rolls back the allow introduced in https://github.com/artichoke/intaglio/pull/67.

A similar PR for rand_mt is here: https://github.com/artichoke/rand_mt/pull/105.


[docs] Update references to XD to Creative Cloud

On the Design System Kit page, we have several references to XD that should be tweaked to reference Creative Cloud instead.
For example on the CTA:
Request an Adobe for Teams plan
Instead should say:
Request a Creative Cloud license
Also may need some tweaks around the steps on installing, for example:
instead of installing XD first, once a license is granted, Associates can install their desired app(s) from within Adobe CC.

Are we looking to replace all mentions of XD? I'm mostly wondering about the "Working in Adobe XD" section, since features like component states and responsive resize aren't available in other CC apps.

@marionnegp Do you have what you need here? Feel free to reach out to Mark directly if not.

@markcaron, could you clarify whether we're removing all mentions of XD?

@marionnegp, TMK, we don't have to remove all mentions of XD. It's really just correcting language around the process of "getting a license". We don't have an XD license, but a Creative Cloud license.

Need to update Adobe XD license wording on:

[ ] Overview (under Get started section)
[ ] FTS Starter Kit page
[ ] Brand/UI icon library
[ ] Product logo library

Related pull request: https://github.com/RedHat-UX/red-hat-design-system/pull/415


Support running pub serve on directories other than "web/"

This is the pub serve dual of #14673.

This issue has been moved to dart-lang/pub#805.


i m trying to add new module screen shot attached. but install button not clicked(not respond)

Describe the bug
i m trying to add new module screen shot attached. but install button not clicked(not respond)
try this sudo journalctl -u mercure_ui.service
error showing
docker.errors.dockerexception error while fetching server api version(connection aborted.,permissionerror(13,'permission denied'))

This looks like a configuration problem on your server. Make sure that the user running mercure (typically the user "mercure") has permissions for run docker commands. This usually requires that the user is added to the docker group under Linux. For instructions, visit this page:
https://docs.docker.com/engine/install/linux-postinstall/
We have recently added improved error reporting in the UI. This is currently only in the DEV version and will be part of the next release.


feat: add enableExternalSource for reactivity interoperability

This pull request brings a new API enableExternalSource which provides more flexible reactivity interoperability (i.e. it's possible to use external reactive source directly in solid primitives (createMemo, createEffect, etc.))
For a concrete example please check out packages/solid/test/external-source.spec.ts.
Potential issue:

enableExternalSource could be piped, but this feature seems not to be reflected well from the naming?

Interesting. However I would like to see a demo other than the test and its use cases.
The concept reminds me of React's external sources however I'm not sure they solve the same problem.

Interesting. However I would like to see a demo other than the test and its use cases.
The concept reminds me of React's external sources however I'm not sure they solve the same problem.

It's primarily designed for reactive library creators rather than normal users. The result is: users are able to use other framework-agnostic reactivity implementations seamlessly.
// before
function Component() {
    const reactiveObject = createThirdPartyReactiveObject();
    const [signal, setSignal] = createSignal(reactiveObject.value);
    createEffect(()=>reactiveObject.subscribe(setSignal))
    return <>{signal()}</>
}

// after
function Component() {
    const reactiveObject = createThirdPartyReactiveObject();
    return <>{reactiveObject.value}</>
}

The principle is: the third-party library will be able to track its own reactive dependencies and notify solid if there is a change.
@ryansolid suggested the name enableExternalSource, I think it has nothing to do with react's concept.

Mobx integration example (haven't tested yet, but I believe it's almost there)
import { Reaction } from 'mobx';

let id = 0;

enableExternalSource((fn,trigger)=>{
  const reaction = new Reaction(`externalSource@${++id}`, trigger);
  onCleanup(()=>reaction.dispose());
  return x => reaction.track(()=>fn(x));
});

I think it has nothing to do with react's concept.

Interestingly, it is. React's external sources provides a way to bridge non-React third-party reactive objects (e.g. Observables, event emitters, etc.) into React components.
I think we can probably simplify this into something that just provides a special kind of signal?

I think it has nothing to do with react's concept.

Interestingly, it is. React's external sources provides a way to bridge non-React third-party reactive objects (e.g. Observables, event emitters, etc.) into React components.
I think we can probably simplify this into something that just provides a special kind of signal? much like how from works

I'm not sure what do you mean by react's external source (google provides nothing useful) so I assume you are talking about useMutableSource. This API has gone (becomes useSyncExternalStore) and the its primary use case is to make reading from external mutable data safe in concurrent mode, not just about triggering re-rendering when data changes (otherwise useSubscription is enough).
I'm not sure what do you mean by from, I think this is already the simplest solution. Could you elaborate it?

I'm not sure what do you mean by react's external source (google provides nothing useful) so I assume you are talking about useMutableSource. This API has gone (becomes useSyncExternalStore) and the its primary use case is to make reading from external mutable data safe in concurrent mode, not just about triggering re-rendering when data changes (otherwise useSubscription is enough).

Yes I got the terminologies mixed up, sorry.

I'm not sure what do you mean by from, I think this is already the simplest solution. Could you elaborate it?

This was a new feature shipped in Solid 1.2.0. I transforms an Observable source into a signal.

The big advantage of this API is, the end user can use external reactive object directly in any place a signal could be placed, and it's almost imperceptible.
// example taken from https://mobx.js.org/README.html

class Timer {
    secondsPassed = 0

    constructor() {
        makeAutoObservable(this)
    }

    increase() {
        this.secondsPassed += 1
    }

    reset() {
        this.secondsPassed = 0
    }
}

const myTimer = new Timer();

render(()=><button onClick={() => timer.reset()}>Seconds passed: {timer.secondsPassed}</button>, document.body)

// Update the 'Seconds passed: X' text every second.
setInterval(() => {
    myTimer.increase()
}, 1000)

Hmmm yeah I think this make sense, however personally I think doesn't this hide any implementation detail. Well I guess this is just by preference

I am super stoked about the direction here. It does add an overhead to every computation but for people basically looking to swap the reactive system I think it's pretty cool. While most of it can be very closely done by just wrapping Solid's APIs to the point I'd say that is the preferred method if it is a syntax thing. Some libraries have subtly different execution. Being able to use Kairo or like MobX State Tree essentially natively is really cool. This may even finally be the thing that lays stuff like mobx-jsx to rest. Really cool stuff.
That being said the form of the API makes sense to me from my knowledge of MobX and even how the React Solid State works. But would be interested in opinions.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

11 of 11 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.07%) to 90.168%

Totals

Change from base Build<PHONE_NUMBER>:
0.07%

Covered Lines:
1140

Relevant Lines:
1212

💛  - Coveralls

@ryansolid  Another thing worth to notice: transition (scheduling) will not work (I mean, not effective, but user will still get the expected output/behavior) with external computations. This is expected and so far I'd say there is no solution to perfectly tackle it. Even react hasn't figured it out.

There are 3 levels of implementations in React Concurrent Feature:

Make it work: useSubscription
Make it right: useSyncExternalStore -> all external state management libraries' the best situation. Essentially level 1 without tearing, to guarantee consistency. But de-opts still occur (sync re-rendering).
Make it fast: use react hooks only

In solid we don't need to worry about consistency. This feature is fully built upon existing mechanisms. But the scheduling won't produce any effect (unless there are HUGE amounts of updates).
If the scheduling is INDEED necessary, the best solution is: external scheduling + solid effects. So it's up to external reactive library to implement the scheduling stuff. 99.9% (I made up) users don't need to worry about this.

Let me think. Would it work? Yes the third party would update on its change, but wouldn't the internal computation still only rerun as scheduled. We are just proxying the signal.  We are leveraging the og computations to synchronize.
Let's say we are under Suspense without transitions being read under an effect. 3rd party triggers update, sets signal, then Effect gets scheduled and held. Then at the end runs again.
I guess the problem with concurrency is we can't hold setting the external source if it is the trigger of the concurrency. So for example you change the tab it blurs both realities. Maybe the side effects get held but any new recalculation reads the updated value. Also any external to Solid side effect runs immediately.  The second is less concerning as there is nothing to be done for that. But the first hmm.. i guess without providing another wrapper where it is read. Or atleast where it enters the app. Hmmm.. it's clunkier but it would potentially be more correct.

I prefer to leave it over for the present...... It's a rare edge case (and likely to be over-engineered). Not a bug but could be enhanced (for better or worse). I may continue investigating on interruptible scheduling.

It's not interruptible or at least what I call interruptible that I'm concerned about. We have the enableScheduling but it's the transitions without that I'm more concerned with. The typical async transitions. Those start a change, wait on async, and then commit. Those are way more common (ie.. baked into the router/all routing?). While I agree for navigation it will probably never get you in trouble it is not an edge case. To be fair I have a couple of holes still there in Solid proper but I intend to resolve those. And introducing something like this we should at least consider if it has an answer. I'm fine as-is for today, assuming there is a way out. We don't have the legacy of something like React, so it's beneficial to at least consider if a new feature can play along.

The critical issue is, should referential transparency be holden in (transition) pure computation? (by the name it should, of course) (reading a signal is seen as input, while untrack(signal) is the same as reading an external source). This could be a common pitfall, not only for this.

After digging into the current implementation and future roadmap of solid transitions(concurrency),  I believe this feature is not compatible with it. The reasons:

There are multiple realities, but externals have no clue in which reality they are

and some of them dynamically collect dependencies. There will be multiple versions of dependency graph, not just multiple versions of value

Referential transparency will be broken but no (easy) way to rescue. This is vital for interruptible scheduling, as it guarantees a computation will produce the same result with the same input no matter when it is evaluated.

But it doesn't mean this feature is not valuable. Without transition enabled it works well. And thanks to the low coupling between reactivity and rendering, it is possible for third party libraries to provide equivalent/similar mechanisms. (so it doesn't means it's impossible to have transition/concurrent scheduling with enableExternalSource anymore,  but this capability is turned over to externals)
That's to say, enableExternalSource will make solid transition noneffective.

I think I need some help. No clue how to test out transition-related stuffs . Seems Transition has been lack of tests


[camera] disable auto focus when using front facing camera on Android

Description
This pull request fixes a bug with setting auto focus (necessary for the flash feature) which made it impossible on most Android devices to take a picture with the front facing camera.
It also resolves an issue were the preview of the front facing camera is very dark, which had to do with the configured FPS range.
Related Issues

Resolves flutter/flutter#73160

Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] My PR includes unit or integration tests for all changed/updated/fixed behaviors (See Contributor Guide).
[x] All existing and new tests are passing.
[x] I updated/added relevant documentation (doc comments with ///).
[x] The analyzer (flutter analyze) does not report any problems on my PR.
[x] I read and followed the Flutter Style Guide.
[x] The title of the PR starts with the name of the plugin surrounded by square brackets, e.g. [shared_preferences]
[x] I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy.
[x] I updated CHANGELOG.md to add a description of the change.
[x] I signed the CLA.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require plugin users to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (please indicate a breaking change in CHANGELOG.md and increment major revision).
[x] No, this is not a breaking change.

@bparrishMines @mvanbeusekom When this PR can be merged?

@bparrishMines @mvanbeusekom When this PR can be merged?

@bparrishMines @mvanbeusekom When this PR can be merged?

@divyanshub024 once we have approval from the Google Team, which usually happens within a few days, we will merge the PR and I will request to publish this new version.

@bparrishMines @mvanbeusekom When this PR can be merged?

@divyanshub024 once we have approval from the Google Team, which usually happens within a few days, we will merge the PR and I will request to publish this new version.

@mvanbeusekom Thanks for the fix, how can we use this with Flutter Stable? is it camera: ^0.6.3+3 ?

@mvanbeusekom Thanks for the fix, how can we use this with Flutter Stable? is it camera: ^0.6.3+3 ?

@aytunch this code is part of version 0.6.4+3 which has now been merged into the master branch of the flutter/plugins repo but has not been released yet to pub.dev. I expect this to happen very soon.
If you want to continue developing earlier you can make a reference to the git repo directly from your pub.spec file (more details can be found here):
  camera:
    git:
      url: git@github.com:flutter/plugins.git
      ref: master
      path: packages/camera/camera

@aytunch this code is part of version 0.6.4+3 which has now been merged into the master branch of the flutter/plugins repo but has not been released yet to pub.dev. I expect this to happen very soon.
If you want to continue developing earlier you can make a reference to the git repo directly from your pub.spec file (more details can be found here):
  camera:
    git:
      url: git@github.com:flutter/plugins.git
      ref: master
      path: packages/camera/camera

@mvanbeusekom I tried it with the new version, however the camera still does not get opened for me
I am testing it in a real device One Plus 7t. It has a Pop-up camera
Pop up camera opens up but the camera view is never shown and init() never ends

@mvanbeusekom I tried it with the new version, however the camera still does not get opened for me
I am testing it in a real device One Plus 7t. It has a Pop-up camera
Pop up camera opens up but the camera view is never shown and init() never ends

Did you override de dependency for the camera_platform_interface in your pubspec.yaml file?
If so remove it, run flutter clean followed by flutter pub get

Did you override de dependency for the camera_platform_interface in your pubspec.yaml file?
If so remove it, run flutter clean followed by flutter pub get

@mvanbeusekom I just tried again as you said and no luck :/

@mvanbeusekom I just tried again as you said and no luck :/

@aytunch that is strange, we are not experiencing any issues on our side on Android (tested on One Plus, Samsung and LG devices).
Could you maybe submit a new issue and list the output of flutter doctor -v and the contents of your pubspec.yaml file?

@aytunch that is strange, we are not experiencing any issues on our side on Android (tested on One Plus, Samsung and LG devices).
Could you maybe submit a new issue and list the output of flutter doctor -v and the contents of your pubspec.yaml file?

B.t.w. the new version of the camera plugin containing this fix has just been released on pub.dev.

B.t.w. the new version of the camera plugin containing this fix has just been released on pub.dev.

not sure if related, but after update to 0.6.4+3, auto focus seems disabled for the back camera while in preview mode on OnePlus 6, Android 10.... please see Attached.

Attached

not sure if related, but after update to 0.6.4+3, auto focus seems disabled for the back camera while in preview mode on OnePlus 6, Android 10.... please see Attached.

Attached


nipapd postgresql dependency (link to  NIPAP on Ubuntu 16.04 #908)

Can someone please update the debian packages? The documented installation still results in version 28.4-1 being retrieved which depends on old postgresql not available on current LTS ubuntu:
Package: nipapd
Versions:
0.28.4-1 (/var/lib/apt/lists/spritelink.github.io_NIPAP_repos_apt_dists_stable_main_binary-i386_Packages)
Description Language:
File: /var/lib/apt/lists/spritelink.github.io_NIPAP_repos_apt_dists_stable_main_binary-i386_Packages
MD5: 8211a2a377e6606ba3f9c3d36c67deb7
nipapd : Depends: postgresql-9.1-ip4r (>= 2.0) but it is not going to be installed or
postgresql-9.3-ip4r (>= 2.0) but it is not installable or
postgresql-9.4-ip4r (>= 2.0) but it is not installable
E: Unable to correct problems, you have held broken packages.
Please reopen issue #908 until fixed.
Thank you!

Sorry for late feedback - I missed this one. I think the recommendation here is simply to move to 0.29, which is currently in testing. I do get that it's an inconvenience while it's still not marked as stable but we just don't have the man power to backport, test and package old maintenance releases.
0.29.2 is turning out rather well, can you give that a go? :)

Going to close this one. Won't be updated 0.28 - go with 0.29 - it will be in stable rather shortly.

Please re-open this issue - The problem persists for users of ubuntu-16.04.1-server:

root@nipap:~# apt-get install nipapd
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
nipapd : Depends: postgresql-9.1-ip4r (>= 2.0) but it is not going to be installed or
postgresql-9.3-ip4r (>= 2.0) but it is not installable or
postgresql-9.4-ip4r (>= 2.0) but it is not installable
E: Unable to correct problems, you have held broken packages.


wait for an isolate runnable event before setting any breakpoints

From https://github.com/dart-lang/sdk/issues/25602.

@turnidge FYI

@johnmccutchan @turnidge, how do I know the state of a particular isolate? When I connect, I get the list of currently known isolate (getVM().isolates). I then respond to events about new isolates (kIsolateStart -> kIsolateRunnable -> [kIsolateUpdate ->] kIsolateExit). But if an isolate existed before I first connected, how to I know whether that isolate is at the started state or the runnable state?

@devoncarew Good catch. I've got a CL out and a fix will be merged in to stable for 1.4.1 that adds a 'runnable' bit.

Fixed in 1e6c5f196befa21b26b3bd40877167691ee9c346. An Isolate has a "runnable" boolean property.

Merge request for stable https://github.com/dart-lang/sdk/issues/25721

Thanks! Will update and use a runnable flag if one is present.

@devoncarew You should probably let the user know if the protocol version < 3.2 that debugging won't work.


Game Screen - Playing Card Components

closed in commit https://github.com/sopra-fs23-group-37/client/commit/4870b29810c566d815ffb048fbaafedebd4fd19d


mat3.normalFromMat4 seems to return unexpected values

The documentation of mat3.normalFromMat4() says, the method returns the transpose inverse of the given matrix. So I would expect the same result when i call transpose & invert on my own. In fact it is not:
const mv = [
  0.88,   -0.27,     0.39,     0.39,
  0.39,    0.88,    -0.27,    -0.27,
 -0.27,    0.39,     0.88,     0.88,
   1.5,       0,       -7,        1
];

const n1 = mat3.normalFromMat4(mat3.create(), mv);

const n2 = mat3.fromMat4(mat3.create(), mv);
mat3.invert(n2, n2);
mat3.transpose(n2, n2);

//  n1                    n2
//  0.8804924488067627    0.8804924488067627
// -0.27054348587989807  -0.27054348587989807
//  0.21384871006011963   0.3900510370731354
//  0.3900510370731354    0.3900510370731354
//  0.8804924488067627    0.8804924488067627
//  0.03931663557887077  -0.27054348587989807
// -0.27054348587989807  -0.27054348587989807
//  0.3900510370731354    0.3900510370731354
//  0.0593346506357193    0.8804924488067627

In my program, this results in wrong lightning calculations. Is there something I do miss?

When setting mv to
const mv = [
  0.88,   -0.27,     0.39,     0,
  0.39,    0.88,    -0.27,    0,
 -0.27,    0.39,     0.88,     0,
   0,       0,       0,        1
];

the results are equivalent.
That behavior is because those values at the "edges" affect the determinant of the matrix, which is used to calculate the inverse.
https://github.com/toji/gl-matrix/blob/518d94fbcc0439c67b15958528a996574feaac0f/src/mat3.js#L531


[spi_device/dv] Disallow certain state changes when busy

Description
On a real SPI flash device, many commands are ignored when the device is busy. We might want to consider adding this behavior to our IP as well.
Currently, command uploads, WREN, WRDI, EN4B, and EX4B all are allowed while the busy bit is set.

I have created this primarily for future reference (and evaluation in the D2 sign-off meeting). I don't think this is necessary for the next earlgrey release.

A similar discussion about this happened here: https://github.com/lowRISC/opentitan/pull/12053#issuecomment-1095739434

@a-will: This is currently in M4, but your proposal would be rather to not do this for this release (thus backlog), right?

In the D2 sign-off, I think we decided to just get it done.

@moidx: I think we already have this feature, so this may only be a coverage gap. Needs to be taken into account for DV.

@moidx: I think we already have this feature, so this may only be a coverage gap. Needs to be taken into account for DV.

@andreaskurth  - This needs doing in both RTL and TB (I just checked with @a-will)

Off the top of my head, I'd probably simply add the current "busy" value as an input to spi_cmdparse, then use it to block the commands that should be ignored. For example, if busy is 1, we could prevent decoding the opcode here:
https://github.com/lowRISC/opentitan/blob/c84524e63e6e88c6f63504722e52b2f605d29312/hw/ip/spi_device/rtl/spi_cmdparse.sv#L194-L195
There are likely a number of equivalent means of doing it, though.

Ah, in that case we were wrong about what needs to happen here during today's triage -- sorry! Thanks for clarifying :+1:
@antmarzam, as current coordinator for spi_device, may I leave it to you to find someone to do the RTL change (or do it yourself)?


refactor(Probability/Kernel/CondCdf): mv atBot_le_nhds_bot/top

Cannot move to order/filter/at_top_bot as suggested in the todo, because TopologicalSpace is not available there and probably not a good place to import it. Hence moving to  Topology/Order/Basic instead.


Alignment Issues on the "About Us" Page

Description:
The text below the "About Us" heading is not centered properly.
The text appears in two lines, but the line break is not correctly formatted. that affects the readability and visual appeal of the "About Us" page, making it important to fix for a better user experience.
Expected Behavior:
The text below the "About Us" heading should be centered, and the line breaks should be correctly formatted to improve readability.

please assign this task to me.

Assigning


Training larger models lead to an error

I tried pretraining larger T5 using TPUs and GPUs on main branch, and I got the same generic error (at the bottom) in both cases, which didn't happen when trained using our repo >~5 days ago., somewhere between 0th iteration and 2000th iterations. For smaller models, it doesn't cause any problem, and the training looks normal. I'll try to figure out what's causing tomorrow. Two possible issues is that I had to install t5 from sources to import preprocessors.full_lmand I changed the t5x/data format dramatically.
I1115 16:00:39.451247<PHONE_NUMBER>62816 trainer.py:501] Training: step 198
I1115 16:00:39.451110<PHONE_NUMBER>40832 trainer.py:501] Training: step 198
I1115 16:00:39.451253<PHONE_NUMBER>68736 trainer.py:501] Training: step 198
I1115 16:00:39.451272<PHONE_NUMBER>64352 trainer.py:501] Training: step 198
I1115 16:00:39.451283<PHONE_NUMBER>60448 trainer.py:501] Training: step 198
2022-11-15 16:03:15.140058: E external/org_tensorflow/tensorflow/core/distributed_runtime/coordination/coordination_service.cc:954] /job:jax_worker/replica:0/task:37 has been set to ERROR in coordination service: UNAVAILABLE: Task /job:jax_worker/replica:0/task:37 heartbeat timeout. This indicates that the remote task has failed, got preempted, or crashed unexpectedly. [type.googleapis.com/tensorflow.CoordinationServiceError='']
2022-11-15 16:03:15.140101: E external/org_tensorflow/tensorflow/core/distributed_runtime/coordination/coordination_service.cc:954] /job:jax_worker/replica:0/task:38 has been set to ERROR in coordination service: UNAVAILABLE: Task /job:jax_worker/replica:0/task:38 heartbeat timeout. This indicates that the remote task has failed, got preempted, or crashed unexpectedly. [type.googleapis.com/tensorflow.CoordinationServiceError='']
2022-11-15 16:03:15.140110: E external/org_tensorflow/tensorflow/core/distributed_runtime/coordination/coordination_service.cc:954] /job:jax_worker/replica:0/task:32 has been set to ERROR in coordination service: UNAVAILABLE: Task /job:jax_worker/replica:0/task:32 heartbeat timeout. This indicates that the remote task has failed, got preempted, or crashed unexpectedly. [type.googleapis.com/tensorflow.CoordinationServiceError='']
2022-11-15 16:03:15.140118: E external/org_tensorflow/tensorflow/core/distributed_runtime/coordination/coordination_service.cc:954] /job:jax_worker/replica:0/task:36 has been set to ERROR in coordination service: UNAVAILABLE: Task /job:jax_worker/replica:0/task:36 heartbeat timeout. This indicates that the remote task has failed, got preempted, or crashed unexpectedly. [type.googleapis.com/tensorflow.CoordinationServiceError='']
2022-11-15 16:03:15.140126: E external/org_tensorflow/tensorflow/core/distributed_runtime/coordination/coordination_service.cc:954] /job:jax_worker/replica:0/task:39 has been set to ERROR in coordination service: UNAVAILABLE: Task /job:jax_worker/replica:0/task:39 heartbeat timeout. This indicates that the remote task has failed, got preempted, or crashed unexpectedly. [type.googleapis.com/tensorflow.CoordinationServiceError='']
2022-11-15 16:03:15.140132: E external/org_tensorflow/tensorflow/core/distributed_runtime/coordination/coordination_service.cc:954] /job:jax_worker/replica:0/task:33 has been set to ERROR in coordination service: UNAVAILABLE: Task /job:jax_worker/replica:0/task:33 heartbeat timeout. This indicates that the remote task has failed, got preempted, or crashed unexpectedly. [type.googleapis.com/tensorflow.CoordinationServiceError='']
2022-11-15 16:03:15.140140: E external/org_tensorflow/tensorflow/core/distributed_runtime/coordination/coordination_service.cc:438] Stopping coordination service as heartbeat has timed out for /job:jax_worker/replica:0/task:37 and there is no service-to-client connection
2022-11-15 16:03:15.145700: E external/org_tensorflow/tensorflow/core/distributed_runtime/coordination/coordination_service_agent.cc:716] Coordination agent is in ERROR: DEADLINE_EXCEEDED: Deadline Exceeded
Additional GRPC error information from remote target unknown_target_for_coordination_leader:
:{"created":"@1668528195.145562804","description":"Deadline Exceeded","file":"external/com_github_grpc_grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
2022-11-15 16:03:15.145732: E external/org_tensorflow/tensorflow/compiler/xla/pjrt/distributed/client.cc:452] Coordination service agent in error status: DEADLINE_EXCEEDED: Deadline Exceeded
Additional GRPC error information from remote target unknown_target_for_coordination_leader:
:{"created":"@1668528195.145562804","description":"Deadline Exceeded","file":"external/com_github_grpc_grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
2022-11-15 16:03:15.145749: F external/org_tensorflow/tensorflow/compiler/xla/pjrt/distributed/client.h:75] Terminating process because the coordinator detected missing heartbeats. This most likely indicates that another task died; see the other task logs for more details. Status: DEADLINE_EXCEEDED: Deadline Exceeded
Additional GRPC error information from remote target unknown_target_for_coordination_leader:
:{"created":"@1668528195.145562804","description":"Deadline Exceeded","file":"external/com_github_grpc_grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
Fatal Python error: Aborted

I copied full_lm to t5x.data.preprocessors in the add_FCM branch (merged). You can check if the problem still persists.

@AranKomat this error starts happening at what size? Large or XL?

@lintangsutawika It starts happening somewhere beyond 250M to 500M parameter count, so Large was affected, for example. But I don't think it's hopeless once I return back from the dataset building :)

I'm testing running on TPUs for XL. The error doesn't seem to trigger. It may be fixed. Currently testing on GPUs to be sure.

@lintangsutawika That sounds great to me!

So it def works on TPUs. Have had no issue training up to 99k steps (I've assumed there is no issue and stopped training).
Testing on GPUs however with 32 Nodes (256 GPUs total), the error does happen at step 2970 with the same error.
Will investigate further.

@AranKomat but we can go ahead a train our experiments on TPUs for now since it's more efficient anyway. Wdyt?

Nvm, the GPU version is also fixed. Use the main branch.

@lintangsutawika

Testing on GPUs however with 32 Nodes (256 GPUs total), the error does happen at step 2970 with the same error.

This is exactly what happened to me for larger models! So, you tested it on XL, I presume?
By the way, you mentioned you tried with 32 nodes. Last time I switched from 8 nodes to 32 nodes (a month ago?), I got very little speedup. Did you experimence something similar? If that's the case, I'm guessing it's due to some interconnect bottleneck argued before.

Yes, tested on XL and it no longer exits on step 2970 or anyother step due to the GRPC error.
Last time I tested on 8 nodes it didnt work due to OOM.  But with 32 nodes (Same setup as NVIDIA) I'm getting about 2200 seq/sec which is about 61% of the reported numbers from the NVIDIA team.

@lintangsutawika That sounds fantastic.
The last commit of the main branch seems to be done 6 days ago. Is the fix included in one of these? Do you know what was responsible for the error?

I wasn't able to pinpoint the exact cause, but I did get the error when I was using another branch that was not main.


v0.6.11 support sm2 sm3, what is Requirements for encryption card model and qat driver vsersion?

This support is using QAT_SW implementation which is only supported in 3rd Generation Intel® Xeon® Scalable Processors family
Please refer Installation instructions on how to build engine with qat_sw.


Fair comparison with Typesense

👋 MeiliSearch team,
I'm Jason, one of the authors of Typesense Search. Our projects started independently at different points in time, but we both have similar goals of being an easy-to-use yet powerful search engine and being a good open source alternative to Algolia. So here's to both of us succeeding in our goals! 🍻
It was brought to my attention recently by a user evaluating Typesense and Meilisearch, that you had mentioned the following statements in your docs about Typesense. I'd like to provide some counter-points to your observations and gently ask that you update your documentation appropriately, so as to make the comparison with Typesense fair and balanced, and more importantly not confuse users.
I'm mostly summarizing my response to the user's query below:

I was browsing on the meiliSearch resources at
https://docs.meilisearch.com/resources/comparison_to_alternatives.html#about-meilisearch
and came across a few comments referencing Typesense that I hope you can comment on:

Typesense uses a default_sorting_field (opens new window)on each document, it means that before indexing your documents you need to compute a relevancy score for Typesense to be able to sort them based on your needs.
On the other hand, MeiliSearch uses a bucket sort, which means that there is a default relevancy algorithm based on the proximity of words in the documents, the fields in which the words are found and the number of typos. And you can still add your own custom rules if you want to alter the default search behavior.

is this true?

This is not true. You could set default_sorting_field to a constant integer value for all records and Typesense will still pull up results based on just the keyword relevancy score it computes automatically by default, just like Algolia and Meilisearch.
However in our experience working with search engines (not just Typesense) and large datasets, keyword-based relevancy scoring only gets you so far, especially in large enough datasets where many records might have similar/overlapping keywords and so end up having the same computed keyword-relevancy score. So you eventually end up having to set up custom ranking rules to tie-break records having the same keyword-relevancy score.
This is where the default_sorting_field comes into play - for tie-breaking, when the keyword-relevancy score is identical for similar looking records. In Algolia and Meilisearch, you'd need to keep an eye out for when this happens as you index records over time and then define custom ranking rules separately to break ties.
In Typesense, default_sorting_field is just a "custom ranking rule". We wanted to elevate the importance of defining a tie-breaking field upfront, especially if you have a good data point already (like say number of page views in the case of indexing web pages, or number of purchases in the case of indexing products, etc). We did this because most search applications eventually end up needing this tie-breaking mechanism and so we took the approach of having users think about it upfront, so search relevancy continues to stay good as you index more and more records.
Now, if your dataset doesn't have a field that could be used for tie-breaking, that's totally fine as well - like I mentioned above, you can just set default_sorting_field to say 10 or any constant integer for all records and effectively turn off this feature. Typesense will then just go off of keyword-relevancy score, just like Algolia and Meilisearch.

... the most similar solutions to MeiliSearch are Algolia and Typesense. The three of them are search-as-you-type relevant search engines.
Typesense is a great tool but is not recommended for achieving a lot of operations with minimal configuration.

Is this true?

This is not true. This one stung a bit, because Typesense's explicit stated goal is to make high-performance out-of-the-box search accessible to all developers with minimal configuration and minimal operational overhead. In fact, Typesense was born out of our personal frustrations with the complexities of using ElasticSearch (and on occasion even Algolia) in the past. So the last thing we want to do is to end up becoming another complicated piece of software to operate.
We've been chipping away at this goal for just over 5 years now, we've heard about teams successfully switching from Algolia to Typesense in production, we try to be on top of any issues and questions, we've deployed 100s of production-grade clusters on Typesense Cloud, and so I'd like to think that we got something right!
It's hard to address a broad unsubstantiated statement like that, but I'm open to any feedback you might have on why you think Typesense "is not recommended for achieving a lot of operations with minimal configuration".

Typesense aims to be simple to set-up and focus on developer productivity and experience by providing a clean API. It is best suited for light projects.

What is meant by a light project?

This is another head-scratcher of a statement! Could you expand on what you mean by "light projects"?
In terms of dataset size, here's a showcase showing Typesense in action on a 32 Million songs dataset: https://songs-search.typesense.org/. Most results should be returned in 50-60ms.
Here's another showcase showing Typesense in action on a 28 Million books dataset: https://books-search.typesense.org/
And another showcase with 2 Million recipes: https://recipe-search.typesense.org/
In terms of concurrency, I've tested a dataset with ~3 million records (Amazon product data) that was ~13GB on disk and I was able to achieve a throughput of 250 concurrent search queries per second on a 16GB RAM, 8-vCPU 3-node Typesense cluster.
So I'd have to strongly disagree with you on your broad statement that Typesense is "best suited for light projects", if I'm interpreting your definition of "light" correctly. Again, I'm open to any and all feedback you have.

My apologies if this comes across as a rant, but having our work over the years be dinged by broad misleading statements, is a little disheartening, especially when it confuses users. I've tried my best to be factual and provide as much context as possible to support my statements. I'd appreciate giving me the benefit of the doubt, if my tone above doesn't convey my genuine intention of wanting to have a well-rounded conversation.

Hey @jasonbosco, thanks for your message. I'll remove the comparison between MeiliSearch and Typesense until we give a new try to Typesense.
I was the one writing the comparison a year ago and I did it based on my reading on Typesense documentation. Apologies from the MeiliSearch team as this was not our intention to confuse anyone.
Have a nice day!

@tpayet Thank you, I'd appreciate that.

I think this is how nations should resolve things
This developer level understanding is what is required in the world!

I read Typesense comparison between your two tools (and Algolia / Elastic).
Would still be nice if Meili would re-add an updated comparison between yourself and Typesense. Right now there is nothing at your comparison page.
Here is typesense comparison (which I'm sure also has some bias, at least in what aspects they decide to compare/include):
https://typesense.org/typesense-vs-algolia-vs-elasticsearch-vs-meilisearch/

Hey @sandstrom,
Thank you for your suggestion, I'll check with the team how we can prioritize this :)
Also, thank you for the rethinkdb inspiration!
Is there any specific area you would like to have better insight about or just a technical comparison from our POV?

Some type of feature/table comparison with ElasticSearch and Typesense would be great.
Without it, the actual process of switching e.g. from ES is unknown. It's obvious that you'll lack things (since you're a less mature product, you'll have fewer features).
If it's easy for me to scroll through a table, see that you have most of the things that we need, and then maybe 2-3 things that are lacking, I can then focus on those and see if there are workarounds (or ping you and ask for an ETA on them), that is very helpful (plus you could get feedback on specifics that's missing).
Right now it's just hazy and I probably won't do this work myself by digging through your docs (since that takes time).
I'd also add some broad stroke details on your direction, i.e. if there are philosophical difference between you and ES/typesense (things like rust, how you're planning to make money, anti-feature (things you don't want to be) etc).

Hello @sandstrom, thanks for your feedback, I opened an issue in the documentation side -> https://github.com/meilisearch/documentation/issues/1044

Specifically, Typesense in their current comparison state that Meilisearch is no production ready. ???
https://typesense.org/docs/overview/comparison-with-alternatives.html#typesense-vs-algolia

Based on the project's documented limitations, it seems to be geared for small dataset sizes, and specifically for cases where high availability is not a requirement. Since it does not have multi-node clustering or node-node replication, it is not production-ready yet.

So what?


Human animation?

Hello,
Thanks for the great work.
I found the recent Omniverse has integrated human animation into scenes.
(see https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/ext_omni_anim_people.html)
Do you have the plan to integrate human animation into OmniGibson?

Hi youngwoo-yoon, thanks for sharing with us the human animation extension from Omniverse. We don't have an immediate plan to integrate human animation into OmniGibson, but we receive feedback from many researchers who are interested in having humans in OmniGivson, so we might be able to incorporate this feature in the second half of this year.

Thanks, looking forward to that!


Fix #2094: Make Scala Arrays instances of Serializable and Cloneable.

Review by @nicolasstucki

That's all. Otherwise LGTM.

Refer to this link for build results (access rights to CI server needed):
https://scala-webapps.epfl.ch/jenkins/job/scalajs-pr/2620/
https://scala-webapps.epfl.ch/jenkins/job/scalajs-matrix-build/3385/
Test PASSed.

Updated.

Refer to this link for build results (access rights to CI server needed):
https://scala-webapps.epfl.ch/jenkins/job/scalajs-pr/2622/
https://scala-webapps.epfl.ch/jenkins/job/scalajs-matrix-build/3387/
Test PASSed.

LGTM


Light client initialization from checkpoint

Motivation
Further simplify light client initialization. Initializing via checkpoint is more natural than from stateRoot+slot. This mirrors weak subjectivity initialization.
Description
Replace initialization from stateRoot with initialization from checkpoint.

@dapplion fyi the mock light client server, used in tests, is severely broken by this PR, since now the beacon namespaced api is used to fetch the header.

@dapplion I disabled the mock light client server test for now, we can re-enable or put together a more comprehensive e2e test with a real beacon node in a future PR


AssemblyReader fails to parse assembly attributes when using []

Opening an issue to track the bug previously in F# Data: https://github.com/fsharp/FSharp.Data/issues/918

This fix was applied


Tutorial does not mention changing the module version if building again

I have walked a number of people through this tutorial and one common mistake they all make is that if they continue to change the code and build repeatedly, they do not change the module version in VS code's module.json. As a result they build a docker image with the same version as the previous one. They create the deployment, but the new container never runs because the module version was not changed.
Request for change: Please add a step that instructs the user to change the version of the module if building again.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 181d5562-f8d7-9c59-85ea-09efbe397f3e
Version Independent ID: dbedd952-a050-286e-c951-4529ec8e87b8
Content: Azure IoT Edge C# tutorial
Content Source: articles/iot-edge/tutorial-csharp-module.md
Service: iot-edge
GitHub Login: @kgremban
Microsoft Alias: kgremban

Hi @dsajanice - Thank you for your feedback! We have assigned this issue to the author to investigate further and take the right course of action.

@dsajanice to verify, the image.tag.version in the module.json file should be incremented, and then does that update the deployment manifest when the solution is rebuilt?

@dsajanice I will try to reproduce this issue and get back to you. The container engine takes care about the local cache. Even it's the same version, the container engine will try compare the local and remote one. I will double check.

@shizn @dsajanice Do you have any updates around this issue, if the docs should be updated or if there was a caching issue?

@dsajanice and @shizn, Checking to see if there are documentation changes that need to be made or if this issue should be closed?

Hey @dsajanice We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


wordpress: 5.2.4 -> 5.3

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/wordpress/versions.
meta.description for wordpress is: '"WordPress is open source software you can use to create a beautiful website, blog, or app"'.
meta.homepage for wordpress is: '"https://wordpress.org"

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 5.3 with grep in /nix/store/j07200l6n08l6yf3wbsa8x7ng9f8k4a4-wordpress-5.3

directory tree listing: https://gist.github.com/7d847a5c0aef5ddaf37c1e79b0f2cbf4

du listing: https://gist.github.com/f23ea3c586e85a27eb9bc97ccebfc28c

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)
1 package rebuild(s)
1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)
First fifty rebuilds by attrpath
wordpress

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/j07200l6n08l6yf3wbsa8x7ng9f8k4a4-wordpress-5.3 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A wordpress https://github.com/r-ryantm/nixpkgs/archive/7cd3628a1549882c457fb80588253474a60dca93.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/j07200l6n08l6yf3wbsa8x7ng9f8k4a4-wordpress-5.3
ls -la /nix/store/j07200l6n08l6yf3wbsa8x7ng9f8k4a4-wordpress-5.3/bin

cc @basvandijk for testing.

@GrahamcOfBorg test wordpress


Set up CI for the free threaded build

I can't see any reason why codespell has suddenly become unhappy. I'd like to work out if it's something here or just bad timing....
Other than that, looks fine.

@da-woods Could you maybe restart the codespell job to see if the failure persists?

It also failed on https://github.com/cython/cython/pull/6219 so I think it's something external that's changed. I'll merge this and ignore codespell for now.


inputTpl from themes.js isn't propagated

I found this issue and I think it is a bug. I'm trying to define my own theme, however, if I define something in object inputTpl in themes.js (in the object with my own theme) there is no propagation and this definition has no effect. Possible fix in EditableController render() function:
change this
//build input
self.inputEl = angular.element(self.inputTpl);

//build controls
[...]
self.controlsEl.append(self.inputEl);

into this
//build input
self.inputEl = angular.element(self.inputTpl);
self.inputThemeEl = angular.element(theme.inputTpl);
self.inputThemeEl.append(self.inputEl);

//build controls
[...]
self.controlsEl.append(self.inputThemeEl);

Is this still an issue?

I just tried the fix listed above and it did not work.  Since the default theme inputTpl is blank, nothing gets displayed.

@eugef please close due to lack of response


PL [3.1.24] Camera not focusing after first scan - Android

Camera is not focusing after user returns to scanner screen.
How to reproduce:
Open the app
Scan valid barcode from the closest distance possible ( e.g.1-2 cm), the camera should focus normaly and keep the picture sharp, after scan you are advanced to product screen
When on Product screen go back
Position the phone much further than in step 1 (e.g. 10-20cm)
Notice the camera is stuck on fixed focus from the previous scan
Video attached.
This happens on Samsung S8+ on Android 9 (OneUI 1.0)
JS packages updates which may have causes this (yarn lock and package json attached):
react-native-camera 3.22.0
react-native 0.62.0

Irrelevant

Irrelevant


misc: Add compression format suffix to the written Parquet file

Adding a compression format suffix to the written Parquet file, and make it easier to determine the compression format of the Parquet file.

@majetideepak Could you help to review?

@liujiayi771 I have mixed opinions on this. I do see the benefit but then the question becomes why not other file details. I also do not see this for files generated by other tools. Ideally, we could make this pluggable.
@Yuhta, @pedroerp Do you have any thoughts on this?

@majetideepak I added this because the Parquet files written by Spark have compressed format suffixes. However, I also tested the Parquet files written by Presto, which do not even have the .parquet suffix.

@liujiayi771 It makes sense that Spark users of Gluten+Velox would like to see Spark filenames and similarly, Presto users would prefer Presto filenames.
Maybe we can add a simple HiveConfig for this? Something like: hive.parquet.write-filename-suffix="presto". "spark" can be another option. Velox already has Spark functions. So using the application name here should be okay imo.

@liujiayi771 @majetideepak I think we need to make it more general than just Presto or Spark, there are many more engines inside Meta using Velox than these two. We can make this a global HiveDataSinkFileNameGenerator for now that takes bucket ID, connector query config, user specified file name (targetFileName), commit strategy, etc.

@liujiayi771 Can we add it from Gluten instead before pass to velox to make Velox more general?

@FelixYBW Currently, the parquet file name written by Gluten is
Gluten_Stage_3_TID_2124_VTID_257_0_3_0946dfb5-f773-42c9-ac8e-d4e70bede02b.parquet
which is generated by
targetFileName = fmt::format(
        "{}_{}_{}_{}",
        connectorQueryCtx_->taskId(),
        connectorQueryCtx_->driverId(),
        connectorQueryCtx_->planNodeId(),
        makeUuid());

I found that Jimmy add a new targetFileName in LocationHandle, we can specify the targetFileName that contains compression kind suffix. However, we will not be able to retain information such as task ID, driver ID, etc., because this information cannot be obtained in the planning phase. We can only generate file names similar to Spark that only contain UUID, for example, Gluten-0946dfb5-f773-42c9-ac8e-d4e70bede02b.zstd.parquet. Do you think it is necessary to retain task ID, stage ID, and driver ID in the file name?
cc @zhztheplayer, @rui-mo, @JkSelf.

@liujiayi771
In versions prior to 3.4 of Gluten, the fileName could be guaranteed to contain task ID information. This is because this information is created when the executor actually retrieves the data to write to the file, and then it is offloaded to the Parquet writer.
However, in versions after 3.4, it is difficult to obtain this information when defining the file name during the planning phase. Therefore, I believe it is not feasible to maintain consistency with Spark in this regard. Unless we modify the file name within the actual Parquet writer, which I think is not very meaningful.

@JkSelf The Parquet files written by Spark only contain UUID in the format part-uuid.parquet. What I mean is, if the files produced by Gluten also only contain UUID, do you think there is a problem with that? Is it necessary to include task ID and driver ID information like we currently do?

@liujiayi771
I understand, thank you for your explanation. It's fine with me to remove the task ID and driver ID.

Agree, let's keep the exact the same parquet name as Spark. Customer may use it in their tools. who know.

Close this PR, as we have handled this on the Gluten side. Thanks all.

Close this PR, as we have handled this on the Gluten side. Thanks all.


Cannot install any distributions, WslRegisterDistribution failed with error: 0x80070002

Your Windows build number:  (Type ver at a Windows Command Prompt)

C:\Users\owen>ver
Microsoft Windows [Version 10.0.18362.10024]

What you're doing and what's happening:

C:\Users\owen>wslconfig /list /all
Windows Subsystem for Linux has no installed distributions.
Distributions can be installed by visiting the Microsoft Store:
https://aka.ms/wslstore

C:\Users\owen>ubuntu1804
Installing, this may take a few minutes...
WslRegisterDistribution failed with error: 0x80070002
Error: 0x80070002 The system cannot find the file specified.

Press any key to continue...

What's wrong / what should be happening instead:

I cannot install any distributions of Linux on Windows.
I should be able to install Ubuntu or any other distributiuon.

I have tried the following suggestions already but they have not helped:

- Uninstall WSL feature, reboot, reinstall WSL feature, reboot
- Uninstall and reinstall (between reboots) all distributions of linux from the
Windows Store, they all show the same error 0x80070002
- I have made sure windows content is set to install to C:\
- I have tried running "sfc /scannow" and rebooting
- I have tried running "DISM /Online /Cleanup-Image /Scanhealth" and
nothing bad was reported

Strace of the failing command, if applicable:

I cannot strace as I am not using WSL yet.

For WSL launch issues, please collect detailed logs.

I have attached `lxcore_kernel.etl` `lxcore_service.etl` `lxcore_user.etl` here:

lxcore_logs.zip

Thanks for the trace.  We are attempting to open the "nul" device and that is failing with ERROR_FILE_NOT_FOUND.  Do you have any third party AV or software that could explain this?

I use BitDefender and Malware Bytes. I have tried compeltely disabling these but it doesn't make a difference.

 We are attempting to open the "nul" device
Genius!
I found the solution. A while back, I was having issues with my non-paged memry pool growing massively over time (usingup to my entire 32gb of ram). At the time, I was suggested to try a nuber of things, including setting the following reg key:
Computer\HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Null\Start to 4
And disabling the null service (sc config Null to disable and not start)
I just changed that reg key backl to 4, and ran from an elevated prompt:
sc config Null start= system
sc start Null
And now I just successfully installed Ubuntu and it works fine.
When I read you saying about 'nul' i immediately tried running echo hello> nul and saw the same message The system cannot find the file specified.
After reenabling the null service, i can run echo hello > nul with no errors.
Excellent! Than you so much for pointing me on the right track.
It might be useful though, for wsl to state the file that it cannot open in these cases, because if I had seen a message earlier like Failed to open 'nul'. The system cannot find the specified file I would have been able to self diagnose this months ago.

Hi @benhillis, could I know how you analyzed the log files to figure out the missing file? I'm trying to figure out a similar error for me. #5107

Used this "wsl --set-default-version 1" ,
It worked for me.

After a few years, I had the same problem, that was just the problem...


nixos/curator: init elasticsearch curator

See elasticsearch curator
Motivation for this change
I needed to use this tool on my NixOS ES cluster
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

@basvandijk I have updated the PR according to your PR, do I need to squash the extra commit?

fixed. should I squash @basvandijk ?

@shmish111 thank you. A squash would be nice.
Any chance you could add a NixOS test for this? Maybe extend the existing ELK test.

@basvandijk do you think it is ok to add it to the ELK test? I feel like it would be unnecessary overhead to have a test that boots up another ES machine when I could just add in some test cases to the existing ELK test. I suppose I would just create an index in ES, run a curator action that should delete it and then check it has been deleted.

@shmish111 extending the existing ELK test like you suggest is the way to go IMHO. It would not only test curator but also elasticsearch itself.

@basvandijk how do I run the nixpkgs tests? I can't find documentation anywhere :-(

@basvandijk I have squashed. Also, I can't get the test passing on master so I can't really do anything until I do that.
uname -a
Linux ip-10-0-4-156.us-east-2.compute.internal 4.14.32 #1-NixOS SMP Sat Mar 31 16:10:43 UTC 2018 x86_64 GNU/Linux

nix-build nixos/tests/elk.nix

This fails to connect to elasticsearch. It seems that elasticsearch has failed to start but I can't work out how to debug in order to find out why.

See the following on how to run tests interactively:
https://nixos.org/nixos/manual/index.html#sec-running-nixos-tests
That will get you into a console on the VM with which you can debug the issue.

@basvandijk I've tried to play around but it's difficult to debug as I can't get into a shell in the running VM. All I can see is the output of startAll etc. I managed to see that elasticsearch does start eventually however it take a long long time for some reason. Then logstash is failing, with "TypeError('Invalid argument: family must be 4 or 6');". Also cat /tmp/logstash.out never appears.
Have you managed to run these tests successfully? I am trying to run these on a NixOS machine on AWS with the image NixOS x86_64-linux 18.03.132946.1caae7247b8 (hvm-ebs) as a t2.xlarge instance.
I think the test maintainers need to look at this, does this need to prevent my PR? I'm happy to add a test but the existing test needs to be fixed first and at the moment that is beyond me.

@basvandijk ok, I've fixed the cfg.enable thing. I've also added a test for curator, this just runs curator after the other tests and checks that the logstash index has been deleted. This index must have been created for the logstash queries to succeed. Can you run these tests because I have no access to a NixOS machine that isn't on AWS at the moment, and I suspect you are correct and that VMs aren't running correctly on AWS instances.

Great we're getting closer! I pushed some more fixes to your branch. In the end you might want to squash all of them.
Could you look into fixing the elasticsearch-curator python package because building it fails with:
nix-build -A python36Packages.elasticsearch-curator
...
Collecting requests-aws4auth>=0.9 (from elasticsearch-curator==5.5.4)
  Could not find a version that satisfies the requirement requests-aws4auth>=0.9 (from elasticsearch-curator==5.5.4) (from versions: )
No matching distribution found for requests-aws4auth>=0.9 (from elasticsearch-curator==5.5.4)

I see we don't have a package for requests-aws4auth so that needs to be packaged.

You might also want to add yourself as maintainer of the module using:
  meta.maintainers = with maintainers; [ shmish111  ];

and add shmish111 to the maintainer-list.nix.

@GrahamcOfBorg test elk.ELK-5 elk.ELK-6

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: tests.elk.ELK-5, tests.elk.ELK-6
Partial log (click to expand)
Cannot nix-instantiate `tests.elk.ELK-5' because:
error: attribute 'ELK-5' in selection path 'tests.elk.ELK-5' not found

Cannot nix-instantiate `tests.elk.ELK-6' because:
error: attribute 'ELK-6' in selection path 'tests.elk.ELK-6' not found

Timed out, unknown build status on x86_64-linux (full log)
Attempted: tests.elk.ELK-5, tests.elk.ELK-6
Partial log (click to expand)
one: running command: cat /tmp/logstash.out | grep flowers
one# cat: /tmp/logstash.out: No such file or directory
one: exit status 1
one: running command: cat /tmp/logstash.out | grep flowers
one# cat: /tmp/logstash.out: No such file or directory
one: exit status 1
one: running command: cat /tmp/logstash.out | grep flowers
one# cat: /tmp/logstash.out: No such file or directory
building of '/nix/store/q5wv95vqb8bifpzyz0qvcvb78r5vddi8-vm-test-run-ELK-5.drv' timed out after 1800 seconds
error: build of '/nix/store/mp71gw6kysf1y7nilnnvz5gq70dyh7di-vm-test-run-ELK-6.drv', '/nix/store/q5wv95vqb8bifpzyz0qvcvb78r5vddi8-vm-test-run-ELK-5.drv' failed


Use go modules, enable GitHub Actions

@amariucaitheodor could you please review the CI code and other changes?

There are 10 unresolved error annotations, other than that looks good to me.

yeah, I know that there are linter errors. this code is 3 years old
thanks!

@ustiugov Ready for review.


Marks parts of an empty arrow function as a syntax error

() => {};
Will sometimes produce Parsing error: Unexpected token ) or Parsing error: Unexpected token =>

Sorry, we cannot reproduce an error which occurs "sometimes". We need a little more information; for example a code snippet containing () => {}; somewhere.
Just for the record: Linting a file, which contains only () => {}; works as expected.


Fix #652 by updating Bolt to work with multiple WorkflowStep middleware

This pull request fixes a bug reported at  #652
See also: https://github.com/slackapi/java-slack-sdk/pull/571
Category (place an x in each of the [ ])

[x] bolt (Bolt for Java)
[ ] bolt-{sub modules} (Bolt for Java - optional modules)
[ ] slack-api-client (Slack API Clients)
[ ] slack-api-model (Slack API Data Models)
[ ] slack-api-*-kotlin-extension (Kotlin Extensions for Slack API Clients)
[ ] slack-app-backend (The primitive layer of Bolt for Java)

Requirements
Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to the those rules.

To reviewers, I will be merging this PR after waiting for one more day. This is a pretty straight-forward change but let me know if you have any concerns/comments.

To reviewers, I will be merging this PR after waiting for one more day. This is a pretty straight-forward change but let me know if you have any concerns/comments.


haskell-language-server: low-tech solution for removing hls plugins

Description of changes
extremely low tech solution to easily removing unwanted plugins

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

cc @maralorn @Profpatsch

Well, hls does not use the hls-...-plugin packages anymore.

      packages = [
        "hls-floskell-plugin"
      ];
``` @maralorn so the current solution here doesn't work?

Correct. This change is really new. It hit haskell-updates this night and since then hydra hasn’t compiled our ghc.

so this hasn't been changed in haskell-updates, yet, either? I will look up how that works then :eyes:

mmh; but the haskell-languag-server.cabal still says something about library hls-stan-plugin, I think I'm missing your point, can you say a bit more?

I think they changed the plugins from being separate cabal packages to being one cabal package with multiple components. That is a significant difference for use because now all of hls builds in one nix-derivation instead of dozens. I think this means that a) what you want to achieve is even easier: Simply disable the stan flag and then hls-stan-plugin won’t be build. However now the hls derivation depends directly on stan and we probably want to set that dependency to null.

I think they changed the plugins from being separate cabal packages to being one cabal package with multiple components. That is a significant difference for use because now all of hls builds in one nix-derivation instead of dozens. I think this means that a) what you want to achieve is even easier: Simply disable the stan flag and then hls-stan-plugin won’t be build. However now the hls derivation depends directly on stan and we probably want to set that dependency to null.

Alas, it's not that simple: ./Setup configure will still try to configure all components, including the ones we don't want to build. There is an option to specify one component to be configured (and built) (see https://cabal.readthedocs.io/en/stable/setup-commands.html#runhaskell-setup-hs-configure), but we don't want to build just one either.
AFAICT the only way of completely removing a plugin and its dependencies (as is needed right now to fix ghc92.haskell-language-server in haskell-updates where ormolu doesn't build) would be to patch the cabal file and mark the undesired component as buildable: False (in addition to disabling the relevant flag and setting the dependency to null).

I had a discussion with michaelpj and they will rework the cabal filenfor the next release so that it will again be easier to disable plugins.


[Bug report] Page.title is not type guarded

Description
---
title: 1
---

---
title: {}

There will be noob users who set titles like this, and page.title (with string type) will have other types, and this may break the application.
Unless we set page.title: unknown , we should guard the types of it.
Reproduction
Not needed
Used Package Manager
npm
System Info
.

It's too verbose to use Record<string, unknown> for all frontmatter properties. We'd better assume that user use it correctly.
BTW, does title: 1 result in number or string?

number, so that's why I want this. No trim, replace or other functions are on numbers, so an errror will be thrown while it bypass typescript checks.


ImportError: No module named zbar

I installed iconv and then zbar-py (pip install zbar-py).
And then I write import zbar into y code.
However, python cannot recognize the library.
The error is simple:
Traceback (most recent call last):
File "C:\Python27\GDrive\LAB\QRCODE_Read.py", line 16, in
import zbar
ImportError: No module named zbar
What can be the source of the error?

This is not giving me a lot to go on. Please provide more information. Did pip install command complete successfully? Could you include the output of that command?
How did you invoke the above script (QRCODE_Read.py) that led to the error?
Is it possible that you have multiple versions of python installed and that you used pip to install zbar into a different python version than was used in the script?


fix: media links in org files not liked to media files

In Org files, links to images and videos will be linked to "/media/" path, similar to
https://github.com/go-gitea/gitea/blob/6a1a6332de5b5000d31ec26efaef6f78a4ee10f0/modules/markup/markdown/goldmark.go#L87-L93
Fixes #12831

Codecov Report

Merging #12997 into master will decrease coverage by 0.00%.
The diff coverage is 55.55%.

@@            Coverage Diff             @@
##           master   #12997      +/-   ##
==========================================
- Coverage   42.59%   42.59%   -0.01%
==========================================
  Files         671      671
  Lines       73625    73632       +7
==========================================
+ Hits        31363    31364       +1
- Misses      37183    37185       +2
- Partials     5079     5083       +4

Impacted Files
Coverage Δ

modules/markup/orgmode/orgmode.go
75.51% <55.55%> (-3.07%)
:arrow_down:

modules/util/timer.go
42.85% <0.00%> (-42.86%)
:arrow_down:

modules/git/repo.go
46.70% <0.00%> (-1.02%)
:arrow_down:

models/unit.go
49.31% <0.00%> (+2.73%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c6da033...036766c. Read the comment docs.

Thanks for the PR - this looks like it would solve the problem - however, as a R is an io.Writer we should just get fmt.Fprintf to write to it directly instead of using the r.WriteString method.


Fixed: Removed PO reallocation job from UI(#32129kj)

Related Issues

Closes #282
Short Description and Why It's Useful

Removed reallocation section from preorder page as it do not solve any problem and can create confusion to the user
Screenshots of Visual Changes before/after (If There Are Any)

IMPORTANT NOTICE - Remember to add changelog entry
Contribution and Currently Important Rules Acceptance

[ ] I read and followed contribution rules

We should also remove data for this job from the env files etc.

We should also remove data for this job from the env files etc.

Done sir


Returning Bloomberg blpapi errors.

It would be very useful to be able to use the errors issued by blpapi (e.g., ConnectionError, etc).  Is there a way to access these through the API.  An example would be very useful.

Works, thanks.  Part of the issue is my relative newness with python.  Best.


Wake timer causes system wake from sleep every 3 minutes

Microsoft PowerToys version
0.37.2
Running as admin

[X] Yes

Area(s) with issue?
General
Steps to reproduce
Put computer to sleep
✔ Expected Behavior
Not waking up my computer from sleep with no reason
❌ Actual Behavior
Using the Event Viewer, find that your PC has resumed from sleep every 3 minutes because of PowerToys.exe
Other Software
No response

thanks for reporting, looks like this was reported in #11467


overrideDerivation doesn't propagate passthru which was added by extendDerivation

Issue description
It's a weird use case (which I'll describe below), but basically: if I use lib.overrideDerivation on a derivation which has already been extended (via lib.extendDerivation), it drops the extended passthru attributes and reverts to the original derivation's passthru. This is still the case when the original derivation doesn't have any passthru attributes.
Steps to reproduce
$ curl https://gist.githubusercontent.com/timbertson/1d7beddd3a647ba8fcb589747fa81240/raw/85c5363a07c751bbc3845f051774ea29c107a80c/testcase.nix > testcase.nix

# sanity checking:
$ nix-instantiate --eval testcase.nix -A emptyExtended.attr
1

$ nix-instantiate --eval testcase.nix -A simpleExtended.attr
2

$ nix-instantiate --eval testcase.nix -A simpleOverride.attr
1

$ nix-instantiate --eval testcase.nix -A hasOtherAttrExtended.attr
3

# extend merges original and extended `passthru`
$ nix-instantiate --eval testcase.nix -A hasOtherAttrExtended.someOtherAttr
1

# but extend -> override ends up with only the original derivation's `passthru`
$ nix-instantiate --eval testcase.nix -A hasOtherAttrOverride.attr
error: attribute 'attr' in selection path 'hasOtherAttrOverride.attr' not found

# nix-instantiate --eval testcase.nix -A hasOtherAttrExtended.passthru
{ someOtherAttr = 1; }

$ nix-instantiate --eval testcase.nix -A hasOtherAttrOverride.someOtherAttr
1

Technical details

system: "x86_64-linux"
host os: Linux 4.17.17-100.fc27.x86_64, Fedora, 27 (Workstation Edition)
multi-user?: no
sandbox: no
version: nix-env (Nix) 2.0.2
channels(tim): "nixpkgs-18.09pre139886.183f56f2d62"
nixpkgs: /home/tim/.nix-defexpr/channels/nixpkgs

Why am I doing something so weird?
If I were hand crafting the derivations, I wouldn't. However in my case:

the original derivation is generated by the opam2nix utility, as a .nix file on disk. It includes one passthru attribute. This is a pure recipe for building a package, but requires compatible versions of all dependencies to be injected as function args.
this nix file is imported, called and extended by the nix function buildOpamPackage in opam2nix-packages. For example, it adds a packages attribute containing the transitive dependencies of this package.
In development later, I call overrideDerivation on the result of buildOpamPackage to override the src attribute only in development. This way I can build my local checkout without modifying the official derivation.

Yes, it does.

I came up with a fix that seems to work, but I don't know if it's a good idea: #45972

I wonder if foo.overrideAttrs(old: { passthru = old.passthru // ...; }) would work instead of extendDerivation?

I tried that too, it doesn't seem to set the actual derivation attrs (i.e. drv.passthru.attr would work, but not plain drv.attr)

I can't replicate that behaviour:
nix-repl> hello.foo
error: attribute 'foo' missing, at (string):1:1

nix-repl> hello2 = hello.overrideAttrs (old: { passthru = (old.passthru or {}) // { foo = (old.passthru.foo or []) ++ [1]; }; })

nix-repl> hello2.foo
[ 1 ]

nix-repl> hello3 = hello2.overrideAttrs (old: { passthru = (old.passthru or {}) // { foo = (old.passthru.foo or []) ++ [2]; }; })

nix-repl> hello3.foo
[ 1 2 ]

This is not a bug, WONTFIX.

hasOtherAttrExtended = lib.extendDerivation true { attr = 3; } hasOtherAttr;
hasOtherAttrOverride = lib.overrideDerivation hasOtherAttrExtended (orig: {});

The problem is that you're only supposed to call extendDerivation over plain builtin derivations. It's not designed to be called over its own result, in particular, over the results of stdenv.mkDerivation and overrideDerivation. (Nix lacks types, so the problem here is that we can't express that fact in the code explicitly without losing some good chunk of performance (extendDerivation is a very hot function).) Note that if you look at the source of overrideDerivation carefully, you'll notice that it unwraps the given derivation's original drvAttrs, and rewraps that into new builtin derivation before using extendDerivation. That's not an accident.
In your use case, if you want to override passthru, you are expected to do something like

hasOtherAttrOverrideP = hasOtherAttr.overrideAttrs (orig: { passthru = orig.passthru // { attr = 3; }; });

The documentation can be improved, though, I agree.

@dezgeg Hmm, I must have mistyped, your workaround does appear to work after all. Thanks for that, I will try using it as a workaround.
@oxij it was my understanding that stdenv.mkDerivation produces a plain derivation, because it returns you the result of calling the builtin derivation. What distinguishes a plain derivation from one what has been built using a utility function like this? Is it that passthru is an attribute given to mkDerivation, but that it isn't stored on the underlying derivation object?

What distinguishes a plain derivation from one what has been built using a utility function like this? Is it that passthru is an attribute given to mkDerivation, but that it isn't stored on the underlying derivation object?

Yes. stdenv.mkDerivation attrs == extendDerivation a passthru (derivation someOfTheAttrs) ~= (derivation someOfTheAttrs) // passthru (where passthru is attrs.passthru and some other stuff like e.g. overrideAttrs).

Is there any harm in also adding passthru as a native derivation attribute? It seems like it would make this work, and wouldn't affect the derivation's hash, just like any other passthru attribute. Do we want this to be unsupported, or do we just happen to not support it at the moment?

It seems like it would make this work, and wouldn't affect the derivation's hash

Everything you supply to derivation function does affect the hashes. Maybe I misunderstood what you were trying to say.
Think of derivation attr as being equivalent to attr // { drvPath = drvhash attr; outPath = outhash attr; } with side-effects.

Maybe I just don't understand how passthru works - the general passthru mechanism adds attributes to the result without changing the derivation's output hash, doesn't it? So can't that be extended to add a passthru attribute as well?

Maybe I just don't understand how passthru works - the general passthru mechanism adds attributes to the result without changing the derivation's output hash, doesn't it?

Yes.

So can't that be extended to add a passthru attribute as well?

Yes. It does that too already. See the bottom of pkgs/stdenv/generic/make-derivation.nix.

Is there any harm in also adding passthru as a native derivation attribute?

However, now I think I see what you meant there. You see, drv = derivation x implies drv.drvAttrs == x, there's no middle ground there, either it's in the drvAttrs and influences the hash, or it does not and hence overrideDerivation doesn't know about it. Introducing a middle ground like your PR does will multiply nix memory consumption for the whole of nixpkgs.
However, I do see your point ("it's strange that lib.overrideDerivation is in a public API, but it does something strange to passthrus if you naively consider them to be part of derivations") and I do agree that something needs to be done about that. I'm inclined to say "lets just replace all overrideDerivation calls with overrideAttrs" and deprecate and then remove the former (which is best, semantically speaking), but I'm not sure if overrideAttrs can be made as efficient as overrideDerivation. I thought through four immediate ideas on how to do that, but all of them turned out to be meh. Now I have to wait for the background brain to come up with something better.


Accessing dependencies in background tasks

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the FastAPI documentation, with the integrated search.
[X] I already searched in Google "How to X in FastAPI" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to FastAPI but to Pydantic.
[X] I already checked if it is not related to FastAPI but to Swagger UI.
[X] I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code
from fastapi import BackgroundTasks, FastAPI, Depends

app = FastAPI()

def add_some_record(email_id: str, db = Depends(get_db)):
    record = Record(parent=email_id)
    db.add(record)
    db.commit()

@app.post("/send-notification/{email}")
def send_notification(email: str, background_tasks: BackgroundTasks, db = Depends(get_db)):
    email_id = db.query(Email).filter(Email.address == email).first().id
    background_tasks.add_task(add_some_record, email_id=email_id)
    return {"message": "Record added in the background"}

Description
Hey there, I want to be able to access my database dependency in my background task, I tried to do it like in the example code but this method doesn't work, and I couldn't find anything in the docs about the interaction of deps and background tasks, is there a way to achieve this? should I just pass in the db session to the task and will stay alive until the task is done? it would be nice to have a best practice written somewhere
Operating System
macOS
Operating System Details
No response
FastAPI Version
0.76.0
Python Version
3.10.0
Additional Context
No response

You can't use Depends but you can try using a context manager to open a database connection, there are many ways.
I would use:
def add_some_record(email_id: str):
    with SessionLocal() as db:
        record = Record(parent=email_id)
        db.add(record)
        db.commit()

You can read dependencies-with-yield
You could background_tasks.add_task(add_some_record, email_id=email_id, db=db)

Indeed, I ended up just passing 'DB' from the route function to the task, and it works great 😃
Regardless, does it make sense to allow background tasks to receive Dependencies ? Or is it breaking FasAPI's paradigm?

Indeed, I ended up just passing 'DB' from the route function to the task, and it works great 😃
Regardless, does it make sense to allow background tasks to receive Dependencies ? Or is it breaking FasAPI's paradigm?

It is kinda breaking Fastapi's paradigm. Anytime you pass an param: ParamType = Depends(some_callable) in a path operation function, the variable param is actually holding a Depends object, holding reference to the callable. This path operation function is then handled by the preceding decorator (like app.get("/path")). When your endpoint is called, the decorator first resolves the Depends object and then passes the outcome to the actual path operation function (in the above example, it would actually pass an object of type ParamType.
So, from an Fastapi perspective, it would not make sense to have arbitrary functions (which is what are put into background tasks) be first resolved for dependencies. However, you could mimic it yourself, by creating a decorator yourself that solely resolves dependencies for any given function.

However, you could mimic it yourself, by creating a decorator yourself that solely resolves dependencies for any given function.

How?
There is no way to grab the dependency graph and ask for a dependency?
If I have an app level dependency I don't want to be passing it through every instance constructor of my project.


feat: add WebDriverIO support and tests

Playwright test not being added, we're testing in Chrome and Edge.
The Playwright detection on hold, we need to find the way to include it.
closes #5
closes #7

👍


Domain Join a VM for AVD

I am trying to use the Microsoft.Compute/virtualMachines/extensions bicep to domain in a VM to AADS.
resource vm_joindomain 'Microsoft.Compute/virtualMachines/extensions@2018-10-01' = {
name: '${name}/joinDomain'
location: location
properties: {
publisher: 'Microsoft.Compute'
type: 'JsonADDomainExtension'
typeHandlerVersion: '1.3'
autoUpgradeMinorVersion: true
settings: {
name: domain
ouPath: ouPath
user: administratorAccountUsername
restart: 'true'
options: '3'
}
protectedSettings: {
password: administratorAccountPassword
}
}
}
Getting this error:-
\"ExtensionNotFoundForCPUArchitecture\",\r\n    \"message\": \"Extension with publisher 'Microsoft.Compute', type 'JsonADDomainExtension', and type handler version '1.3' for CPU Architecture 'x64' was not found in the extension repository.\"\r\n  }\r\n}"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"}]}}
What am I doing wrong? - cant find much in the way of help tbh.

@gsuttie -- I think I saw on twitter that you got this resolved. Would you mind sharing what the final resolution was?


Something weird is happening with unicode

e.g. on https://granary.io/url?hub=https%3A%2F%2Fpubsubhubbub.superfeedr.com%2F&input=html&output=atom&url=https%3A%2F%2Funrelenting.technology%2F — what's happening with the encoding?
see https://github.com/myfreeweb/sweetroll2/issues/1

ugh. thanks for reporting, and sorry for the trouble!
something weird indeed. looks like this may be some interaction between your site and granary. for comparison, here's a similar granary conversion from HTML to Atom with a unicode char that makes it through fine, afaict: https://granary.io/url?input=html&output=atom&url=https%3A%2F%2Fsnarfed.org%2F2019-06-29_37722
i noticed your site returns Content-Type: text/html and has a <meta charset="utf-8"> tag in the HTML. that's fine, obviously, and granary should handle it, but maybe it isn't? i'll investigate more.

Yeah, that makes sense, usually only browsers are smart enough to handle that. I'll fix the server config.

finally fixed this on my end too. motivated a big (but good!) refactoring. thanks again for the nudge!


[Bug]: When a file is modified, its mtime attribute is not updated

Contact Details
No response
Is there an existing issue for this?

[X] I have searched all the existing issues

Priority
high
Environment
- CubeFS version: all
- Deployment mode(docker or standalone or cluster): all
- Dependent components:
- OS kernel version(Ubuntu or CentOS):
- CPU/Memory:
- Others:

Current Behavior
When a file is modified, its mtime attribute is not updated. This is not compatible with standard POSIX semantics and will cause confusion for users.
Expected Behavior
When a file is modified (not appended), its mtime attribute is updated.  This feature can be configured to enable
Steps To Reproduce
No response
CubeFS Log
No response
Anything else? (Additional Context)
No response

i want to resolve this , please describe how to reproduce it


All of the admin list filters need to be updated to use Django ORM

All of the admin list filters need to be updated to use Django ORM for filtering rather than pure Python. This should result in performance improvements on admin side.

I would say this is mostly done except for in position_role_list_filter.py that has some rather complicated logic that just doesn't seem worth implementing in ORM.
   def queryset(self, request, queryset):
        """
        Returns the filtered queryset based on the value
        provided in the query string and retrievable via
        `self.value()`.
        """

        # Compare the requested value (either True or False)
        # to decide how to filter the queryset.
        filtered_person_ids = []
        for person in Person.objects.all():
            if person.is_current_member is True and self.value() is None:
                filtered_person_ids.append(person.id)
            elif person.is_alumni_member is True and person.is_current_member is False and self.value() == "past_member":
                filtered_person_ids.append(person.id)
            elif person.is_current_collaborator is True and self.value() == "current_collaborator":
                filtered_person_ids.append(person.id)
            elif person.is_past_collaborator is True and self.value() == "past_collaborator":
                filtered_person_ids.append(person.id)
            elif person.is_current_member is False and person.is_alumni_member is False and\
                    person.is_current_collaborator is False and person.is_past_collaborator is False and\
                    self.value() == "other":
                filtered_person_ids.append(person.id)
            elif self.value() == "all":
                filtered_person_ids.append(person.id)

            # also check for graduated phd students
            if self.value() == "graduated_phd_student" and person.is_graduated_phd_student:
                filtered_person_ids.append(person.id)

        return queryset.filter(id__in = filtered_person_ids)


Set the correct parent docObject

Coverage remained the same at 96.465% when pulling 8858af4200d3905ac6c8a43e2d5a935e42cfc13f on doc-place into d942044d221f18da40d61e7a45d39dddd4e00896 on master.


Error message: slice index 0 of dimension 0 out of bounds.

Hello,
Thank you for your interesting paper. While executing the code, I encountered the error below:
File: general_child.py
Line: 362 - res_layers.append(tf.cond(tf.equal(skip[i], 1), lambda: prev_layers[i], lambda: tf.zeros_like(prev_layers[i]))).
ValueError: slice index 0 of dimension 0 out of bounds.
Could you please help.

Hi @YasminaJaafra : I got the same problem when trying to run the code.
Have you figured it out on how to fix this error ? Thank you


pkg: archive: don't fail Untar if xattrs are not supported [rhel7-1.10.3]

@rhatdan @maxamillion @mrunalp PTAL
This is probably meant as just a workaround - upstream issue is tracking this whole thing here https://github.com/docker/docker/issues/20331
Fix https://bugzilla.redhat.com/show_bug.cgi?id=1343702
Since certain filesystems don't support extended attributes, ignore
errors produced (emitting a warning) when attempting to apply extended
attributes to file.
Signed-off-by: Aleksa Sarai<EMAIL_ADDRESS>

Merging anyways we should open discussion upstream.


Fix undefined company prop type error.

 Fix Company component TypeError when company hasn't been fetched yet from API.

@laurandarius Please resolve conflicts.

@lgnstn updated.


add support for colorized,grayscale,raw depth map; by default enable …

…colorized depth

@GergelySzabolcs - thanks. Is the color depth map the default?

Ha ... saw that is in the truncated portion of commit message. thx!

@GergelySzabolcs - I'm seeing 3-5 second latency w/COLORMAP_JET+COLORMAP_HOT on the RPi Compute. Grayscale has much lower latency.
@Luxonis-Brandon @Luxonis-Brian - what's an acceptable latency, in your opinion? The colorized options feels pretty slow as a default.
👉  🎥 Video sample.

That's what the PI is capable of in terms of applying a color LUT. @itsderek23

I think the previous implementation was much faster.  I did take some video of it back then and will share below.
So here's what the depth_color_h looked like as of February 3rd, 3:51pm Colorado time (GMT-7:00, in case that helps for tracking down the commits):
https://photos.app.goo.gl/EdgWJMAasHudjjRe9
https://photos.app.goo.gl/ANqghT4KNXt9bEXA6
https://photos.app.goo.gl/cdMQmr3bdEuAT4VVA
Seemed to be about 7-8FPS, but I'm unsure on the actual latency, but about 1 second maybe?
So I'm thinking the way the format was passed from the Myriad X makes it easier/faster for the Pi to convert it?
This was run on a BW1097 - so the Pi Compute Module 3+, which is the same processor/specs as the Raspberry Pi 3B+ Pi.
This is the same hardware that @itsderek23 is testing on as well.
So I think we need to do what's necessary to get back to this performance we saw with this Feb. 3rd version of the codebase.  This new version of colorized depth (video here) is too slow to be usable for checking calibration and how the depth picking up objects.
The 7-8FPS w/ whatever latency it is seems totally fine for this purpose however.
Thoughts?
Thanks,
Brandon

Agreed. I'm on my way of debugging.

Perfect, thanks @GergelySzabolcs !

@GergelySzabolcs found that performance was better when running bc91e1f60b1fb3549994590651621fdab763acb3 w/our current internal depthai devices_auto_reset branch. This PR is actually identical to that commit.
I'm able to get a low-latency depth_sipp stream by also running a left stream:
python3 test.py -co '{"streams": ["left","depth_sipp"]}'

The high-latency when using just the depth_sipp stream might be caused by the high frame rate triggered by using a single stream. I'll see FPS > 30 w/the single stream vs. 5 FPS when using both streams. For testing calibration results, low latency > high FPS. My hunch is the host-side Python logic for adding color adds significant overhead because it is being applied to a higher number of frames. I'll also see "data queue is full" which happens when the Python code can't keep up.
Next steps:

Update --help output to include the left stream in the sample for calibration
Update our calibration docs to use the left stream as well.

We have an internal FPS limit branch in progress that might help us with displaying just the single stream w/low latency, but using 2 streams seems like an acceptable workaround.
Does this make sense @Luxonis-Brian and @Luxonis-Brandon?

To use JET color mapping:

Comment out L204 of test.py
Uncomment L205

...if the usage of different depth visual outputs becomes common (or code toggling becomes painful), we can make this an argument when running the script.


floyd run: add --follow option

My workflow involves checking job output straight after launching the job. Having to run floyd run then floyd logs has a little friction to it, so here's a patch for floyd run to automatically follow logs after having launched the job.
Relies on https://github.com/floydhub/floyd-cli/pull/164 to be useful.
(I feel a little dirty about this because http://click.pocoo.org/6/advanced/#invoking-other-commands says that invoking Click commands from other commands is "a pattern that is generally discouraged with Click, but possible nonetheless". I'm open to suggestions if there might be a cleaner way of doing it.)

This is good feature to add. One thing we can improve is to factor out the code from the log command into a python function that takes job id as argument. Then you can call the same function in both log and run command. You probably want to wait for https://github.com/floydhub/floyd-cli/pull/164 to be merged first ;)

OK, here's a new version with the key things that run needs to do refactored into separate functions.
Other small changes:

In the code which does the log following, switch the method which keeps track of how much of the fetched log we need to print. Instead of storing the current shell output, just store the current index in the log file.
rstrip all log outputs to avoid extra newlines.
log_url if only used by the if url statement, so move it into the if statement's block.

Everything else looks good to me, thanks for the PR! Looking forward to release this feature to all other users :)

Awesome, squash merged :)


Implement inventory management logic from gameplan

Description
As a developer, I would like to ensure that the business logic outlined in the gamemplan doc is implemented.
Acceptance Criteria
Reference: [Done-Done Checklist] (https://github.com/Microsoft/code-with-engineering-playbook/blob/master/Engineering/BestPractices/DoneDone.md)

[ ] Should be able to demonstrate a realistic simulation of aggregated inventory count for stores based on inventory events

Resources
Technical Design Document
https://github.com/ianphil/spike-inventory-de/blob/master/docs/gameplan.md
Tasks

[ ] Implement logic and constraints found in gameplan doc
[ ] Demonstrate with sample simulation (in the form of a test or otherwise)

I think we should add a couple of items to the game plan.

The collections are static.
The store id is a property.
Example data schema for: S, D, H, L collections


Using pimoroni libraries outside of examples tree

If I want to code a programme based on your examples but outside of the pimoroni source tree, how would I include the libraries?
eg:-
-pimoro-pico
-pico-sdk
-my_project_directory
I think this would be useful for (ancient) developers like me not overly familiar with cmake

I have solved this by adding:-
include(../pimoroni-pico/libraries/pico_explorer/pico_explorer.cmake)

To CmakeLists.txt

The Pico Boilerplate is a good starting point for using external libraries in your own project - https://github.com/pimoroni/pico-boilerplate
It also has the necessary GitHub action setup and so forth to continuously build/test your code.