Can't run require optimizer

sebastien@sebastien-xps:build (seb/front/editor-rework *$)$ nodejs r.js -o stample.build.js
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/admin.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/annotator.min.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/bootstrap-wysihtml5.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/bootstrap.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/clipping.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/clippingframe.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/colorpicker/colorpicker.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/email.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/joyride.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/jquery-ui/jquery-menu.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/jquery-ui/jquery-ui-OLD.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/jquery-ui/jquery-ui.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/login.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/loginOLD.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/mobile.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/print.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/prism.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/signup.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/style.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/tipsy.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/tomorrow.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/tomorrow.min.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/css/wysiwyg.css
Optimizing (standard.keepLines) CSS file: /home/sebastien/Desktop/Stample/stample-web/public/public-min/images/sprites/sprite.css

Tracing dependencies for: mainDesktop
Error: ENOENT, no such file or directory '/home/sebastien/Desktop/Stample/stample-web/public/public-min/js/eventie/eventie.js'
In module tree:
    mainDesktop
      router
        appView
          startUploadProcess
            uploadProcessFiles
              uploadFile
                createHiddenEditor
                  stampleFormView
                    imagesloaded

Error: ENOENT, no such file or directory '/home/sebastien/Desktop/Stample/stample-web/public/public-min/js/eventie/eventie.js'
In module tree:
    mainDesktop
      router
        appView
          startUploadProcess
            uploadProcessFiles
              uploadFile
                createHiddenEditor
                  stampleFormView
                    imagesloaded

    at Object.fs.openSync (fs.js:427:18)

It works fine otheriwise but I can't make it work with the optimizer
The lib is simply declared as "imagesloaded": "lib/imagesloaded", in paths.
I use the packaged version. eventie and EventEmitter are inside the js file.
I don't know if it's a misuse on my side or a bug in your lib or in require optimizer. Just wanted to warn you of this integration problem :)

I've added docs for RequireJS: http://imagesloaded.desandro.com/#requirejs Take a look and report back if you continue to have issues.


Added missing tests to kernel32 and DbgHelp.

Fixes #224
(Tests created on my Mac)

Wait why doesn't the pipeline test ns2.0 as well so it can expose the failures on these when targeting that TFM?

why doesn't the pipeline test ns2.0 as well

netstandard2.0 isn't a runtime. It's just a standard. Tests can only run on a runtime like .NET Core, .NET Framework, or mono.
It looks like we only run tests on .NET 5, regardless of OS. For the code generation tests that seems adequate since I do not expect the runtime to influence the code generation itself. But we could add more runtimes if necessary.

We don't want a test for every API to make sure it generated. We have tests for specific APIs that tend to be problematic and throw exceptions during generation, but ultimately those are redundant with our "generate everything" tests. Why do we need these new tests?

Because a lot of those api's I added are technically some of the common ones I find that people want to use on the internet, myself included.

I see this is why I have my issue on my codebase not generating this enum:
  Failed GeneratorTests.GetMINIDUMP_TYPEGeneration [126 ms]
  Error Message:
   Assert.True() Failure
Expected: True
Actual:   False
  Stack Trace:
     at GeneratorTests.GetMINIDUMP_TYPEGeneration() in D:\a\1\s\test\Microsoft.Windows.CsWin32.Tests\GeneratorTests.cs:line 248

Which is odd because it should be constant across all cpu specific builds of Windows 10 that has the latest dbghelp.dll.

Because a lot of those api's I added are technically some of the common ones I find that people want to use on the internet, myself included.

To be clear, I'm not arguing that we shouldn't generate those APIs. We do. I'm just saying they already have test coverage as part of the full-generation test. If we have a bug where certain APIs are simply missing in the generated code, then that's certainly worth taking a look at. It sounds like that's what you're saying, so I'll take another look.


mobile network status icon is showing on devices withnout sim card

i have tablet withnout sim card support and mobile newtwork icon is still showing
change it to: if device donthave simcard it will be invisible

Which device are you seeing this issue with?  On two of my devices that don't support SIM cards (Nvidia Shield and Pixelbook), the mobile network icon is successfully hidden.

Huawei media pad t3 10


Invoking Dart code from service message OOB is dangerous

As uncovered by developer_extension_test- invoking Dart code from a service message OOB can break the running system.

Others:

Isolate, Library, Class, and Instance evaluate methods.
Disabled dart io support.
Dart based metrics.

Service extensions are safe now.


Update downloadGJMessage20.md

Fix description

Naaaaaah
downloading a message ofc blocks the user as well, you don't want any responses after all 😉


chore: bumped driverkit to v0.17.1.

What type of PR is this?
Any specific area of the project related to this PR?
/area build
What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

update: bumped driverkit to v0.17.1.

@FedeDP: The label(s) area/build cannot be applied, because the repository doesn't have them.

In response to this:

What type of PR is this?
Any specific area of the project related to this PR?
/area build
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
update: bumped driverkit to v0.17.1.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

There is not a label identifying the kind of this PR.
Please specify it either using /kind <group> or manually from the side menu.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


code implementation is susceptible to replay attacks

Consider the following test cases to ensure the flow is hardened against such attacks:

Limited validity period: The code should only be usable for 5 to 10 minutes for the time of issue.
Reuse: The code should only be usable once. Consider race conditions on submission as well.
replay attacks: Upon attempted reuse of the code all access tokens previously issues based on the code should be revoked, as it is an indicator of code compromise.

@seriouszyx @Steve0x2a @ComradeProgrammer

@sam-blackfly if you are using Casdoor, please help us input here: https://github.com/casdoor/casdoor/issues/361 , thanks!


Fix a couple of GCP docs links

This change is

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/9c3a768c4d6970b871e9b192e3dde63181c4b9c6/

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/9c3a768c4d6970b871e9b192e3dde63181c4b9c6/


📃: Adding Music record manager

:red_circle: Title : Music record manager
:red_circle: Aim :Its a app that allows users to add and manage music records, capturing details like record name, artist, and genre.
:red_circle: Tech Stack Used : HTML, CSS, and JavaScript

:white_check_mark: To be Mentioned while taking the issue :

Full name :  Aditi
What is your participant role?  GSSOC EXT

Happy Contributing 🚀
All the best. Enjoy your open source journey ahead. 😎

Title : Music record manager
Aim :Its a app that allows users to add and manage music records, capturing details like record name, artist, and genre.
Tech Stack Used : HTML, CSS, and JavaScript
Name: Sanat Kumar Gupta
Participation: Gssoc'24 ext, hackoctober
I would like to implement this. I have tried this on my local device.

Assign this to me.
Thank you!!


Fix nil pointer exception looking up no workers

Customer reports a panic in:
github.com/openshift/hive/pkg/controller/clusterprovision.(*ReconcileClusterProvision).getWorkers(_, {{{0x4581c54, 0x11}, {0xc003588a20, 0x14}}, {{0xc00562e9d6, 0x9}, {0x0, 0x0}, {0xc00562e9e0, ...}, ...}, ...})
        github.com/openshift/hive/pkg/controller/clusterprovision/clusterprovision_controller.go:611 +0x263

(This is running hive commit cef3cb07)
This can happen if the install-config.yaml is improperly formatted in any of several ways:

There is no compute section.
The compute section has no item with name: worker.
That section has no replicas field.

This commit adds some simple nil checking and reports zero workers in those scenarios.
HIVE-2200
(cherry picked from commit 243608d0131a229d7cf0a4c19f852068a7ef1434)

/assign @jstuever
Trivial manual cherry-pick (had to reorganize imports in the test file).

/cherry-pick mce-2.1

/cherry-pick mce-2.1


Offline support?

It seems there's no support for enqueuing notifications for later when the app is offline. Is there any plan to add this in the future?

How exactly would that work. And how is that related to JS errors?

Client-side analytics platforms (one example being Localytics) often have error handling on their uploads, such that if the upload fails, it's saved for a later upload in localStorage (but perhaps just storing in memory is all we need). Then at some point in the future, it tries to send a batch of all prior unsent payloads. Bugsnag could adopt a similar approach.
It seems like an important feature just because, without it, you're losing data on any exceptions thrown if the user has a flaky connection at the time of the exception (mobile apps are particularly prone to this).

I see what you mean. To be honest I'd find it unlikely to be implemented into the library itself, but I like the idea. It would seem that it's something that can be implemented on a per-app basis, based on your own specifications. Having such a system would add additional 'bloat' to the existing js file, for something that would probably be an opt-in feature anyway.
Just my opinion, sorry. But I'd never thought about it before now. Very smart.

For me, this is actually very important. Do any other Bugtracking Services support this for JavaScript-based Errors?

This will also enable an easier integration with Cordova/Ionic applications

Notable: https://developers.google.com/web/updates/2015/12/background-sync

So I've put together some code for supporting in-memory error event retrying. There are a few of things to note, which need some feedback (cc @wordofchristian @kattrali).

Right now, the notify.bugsnag.com/js endpoint responds with an empty response. Meaning the onerror event on the Image object (when using the default notifyHandler) is fired instead of the onload event (because a zero byte image isn't valid).
This would need to be changed so we can retry requests for users with flakey connections, where they do have "online capability", but no actual connectivity.
XHR doesn't have this issue, as you can extract the response status from the XHR object, and determine whether the request was successful or not.
I still have yet to implement support for IE8, which does have online/offline events & support, but triggers events off the document.body rather than window.
There is currently no limit on how many events can be queued in-memory. Do you reckon it's worth adding one? If so, how large/should it be configurable?

So, just an update, my PR now includes support for using sessionStorage/localStorage for storing the errors to be sent at a later time. I still need to do some more testing (for bugs and browser support), but I'd love to get some feedback on this 😄
cc @kattrali

There's actually a huge bug I found this week that renders our servers useless if it accidentally goes offline. This would definitely solve that.
https://github.com/bugsnag/bugsnag-js/issues/628

Hi @mathiasrw - yes this is something that we are looking into and hope to implement in a future release :)

Did this get released? Would need this as well!

Hi @cqcmdwym - this is still something on our backlog, and I don't have a firm ETA at the moment. As Xander said above we'll notify this thread as soon as there's any update.

@luke-belton great, looking forward to it.

Any update on this feature since 2021?

Hi @Taxman972, offline support for JS events is still on our backlog, and while I don’t have an ETA on this, we will continue to keep this thread updated once there are any further updates.


Extend widgetpreview to contain preview for more components

The widgets preview is very useful for a) quickly checking if widgets are rendered correctly and b) to give new developers an idea what form components we have available.
This should be extended to most other components as well. The widgetpreview view should be changed to be named "componentpreview" and contain five tabs. Each tab contains examples of different components. I suggest we use one tile (not expandable) per component class. One tile can contain multiple examples of the same component with different options. And each tile should have a two-column layout: In the first column we show the component examples and in the second we display the docstring of the components init method. That way we can also easily see which components are still missing good docstrings.
Here is the list of tabs and which modules should go into which tab:

Layout components

grid.py
tabs.py
modal.py
tile.py

Informational components

icon.py
tag.py
loading.py
progress_indicator.py
tooltip.py
notification.py

Interactive components

button.py
content_switcher.py
toggle.py
overflow_menu.py
pagination.py
searchbar.py

Datatable

datatable.py

Form components

content of current widgetspreview

Because this one is a bit long and detailed, it is best to keep tracking the progress so we can all see how it goes.

Layout components

[x] grid.py
[ ] tabs.py
[ ] modal.py
[ ] tile.py

Informational components

[ ] icon.py
[ ] tag.py
[ ] loading.py
[ ] progress_indicator.py
[ ] tooltip.py
[ ] notification.py

Interactive components

[ ] button.py
[ ] content_switcher.py
[ ] toggle.py
[ ] overflow_menu.py
[ ] pagination.py
[ ] searchbar.py

Datatable

[ ] datatable.py

Form components

[x] content of current widgetpreview


[tool_crash] FileSystemException: Cannot copy file to 'C:\Users\User\Documents\data_robotics\android\app\src\main\res\mipmap-hdpi\ic_launcher.png', OS Error: Access is denied., errno = 5

Command
flutter create --ios-language swift --android-language kotlin .

Steps to Reproduce

...
...
...

Logs
FileSystemException: Cannot copy file to 'C:\Users\User\Documents\data_robotics\android\app\src\main\res\mipmap-hdpi\ic_launcher.png', OS Error: Access is denied.
, errno = 5
#0      _File.throwIfError (dart:io/file_impl.dart:635:7)
#1      _File.copySync (dart:io/file_impl.dart:340:5)
#2      ForwardingFile.copySync (package:file/src/forwarding/forwarding_file.dart:30:50)
#3      ForwardingFile.copySync (package:file/src/forwarding/forwarding_file.dart:30:50)
#4      Template.render.<anonymous closure> (package:flutter_tools/src/template.dart:267:18)
#5      _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:377:8)
#6      Template.render (package:flutter_tools/src/template.dart:190:24)
#7      CreateCommand._renderTemplate (package:flutter_tools/src/commands/create.dart:799:21)
#8      _rootRunUnary (dart:async/zone.dart:1198:47)
#9      _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#10     _FutureListener.handleValue (dart:async/future_impl.dart:143:18)
#11     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:696:45)
#12     Future._propagateToListeners (dart:async/future_impl.dart:725:32)
#13     Future._completeWithValue (dart:async/future_impl.dart:529:5)
#14     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:40:15)
#15     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:311:13)
#16     Template.fromName (package:flutter_tools/src/template.dart)
#17     _rootRunUnary (dart:async/zone.dart:1198:47)
#18     _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#19     _FutureListener.handleValue (dart:async/future_impl.dart:143:18)
#20     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:696:45)
#21     Future._propagateToListeners (dart:async/future_impl.dart:725:32)
#22     Future._completeWithValue (dart:async/future_impl.dart:529:5)
#23     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:40:15)
#24     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:311:13)

[✓] Flutter (Channel stable, 1.22.4, on Microsoft Windows [Version 10.0.19042.685], locale en-US)
    • Flutter version 1.22.4 at C:\flutter\flutter
    • Framework revision 1aafb3a8b9 (4 weeks ago), 2020-11-13 09:59:28 -0800
    • Engine revision 2c956a31c0
    • Dart version 2.10.4

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at C:\Users\User\AppData\Local\Android\sdk
    • Platform android-30, build-tools 30.0.2
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
    ! Some Android licenses not accepted.  To resolve this, run: flutter doctor --android-licenses

[!] Android Studio (version 4.1.0)
    • Android Studio at C:\Program Files\Android\Android Studio
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[✓] VS Code (version 1.52.0)
    • VS Code at C:\Users\User\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.17.0

[!] Connected device
    ! No devices available

! Doctor found issues in 3 categories.

Flutter Application Metadata
No pubspec in working directory.

possible duplicate of https://github.com/flutter/flutter/issues/69116

how did you fix this crash

how did you fix this crash


Type error with functional component

Subject of the issue
Describe your issue here.
Wrapper and mount/shallowMount do not support functional components defined with Vue.extend()
Steps to reproduce
Tell us how to reproduce this issue. Please provide a working and simplified example.
// test.ts
import Vue, { CreateElement, VNode, RenderContext } from 'vue'

export default Vue.extend({
  name: 'Test',
  functional: true,
  render(createElement: CreateElement, context: RenderContext): VNode {
    return createElement('span', context.data, ['test'])
  },
})

import { RouterLinkStub, shallowMount, mount, Wrapper } from '@vue/test-utils'
import Test from './test'

describe('test', () => {
  // error 1
  let wrapper: Wrapper<Test>

  beforeEach(() => {
    wrapper = mount(Test, {
      context: {
        props: {},
        // error 2
        children: []
      }
    })
  })
})

Error 1
'Test' refers to a value, but is being used as a type here. Did you mean 'typeof Test'?

Error 2
No overload matches this call.
  Overload 1 of 3, '(component: VueClass<CombinedVueInstance<Record<string, any> & Vue, object, object, object, Record<never, any>>>, options?: ThisTypedMountOptions<...> | undefined): Wrapper<...>', gave the following error.
    Type '{ props: {}; children: never[]; }' is not assignable to type 'VNodeData'.
      Object literal may only specify known properties, and 'children' does not exist in type 'VNodeData'.
  Overload 2 of 3, '(component: ComponentOptions<any, DefaultData<any>, DefaultMethods<any>, DefaultComputed, PropsDefinition<Record<string, any>>, Record<...>>, options?: ThisTypedMountOptions<...> | undefined): Wrapper<...>', gave the following error.
    Type 'VueConstructor<Record<string, any> & Vue>' has no properties in common with type 'ComponentOptions<any, DefaultData<any>, DefaultMethods<any>, DefaultComputed, PropsDefinition<Record<string, any>>, Record<...>>'.
  Overload 3 of 3, '(component: FunctionalComponentOptions<Record<string, any>, PropsDefinition<Record<string, any>>>, options?: MountOptions<Vue> | undefined): Wrapper<...>', gave the following error.
    Argument of type 'ExtendedVue<Vue, {}, {}, {}, Record<string, any>>' is not assignable to parameter of type 'FunctionalComponentOptions<Record<string, any>, PropsDefinition<Record<string, any>>>'.ts(2769)

Expected behaviour
It should compile and work fine without any error.
Actual behaviour
descried above.
Possible Solution
When I change let wrapper: Wrapper<Test> to let wrapper: Wrapper<typeof Test>, it then shows a different error.
No overload matches this call.
  Overload 1 of 3, '(component: VueClass<CombinedVueInstance<Record<string, any> & Vue, object, object, object, Record<never, any>>>, options?: ThisTypedMountOptions<...> | undefined): Wrapper<...>', gave the following error.
    Type '{ props: {}; children: never[]; }' is not assignable to type 'VNodeData'.
      Object literal may only specify known properties, and 'children' does not exist in type 'VNodeData'.
  Overload 2 of 3, '(component: ComponentOptions<Vue, DefaultData<Vue>, DefaultMethods<Vue>, DefaultComputed, PropsDefinition<Record<string, any>>, Record<...>>, options?: ThisTypedMountOptions<...> | undefined): Wrapper<...>', gave the following error.
    Type 'VueConstructor<Record<string, any> & Vue>' has no properties in common with type 'ComponentOptions<Vue, DefaultData<Vue>, DefaultMethods<Vue>, DefaultComputed, PropsDefinition<Record<string, any>>, Record<...>>'.
  Overload 3 of 3, '(component: FunctionalComponentOptions<Record<string, any>, PropsDefinition<Record<string, any>>>, options?: MountOptions<Vue> | undefined): Wrapper<...>', gave the following error.
    Argument of type 'ExtendedVue<Vue, {}, {}, {}, Record<string, any>>' is not assignable to parameter of type 'FunctionalComponentOptions<Record<string, any>, PropsDefinition<Record<string, any>>>'.ts(2769)

It seems like mount and shallowMount methods work with functional components without Vue.extend(), but does not with Vue.extend()

versions
vue: 2.6.12
@vue/test-utils: 1.1.2
typescript: 4.0.5

Primarily a typing issue, correct?
The types here are really not idea; both Vue 2 and the types here are written manually, so they have some problems. Are you interested in submitting a PR to fix this?
I am guessing the work around for issue 2 is something like mount(Test, {...} as any).


Set longer timeout for localAlluxioCluster to get ready

Introduced in https://github.com/Alluxio/alluxio/pull/16067/, the timeout for the master in AbstractLocalAlluxioCluster is set to 20s. This brings about >75% failure rate of the test updateConfIntegrationTest in github actions.
Tested on different machines, running same tests the slower machine can be 5x slower than the faster machine. Since Github actions machines are not considered powerful, set the timeout to 30s.

@tcrain @jenoudet @jiacheliu3 @yuzhu Could you take a look? Thanks!

Perhaps we could try rerunning the workflow where the problematic test lies and see if it passes systematically. If not, we can increase the timeout again.

Just as a guess, this test is run concurrently with alluxio.client.cli.fsadmin.command.QuorumCommandIntegrationTest which is quite heavy to run even on my laptop, so I wonder if that has anything to do with it. Since I am not able to get updateConfIntegrationTest to fail on my laptop, I am wondering if the GitHub action runner is just overloaded?

alluxio-bot, merge this please.


100 years ago twitter bot & review page

I'm working on this as R&D, but have some questions and notes and thought it would be easiest to create an issue to track them and test the work.
6/27/2020 still todo

[x] handle multiple authors
[x] include editor(s) if no authors
[x] twitter credentials in local settings
[x] unit tests

@jkotin @clmahoney some notes and questions on my 100 years ago twitter implementation
I'm building a login-only view where you'll be able to see upcoming tweets and check if there are any problems with the data or the twitterbot code — proposing we use the review for a while without automatically tweeting to check for any errors.
I propose we link to specific lending cards when possible (i.e. for borrows & purchases that are footnoted and linked to an image).
Questions:

How important do you think is it to consolidate events for the same member? ("So and so joined & borrowed x" ) This is feasible, just a little harder.
I noticed from the current account that periodicals are listed as 'name borrowed an issue of "title"'; should we use known issues/volumes if possible? Can you suggest the format, and what are the earliest events with known issues/volumes? (i.e. how soon will this come up)
How should we handle multiple authors?
Should we include editor(s) if no author?
Are there still notations in work or event notes that need to be skipped for twitter? The guidelines document I found indicate ERROR in the event notes or "GENERIC" or "PROBLEM" in the work notes.
How far ahead would you want to review these and where would you want to start? I can do two weeks like you've been doing, but it isn't not hard to do more. Currently I have it set to start with events from 100 years ago today and go out two weeks, but I don't think you'd actually want to review the current day's tweets.

Here's a screenshot of what the review page currently looks like:

@rlskoeser This looks great. Responses:

Re: "How important do you think is it to consolidate events for the same member?" Not important. If you mean that there will be separate tweets if a member joins and borrows books on the same day. That's totally fine, maybe even preferable.

Re: I'm happy with either option: "an issue of" or given the exact issue number or date. (Different periodicals will have different systems: "issue number two of," "the May 1935 issue of," "volume 3 and issue 4 of," etc.) I'll ask @clmahoney to let you know when this comes up next. We haven't discussed periodical numbering in depth yet: some cards specify a number, but a lot can be inferred. Cate and I need to work out a system.

Re: multiple authors—ideally it would be: Francis Beaumont and John Fletcher's "Plays." If there are more than two authors you could have: John Smith et al.'s "Book of Gardening." I'm happy to follow CMS on this if it suggests a good system.

Re: editor—yes, I would prefer to include an editor if there's no author. I would like the format to be: "Book of Gardening," edited by John Smith (DATE). If that format is too much trouble, better to exclude the editor.

By the end of the months, there aren't any tags to skip, except "UNCERTAINTYICON."  You should probably skip that, but maybe include at first so we can see what the tweets look like?

Maybe a month out?  I'm flexible.

@jkotin thanks for the responses.

Great — much easier to do separate tweets. Maybe we could schedule them near each other, if we want.
Given the variety of periodical and volume formats, I'm inclined to just stick with "an issue of"
I like your proposal for multiple authors. Looks like CMS doesn't switch to et. al. until over 3, but three names seems like a lot for a tweet. Ok to do two and for two names, et. al. for 3 or more?
I'll see if I can include editors as you suggest without too much trouble. Nice to provide more information if possible.
Seems logical to me.
I'll switch to 4 weeks instead of 2 so we can try that.

Perfect. Re: 3 -- I agree that >2 should get an et al. It will be rare anyway.

@jkotin I'd like to revise the language we're using for subscription events — "joined the library" isn't always accurate for subscription events in the database, since sometimes what are effectively renewals (following a preceding subscription) were logged and documented as subscriptions, and there are plenty of people who let their subscription end but then subscribe again later. (In theory I could check if there are preceding events for the same account, but that could introduce errors and I'd rather not make this code any more complicated than it already is!)
Could we use the same language as the renewals, with subscribed instead of renewed? ("renewed for 1 month at 1 volume per month", "renewed for 2 months"). I know you liked explicitly mentioning the "Shakespeare and Company lending library" — if you don't like making them the same as renewals, please suggest alternate revised wording for subscriptions.

Some notes on the tweets on the test site:

Initial “smart” quotation marks are angled in the wrong direction.
Update test site, if drawing from test site—some info is out of date.
Exclude items with uncertainty icons or books (not periodicals) without dates?
We need to fix author names and decide where best to pull author names from—e.g. Jean Riviere borrowed a book by “Edward John Moreton Drax Plunkett Dunsany.” That should be Lord Dunsany.
Punctuation should be inside quotation marks—relevant for items without dates.

Re: language for posts -- I'm OK with using "subscribed" and "renewed."
Would it be possible to include "Shakespeare and Company" like this:
100YearsAgoToday on Saturday, June 27 at the Shakespeare and Company lending library, John Smith subscribed for 1 month at 1 volume per month.
100YearsAgoToday on Saturday, June 27 at the Shakespeare and Company lending library, John Smith renewed for 1 month at 1 volume per month.
100YearsAgoToday on Saturday, June 27 at the Shakespeare and Company lending library, John Smith borrowed "Ulysses" (1922).

An alternative would be:
100YearsAgoToday on Saturday, June 27 at Shakespeare and Company, John Smith renewed for 1 month at 1 volume per month.
Indeed, just say "Shakespeare and Company," instead of "the Shakespeare and Company lending library" may be better.

@jkotin thanks for all the feedback. I like the revised language for subscriptions & including S&co in the prolog of every tweet.

I updated the test site with production data from this morning
I revised to include S&co in opening of all tweets
I revised tweet language for subscriptions
Excluded events connected to books with UNCERTAINTYICON in the notes
I looked at available name fields. I'm using name for authors, which looks like the best option, but it seem that some author names haven't been cleaned up as well as others (like Dunsany). Which name field would you expect/prefer me to use?
This did prompt me to switch the member names to use the lastname_first field that we're using for the heading on the member detail page; I think this is better for the cases where it matters.

The smart quotes aren't as readable in the font used on the site (or the one twitter uses, either 😕 ). Here's how it looks in my command line report:
#100YearsAgoToday on Wednesday, June 30, 1920 at Shakespeare and Company, Marquis Pagan returned George Bernard Shaw’s “The Philanderer” (1905).
https://shakespeareandco.princeton.edu/members/pagan/cards/6d80f641-d632-4123-9a75-fc8ce40fc62c/#e23116
#100YearsAgoToday on Wednesday, June 30, 1920 at Shakespeare and Company, Marquis Pagan borrowed Rudyard Kipling’s “The Light That Failed” (1891).
https://shakespeareandco.princeton.edu/members/pagan/cards/6d80f641-d632-4123-9a75-fc8ce40fc62c/#e23117
I don't know if the GitHub font is any better. If you're still not convinced, would you try cutting and pasting the text into a document so you can see the smart quotes more clearly?
Revisions are available on the test site.

This all looks good to me!  If Ian get the AR position, we'll fix the author names in the database to follow the same rules as the member names.  At that point, you'll want to use the sort name, according to the same rules at the front end.

@jkotin this functionality is now on the test site with all the changes we discussed.
Please check the review page to confirm the format of the tweets and that it will be sufficient for you to review the tweets ahead of time before they are posted.
https://test-shakespeareandco.cdh.princeton.edu/events/100-years-review/
If there are scenarios you want to check that aren't showing up in the next four weeks (i.e., multiple authors, editors but no author, titles with no year) please provide a few dates for events you'd like to check — I can run a command line report that will display the tweets for any specified date and share the output with you.
The test site is configured to tweet using a test bot account that I've set to private. If you request to follow I will grant access so you can see how the new tweets look, and more easily compare them with the existing account. https://twitter.com/ShakesCo100test

This looks good. I can't think of scenarios to test.  Two sets of questions:

What will be the workflow for this moving forward?  When will it replace the current 100years twitter, and what will be the review process? Would it make sense to review a batch (three months?) of tweets just in case there are odd scenarios that arise?  I don't know the data well enough to predict odd events.

Does it make sense to think about a controlled list of hashtags to add to the tweets?  E.g. if one of these 20 authors are mentioned, use hashtag -firstnamelastname?

@jkotin I manually configured the test site to show 3 months of data instead of 4 weeks so you can look and see if there are any odd cases that aren't being handled correctly.  (The report is currently including borrow returns that are outside of that date range; let me know if that's confusing; I can clean it up, but wasn't sure it mattered). I don't see any examples with multiple authors or no authors and editors — but I did include those in my unit tests following your guidelines.
I thought it would be slower with that much more data, but it doesn't seem too bad — so we could change it to show more than 4 weeks if you prefer. Let me know what you'd like.
To answer your questions:

My plan is for this code to go into production with the other small updates sometime this week. I'd like to run the test twitter account from production for a few days first to make sure everything is working (it took me a few tries to get it configured properly on the test server and I want to leave time for any problems). Once it's tweeting reliably to the test twitter account, it's a simple configuration change to switch it to the main 100 years account. This means we could switch over as soon as sometime mid/late next week, but we don't have to switch that soon (we should be in touch with Ara about our plans before August, though!). As for review: I built the review page to try to mimic the current process. My thought was that you should review regularly and have a workflow to address any errors I'm hoping that problems found in review will largely be fixable via data corrections (e.g. the Lord Dunsany thing you pointed out). If there are occasional bug fixes in the twitter code due to oddities or cases we didn't account for, it should be ok as long as you identify them far enough in advance for me to address.

I'd prefer not to complicate the code any further at this point, and I'm not sure how much hashtags help anyway — famous people are pretty findable by search without them. I would consider adding them later if someone has evidence that they make a significant impact. (Personally, I suspect/hope that my updated tweet content, which links to lending cards for book events instead of just the member page, and thus includes the card preview, will make a bigger impact.)


Fix some warnings 3.9

Scope & Purpose
Fix correct warnings

[ ] :hankey: Bugfix
[ ] :pizza: New feature
[ ] :fire: Performance improvement
[ ] :hammer: Refactoring/simplification

Checklist

[ ] Tests

[ ] Regression tests
[ ] C++ Unit tests
[ ] integration tests
[ ] resilience tests

[ ] :book: CHANGELOG entry made
[ ] :books: documentation written (release notes, API changes, ...)
[ ] Backports

[ ] Backport for 3.9: (Please link PR)
[ ] Backport for 3.8: (Please link PR)
[ ] Backport for 3.7: (Please link PR)

Related Information
(Please reference tickets / specification / other PRs etc)

[ ] Docs PR:
[ ] Enterprise PR:
[ ] GitHub issue / Jira ticket:
[ ] Design document:

[ ] https://jenkins.arangodb.biz/job/arangodb-ANY-mac-matrix.x86-64-2/8/

https://jenkins.arangodb.biz/job/arangodb-ANY-mac-matrix.x86-64-2/9/

https://jenkins.arangodb.biz/job/arangodb-ANY-mac-matrix.x86-64-2/10/

[ ] https://jenkins.arangodb.biz/job/arangodb-ANY-mac-matrix.x86-64-2/12/


New package: jEdit.jEdit version 5.7.0

Checklist for Pull Requests

[x] Have you signed the Contributor License Agreement?
[x] Is there a linked Issue?

Manifests

[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[x] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.

Resolves #189016

Microsoft Reviewers: Open in CodeFlow

Verify:

Using ojdkbuild.openjdk.11.jre as the dependency:

@wingetbot waivers Add Validation-Unapproved-URL


lua-cjson escapes slashes

Hi!
Why does lua-cjson escape slashes:
> cjson = require 'cjson'
> print(cjson.encode('/'))
"\/"

Is that a bug?

so / is a control-character then...

@ignacio what if you are doing
cjson.encode({url =  "https://google.com})

This also encodes the /, but I'm trying to stringify this table/json object.


DPL-924 10X PCR binning strategies change

User story
As a user of the scRNA Core pipeline, I would like Limber to generate a robot driver file for the "10X HT 5' Cherrypick --> 10X HT 5' cDNA Dil" transfer (exact naming to be finalised).
This should be similar to the current 10x pipeline (“LBC Cherrypick --> LBC 5p GEX Dil” transfer/ step to ‘Download Cherrypick to Sample Dilution CSV’) however the current process sets volumes to pick based on concentration and bins samples into either 12 or 14 PCR cycles. For the scRNA pipeline, the volume to pick should be set at 20ul and the range of cycles to bin into should be increased (see additional context for rules).
Who are the primary contacts for this story
Lesley, Abby
Who is the nominated tester for UAT
Lesley, Abby
Acceptance criteria

Hamilton robot driver file can be downloaded, to tell the robot which transfers to make, between the "10X HT 5' Cherrypick" and “10X HT 5' cDNA Dil” plates.

Driver file example for existing pipeline, showing layout:

The samples in the "10X HT 5' Cherrypick" plate should be binned for PCR into different columns across the “10X HT 5' cDNA Dil” plate according to their concentrations (see table in ‘Additional context’ for details).
The new positions should be displayed to the user in Limber on the “10X HT 5' cDNA Dil” plate page and ordered so that each column only allows one number of PCR cycles (see Additional context’ for an example in a pre-existing line).
The new well locations should be captured in the ‘Destination Plate Well’ column of the driver file.
The ‘Sample Vol’ column should be set to ‘20’ and ‘Dilution Vol’ to ‘0’ for all samples.

Additional context
Table of corresponding sample concentrations and PCR cycles. We still need to determine the exact cycle numbers where a range is given. I will update you once we know.

Image of an existing line sharing similar logic:

Outstanding questions for stakeholders

It was mentioned that this is also a requirement for the existing use cases of the 5' 10X pipeline. At what point will the decision be made to go down one binning route rather than the other? Will the SSR specify which one to do in the submission?

Hi Katy,
For the existing line, they need to be able to support both binning routes. Eventually the original method may fall out of use (determined by which 10x kit Faculty use for their sample prep) but they will need both options available for some time. Faculty should be able to let the SSRs know which kit has been used at sample submission though and therefore which line to go down.

Thinking that it would make sense to duplicate the pipeline and add 'HT' into all the purpose names. This story would be just relevant in the 'HT' pipeline. Just the 5' pipeline --> 5' HT for now.

Wondering if we can modify the existing labware creator for binning for PCR cycles to be useable for this (possibly with additional purpose config options for the changed rules) rather than duplicate it.

If GEM X kit is used, numbers in table / volume might change but overall strategy is the same.

Integration suite test branch PR here:
https://gitlab.internal.sanger.ac.uk/psd/integration-suite/-/merge_requests/128

TO DO: generate data to allow Abby to test this step.

UAT notes from Abby:

I created SQPU-85893-C from this cherrypick plate. Attached is the concentration file. What strategy is it currently using to bin the samples (pic attached)? For GEM-X I know we haven't specified the exact cycle numbers but there should only be 6 options total. Is it helpful to set a number for now and then change it later?
For the 'hamilton_lrc_ht_5p_cherrypick_to_lrc_ht_5p_gex_dil' driver file, the sample vol should be set to '10' for GEM-X (and same thing with name changes to replace 'HT'.

Concentration file used for above UAT - concentrations_ngul (3).csv
Screenshot from above UAT:

Note from Katy:

Sample volume needs to be updated to '10' rather than '20' - this is a change in requirement due to the GEM-X chip.
Config in config/purposes/scrna_core_cdna_prep.wip.yml should be changed to match bins in screenshot in story description - although the table shows a range for PCR cycles (e.g. 14-16) and a single number will have to be guessed (needs to be tested in the lab before the 'correct' number can be decided).

Moving this to done as Abby is happy with it. PR mentioned in above comment is merged and tested. The PCR cycle numbers in the config will probably need to be changed once R&D have figured out what numbers they'd like - Abby to let us know when that change is needed.


Invalid count argument with Terraform 1.0

Describe the Bug
Seem that you can't supply a bucket policy when using Terraform 1.0
Error: Invalid count argument
│
│   on .terraform/modules/s3/main.tf line 347, in resource "aws_s3_bucket_policy" "default":
│  347:   count      = local.enabled && (var.allow_ssl_requests_only || var.allow_encrypted_uploads_only || length(var.s3_replication_source_roles) > 0 || var.policy != "") ? 1 : 0
│
│ The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the -target argument to first apply only the resources that the count
│ depends on.

Expected Behavior
I would expect that this to work with Terraform 1.0
Steps to Reproduce
Have terraform 1.0
module "s3" {
  source = "git::https://github.com/cloudposse/terraform-aws-s3-bucket.git?ref=0.39.0"
  acl                      = "private"
  user_enabled             = false
  versioning_enabled       = false

  policy = data.aws_iam_policy_document.cloudinit.json
}

For some reason it doesn't like the var.policy != "" bit.

It looks like I can't use data.aws_iam_policy_document, have to pass in a static json document.


Update Google.Picasa version <IP_ADDRESS>9

Updated installer source URL

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?

[x] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Obsolete application from unofficial download link not allowed.

Obsolete application from unofficial download link not allowed.

Discontinued but still popular and widely used. Official link not available due to the same reason, hence the use of arvhive.org link.

Download link must come from the ISV's servers.

ISV source not available as application has been discontinued by Google. Due to being discontinued, the application doesn't even connect to the internet, this should straighten out some security concerns.
If it's still against some policy do tell me. I'll close the PR.

@ItzLevvie can you provide further instructions

A couple months ago, we had people submitting someone else's apps on OneDrive, Dropbox, etc; despite there being an official download link.
If I rememeber correctly, according to Kevin, the URL must come from the ISV's server, and if it's unavailable, it won't be able to be accepted.
Since the application is discontinued and no longer downloadable on Google's website - I can't confirm if the application has been tampered with.

Nice summary @ItzLevvie!  And it is correct, the software needs to come from the ISV's server.

I understand. Even though it is claimed and believed that archive.org only stores the snapshot of the website without any alterations, adding a source from there only adds inconsistencies. I'm taking the PR down.

http://dl.google.com/dl/desktop/R/en/GoogleDesktopSetup.exe
Google still keeps Google Desktop online despite being discontinued for ages


Support "standard" / alternate format arguments for to_timestamp

Is your feature request related to a problem or challenge?
After https://github.com/apache/arrow-datafusion/pull/8886 (thanks to @Omega359) DataFusion supports converting strings to timestamps using a string format:
SELEECT to_timestamp('2020-09-08T12:00:00+00:00', '2020-09-08 12/00/00+00:00', '%c', '%+', '%Y-%m-%d %H/%M/%s%#z'

Which will parse '2020-09-08T12:00:00+00:00' with several possible formats %c', '%+', '%Y-%m-%d
However, as @comphead  points out, the format used is specific to chrono , the underlying Rust library used. These are slightly different semantics than any existing to_timestamp (it isn't postgres format strings, nor is it spark format strings, it is something datafusion specific based on the rust chrono format strings)
Describe the solution you'd like
Ideally users could decide what "dialect" of string format specifiers they wanted to support based on configuration option. For example, either postgres or spark,
However, this is non trivial given the scope of those two implementations
Describe alternatives you've considered
Users can always use DataFusion's user defined functions to define the semantics they want, for example with a ScalarUDF  that rewrites the specified time string from a postgres format into the chrono format
(though there are likely all sorts of corner cases -- see https://github.com/apache/arrow-datafusion/pull/8886#issuecomment-1899745153)
Additional context
@jhorstmann has notes about Postgres: https://github.com/apache/arrow-datafusion/issues/5398#issuecomment-1820970911
@Omega359 notes that the spark format library is entirely different still: https://github.com/apache/arrow-datafusion/issues/5398#issuecomment-1866926774

Also, perhaps the https://github.com/apache/arrow-datafusion-comet project will be able to contribute a spark compatible implementation of timestamp parsing (that is probably being overly optimistic, though we'll know more shortly)

@alamb I am interested in this problem.
You can assigned it to me

Hi @Tangruilin  -- I think this one is a pretty significant amount of work -- it might be worth sketching out how you would proceed. Perhaps you could make a new project in https://github.com/datafusion-contrib (I can create the repo for you if you want) that contains some UDFs that implement the different semantics?

But this is a meaningful work, In my opinion

indeed it is meaningful work, I just want to make sure that it is clear that the project is potentially a very large amount of work in my opinion

I think @Omega359  did this in https://github.com/apache/arrow-datafusion/pull/8886

This ticket was the followup to #8886 to allow for alternate formats instead of chrono. It's really not feasible imho until DataFusion has the ability to select or alter function behaviour based on a Dialect such as Postgresql, Spark, etc.

🤦  -- sorry


Training NMT models?

Hello! Thanks, Tim! I tried bitsandbytes for language models like BLOOM, and it works well.
I have a question about NMT models like NLLB, M2M, mBART, or OPUS. I tried inference for NLLB, and apparently it is not supported. Are any of these models supported for inference, and especially for fine-tuning?
Many thanks!

hi @ymoslem ,
NLLB should be supported, could you make sure you are using the latest version of transformers? pip install --upgrade transformers ? Could you also share a code snippet that you are using to load NLLB in 8-bit? Thanks!

Thanks, @younesbelkada! Now, it works with transformers==4.24.0, bitsandbytes==0.35.4 and accelerate==0.14.0 after running:
!pip3 install --upgrade transformers bitsandbytes accelerate

However, there are a few observations:
1- It is not faster than float16. When loading nllb-200-distilled-600M, with load_in_8bit=True, it takes 17.1 seconds, while with torch_dtype=torch.float16, it takes 17.9 seconds.
2- When adding int8_threshold=2.0, I got "an unexpected keyword argument" error. It seems that AutoModelForSeq2SeqLM does not support it.
3- GPU consumption seems the similar in both cases; 4871MB with 8bit and 4151MB with float16.
The aforementioned results are on an NVIDIA RTX A4000 GPU. I tried also on Google Colab with Tesla T4, and got similar conclusions. I tried also with the facebook/nllb-200-3.3B model
8bit
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

src_lang = "eng_Latn"
tgt_lang = "spa_Latn"

tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M", src_lang=src_lang)
model = AutoModelForSeq2SeqLM.from_pretrained("facebook/nllb-200-distilled-600M",
                                              low_cpu_mem_usage=True,
                                              device_map= "auto",
                                              load_in_8bit= True)

source_text = 'Chinese clinical trials in Wuhan and Shenzhen claimed to show that favipiravir was "clearly effective".'
inputs = tokenizer(source_text, return_tensors="pt").to("cuda")

translated_tokens = model.generate(
        **inputs, forced_bos_token_id=tokenizer.lang_code_to_id[tgt_lang], max_length=30
)
tokenizer.batch_decode(translated_tokens, skip_special_tokens=True)[0]

float16
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
import torch

src_lang = "eng_Latn"
tgt_lang = "spa_Latn"

tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M", src_lang=src_lang)
model = AutoModelForSeq2SeqLM.from_pretrained("facebook/nllb-200-distilled-600M",
                                              torch_dtype=torch.float16,
                                              low_cpu_mem_usage=True,)
model = model.half()
model.to("cuda")

source_text = 'Chinese clinical trials in Wuhan and Shenzhen claimed to show that favipiravir was "clearly effective".'
inputs = tokenizer(source_text, return_tensors="pt").to("cuda")

translated_tokens = model.generate(
        **inputs, forced_bos_token_id=tokenizer.lang_code_to_id[tgt_lang], max_length=30
)
tokenizer.batch_decode(translated_tokens, skip_special_tokens=True)[0]

Hi @ymoslem
Thanks a lot for your message

1- It is not faster than float16. When loading nllb-200-distilled-600M, with load_in_8bit=True, it takes 17.1 seconds, while with torch_dtype=torch.float16, it takes 17.9 seconds.

Yes this is expected, the 8-bit is currently slower than the fp16 model because the 8-bit quantization is done in two stages. You can check out more about that on the 8-bit integration blogpost.

2- When adding int8_threshold=2.0, I got "an unexpected keyword argument" error. It seems that AutoModelForSeq2SeqLM does not support it.

Yes, please use load_in_8bit_threshold instead. Could you point me to the place you have read that says to use int8_threshold? Maybe the documentation has not been updated

3- GPU consumption seems the similar in both cases; 4871MB with 8bit and 4151MB with float16.

Could you share with me how do you measure that? Note that the memory optimization between fp16 and int8 model really depends on the model size, for nllb-600M you get a memory footpint saving of 1.18, for 3.3B you get a saving of 1.41, etc and it linearly grows with the size of the model.
You can check that with this snippet:
`
``
import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
src_lang = "eng_Latn"
tgt_lang = "spa_Latn"
model_id = "facebook/nllb-200-3.3B"
model = AutoModelForSeq2SeqLM.from_pretrained(model_id,device_map= "auto",load_in_8bit=False, torch_dtype=torch.float16)
model_8bit = AutoModelForSeq2SeqLM.from_pretrained(model_id,device_map= "auto",load_in_8bit=True)
print(model.get_memory_footprint() / model_8bit.get_memory_footprint())

Many thanks, @younesbelkada for the detailed explanation!

Note that the memory optimization between fp16 and int8 model really depends on the model size, for nllb-600M you get a memory footpint saving of 1.18, for 3.3B you get a saving of 1.41, etc

I confirm this result on Google Colab, when loading the model only.
What I was trying earlier (and reported in the previous reply) was running everything until the translation, and checking nvidia-smi.

Yes, please use load_in_8bit_threshold instead. Could you point me to the place you have read that says to use int8_threshold? Maybe the documentation has not been updated

I think the documentation is updated, but maybe there is an old notebook that appears in search.

Regarding fine-tuning, is there any value in using bitsandbytes for fine-tuning?
I am using this code, but it gives the following error:
from transformers import TrainingArguments, Trainer, logging
from torch.utils.checkpoint import checkpoint

training_args = TrainingArguments(
    output_dir="run",
    num_train_epochs=40,
    per_device_train_batch_size=2,
    per_device_eval_batch_size=2,
    gradient_accumulation_steps=4,
    eval_accumulation_steps=4,
    gradient_checkpointing=True,
    evaluation_strategy="epoch"
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_finetune,
    eval_dataset=tokenized_validate,
)
trainer.train()

Exception: State must contain either CBt or CB matrix for backward

Thanks again!


[external-metadata] Fix unknown column types

This PR fixes an issue where the SQLAlchemy NullType() cannot be cast to a string. Similar to here the fix is merely to set the type as UNKNOWN.
to: @michellethomas @mistercrunch

Codecov Report

Merging #6196 into master will decrease coverage by 0.01%.
The diff coverage is 60%.

@@            Coverage Diff             @@
##           master    #6196      +/-   ##
==========================================
- Coverage   76.93%   76.91%   -0.02%
==========================================
  Files          47       47
  Lines        9389     9393       +4
==========================================
+ Hits         7223     7225       +2
- Misses       2166     2168       +2

Impacted Files
Coverage Δ

superset/connectors/sqla/models.py
80.71% <60%> (-0.24%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a0479d5...26b3931. Read the comment docs.


Photochem v0.6.0

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

Hi! This is the friendly automated conda-forge-linting service.
I wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found some lint.
Here's what I've got...
For recipe/meta.yaml:

requirements: run: photochem_clima_data==0.2.0 must contain a space between the name and the pin, i.e. photochem_clima_data ==0.2.0

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender


Bump macro to include VS2014 to fix py3.5 build

as titled

Passed small windows smoke test for py3.5


Edit scheduled time for a batch change

Changes in this pull request:

allow a creator or system admin to change the scheduledTime of a scheduled batch change
also explores a different approach through the batch change service for re-validations

Closing as out-of-date, original author MIA


Implement OperationHandler for axum::handler::Layered

Allow adding handlers to ApiMethodRouter that have a layer applied via Handler::layer.

remind me to release if i haven't done it by tomorrow

Actually, I don't really care about this going into aide 0.13 for now, as we're still using 0.12. So if you could make a v0.12.x branch that I can make a backport PR to, that would actually be more helpful for me ^^
See also https://github.com/tamasfe/aide/issues/132

@Wicpar Reminder to follow up


Support bnb l2

:clipboard: Add associated issues, tickets, docs URL here.
Overview
Describe what your Pull Request is about in a few sentences.
Add git patch to support BNB L2
Changes
Describe your changes and implementation choices. More details make PRs easier to review.
L1 -> L2
BNB
Currently, there are two ways to deposit the L1 native token (BNB) to L2. Users can use OptimismPortal or L1StandardBridge to deposit the L1 native token. To simplify things, I disabled the L1 native token deposit in L1StandardBridge. Users have to use OptimismPortal to deposit BNB.
In OptimismPortal, users can force L2 to include transactions from L1. However, the deposit of the L1 native token will mint an L2 ERC20 token on L2. I separated this native token deposit from the original logic and added another event to mint the ERC20 token on L2.
BOBA
I added a depositBoba function in OptimismPortal to emit an event so that we can mint the L2 native token on L2. For the security consideration, this function can only be called from L1CrossDomainMessenger and the initial sender must be L1StandardBridge. In other words, users must call L1StandardBridge to deposit BOBA.
Workflow: L1StandardBridge -> L1CrossDomainMessenger -> OptimismPortal
L2 -> L1
BOBA
Users call the withdrawETH or withdrawETHTo functions in L2StandardBridge. The L2StandardBridge will call the withdrawBoba function in L2CrossDomainMessenger . The difference of the withdrawBoba and the standard sendMessage function is that we override the value in the relayMessage function, because we treat Boba as an ERC20 token on L1. The withdrawBoba  function calls the initiateBobaWithdrawal function in L2ToL1MessagePasser to emit the final event.
During this process, the BOBA token will be transferred from the user account to L2ToL1MessagePasser as the standard process and we can destroy the BOBA token in L2ToL1MessagePasser .
BNB
Users call the withdrawERC20 or withdrawERC20To functions in L2StandardBridge. The L2StandardBridge will call the withdrawBnb function in L2CrossDomainMessenger. The difference of the withdrawBnb and the standard sendMessage function is that we override the value in the relayMessage function, because we treat Bob as the L1 native token on L1. The withdrawBnb function calls the initiateBnbWithdrawal function in L2ToL1MessagePasser to emit the final event.
Testing
Describe how to test your new feature/bug fix and if possible, a step by step guide on how to demo this.
To start the network, run this command:
make devnet-bnb-hardhat-up

To run the deposit and withdrawal tests:
make devnet-bnb-hardhat-test

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 24.92%.  Consider uploading reports for the commit c74916c to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #192      +/-   ##
===========================================
- Coverage    28.21%   24.92%   -3.29%
===========================================
  Files          168      144      -24
  Lines         7294     5540    -1754
  Branches      1311      909     -402
===========================================
- Hits          2058     1381     -677
+ Misses        5130     4053    -1077
  Partials       106      106

Flag
Coverage Δ

cannon-go-tests
82.20% <ø> (ø)

chain-mon-tests
27.14% <ø> (ø)

common-ts-tests
26.72% <ø> (ø)

contracts-bedrock-tests
0.62% <ø> (ø)

contracts-ts-tests
12.25% <ø> (ø)

core-utils-tests
44.03% <ø> (ø)

sdk-next-tests
?

sdk-tests
?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 24 files with indirect coverage changes

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 0.62%. Comparing base (f461eca) to head (c74916c).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #192       +/-   ##
===========================================
- Coverage    28.21%   0.62%   -27.59%
===========================================
  Files          168      94       -74
  Lines         7294    2396     -4898
  Branches      1311     546      -765
===========================================
- Hits          2058      15     -2043
+ Misses        5130    2381     -2749
+ Partials       106       0      -106

Flag
Coverage Δ

cannon-go-tests
?

chain-mon-tests
?

common-ts-tests
?

contracts-bedrock-tests
0.62% <ø> (ø)

contracts-ts-tests
?

core-utils-tests
?

sdk-next-tests
?

sdk-tests
?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 74 files with indirect coverage changes

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 24.87%.  Consider uploading reports for the commit 7435658 to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #192      +/-   ##
===========================================
- Coverage    28.21%   24.87%   -3.35%
===========================================
  Files          168      140      -28
  Lines         7294     5400    -1894
  Branches      1311      897     -414
===========================================
- Hits          2058     1343     -715
+ Misses        5130     3951    -1179
  Partials       106      106

Flag
Coverage Δ

cannon-go-tests
82.20% <ø> (ø)

chain-mon-tests
?

common-ts-tests
26.72% <ø> (ø)

contracts-bedrock-tests
0.62% <ø> (ø)

contracts-ts-tests
12.25% <ø> (ø)

core-utils-tests
44.03% <ø> (ø)

sdk-next-tests
?

sdk-tests
?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 28 files with indirect coverage changes

Codecov Report
Attention: Patch coverage is 8.19672% with 168 lines in your changes are missing coverage. Please review.

Project coverage is 27.72%. Comparing base (f461eca) to head (ab791f2).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #192      +/-   ##
===========================================
- Coverage    28.21%   27.72%   -0.49%
===========================================
  Files          168      169       +1
  Lines         7294     7477     +183
  Branches      1311     1352      +41
===========================================
+ Hits          2058     2073      +15
- Misses        5130     5298     +168
  Partials       106      106

Flag
Coverage Δ

cannon-go-tests
82.20% <ø> (ø)

chain-mon-tests
27.14% <ø> (ø)

common-ts-tests
26.72% <ø> (ø)

contracts-bedrock-tests
0.62% <ø> (ø)

contracts-ts-tests
12.25% <ø> (ø)

core-utils-tests
44.03% <ø> (ø)

sdk-next-tests
35.72% <8.19%> (-2.88%)
:arrow_down:

sdk-tests
35.72% <8.19%> (-2.88%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

packages/sdk/tasks/index.ts
100.00% <100.00%> (ø)

packages/sdk/tasks/deposit-bnb.ts
7.69% <7.69%> (ø)

While reading to try to make sure that I understood the protocol level interactions of the contracts, I stumbled across: https://specs.optimism.io/protocol/granite/custom-gas-token.html
It looks like this path may be officially supported soon in the specs? Even if we don't want to wait for official support, I think it's worth investigating and maybe building from this experimental branch?

Which branch are you referring to?

I found this branch. Optimism blocks some withdrawal paths to simplify the code changes. Basically, you can only withdraw your L2 native token from L2ToL1MessagePasser. We can simplify our codebase and use some functions from Optimism, but the idea is the same.
One different design is that Optimism override the L2 to L1 messages in OptimismPortal, but I send all correct messages from L2 to L1 so that we don't modify the logic of OptimismPortal.
https://github.com/ethereum-optimism/optimism/tree/feat/custom-gas-token

While reading to try to make sure that I understood the protocol level interactions of the contracts, I stumbled across: https://specs.optimism.io/protocol/granite/custom-gas-token.html
It looks like this path may be officially supported soon in the specs? Even if we don't want to wait for official support, I think it's worth investigating and maybe building from this experimental branch?

Which branch are you referring to?

The specification referenced above has begun to be implemented in the current optimism tree.  For instance here:
https://github.com/ethereum-optimism/optimism/blob/943ed0f94f02a2ebad5155334828cc07fca46e0f/packages/contracts-bedrock/src/libraries/GasPayingToken.sol
You can see some of the implementation referenced in the protocol specs.

While reading to try to make sure that I understood the protocol level interactions of the contracts, I stumbled across: https://specs.optimism.io/protocol/granite/custom-gas-token.html
It looks like this path may be officially supported soon in the specs? Even if we don't want to wait for official support, I think it's worth investigating and maybe building from this experimental branch?

Which branch are you referring to?

The specification referenced above has begun to be implemented in the current optimism tree. For instance here:
https://github.com/ethereum-optimism/optimism/blob/943ed0f94f02a2ebad5155334828cc07fca46e0f/packages/contracts-bedrock/src/libraries/GasPayingToken.sol
You can see some of the implementation referenced in the protocol specs.

Yes! Let's merge the upstream, so we can use the same codebase.


[BM-639] Delete action/value messages (unify with iOS)

Task

JIRA

Description

In order to unify webSocket message handling we decided to delete
the {action: String ,value: String} in favor of {fieldName: Type} messages.

          1 Message

:book:
:link: BM-639

JaCoCO Code Coverage 12.36% :white_check_mark:

Class
Covered
Meta
Status

co/netguru/baby/monitor/client/feature/client/home/SendFirebaseTokenUseCase
100%
0%
:white_check_mark:

co/netguru/baby/monitor/client/feature/communication/websocket/Message
0%
0%
:white_check_mark:

co/netguru/baby/monitor/client/feature/server/ChildMonitorViewModel
50%
0%
:white_check_mark:

APK file size

size

30.1MB

Number of method references

Dex file
count

classes.dex
40660

  Generated by :no_entry_sign: Danger


Init master

Reversed in some bug fixes and enhancements to Standard Initiative from next release v2.051

Found issue with initiative actions including "Move", "Change Weapon", "Do Nothing" and "Other".  Closing until fixed

Done thorough further testing, and the issue previously found would seem to have originated elsewhere.  So re-submitting this pull request.


Fallible linker

Give the responsibility of handling machine degrees to the linker:

degree defaults to 1024
submachines must have a matching degree, or no degree

Also maybe a test?


[Security] Android keyboard app does store the password entered in Mattermost App for sign-in

Summary
On android when using Mattermost App to sign into a server the keyboard stores (parts of) the password and uses it for autocomplete/suggest later on.
Environment Information

Device Name: Xiaomi Redmi K20 Pro
OS Version: Android 9
Mattermost App Version: 2.0.1 (Build 6000456)
Mattermost Server Version: 7.7.1
Keyboard: SwiftKey <IP_ADDRESS>

Steps to reproduce

Use a generated, complex password like Hp!JTqSzp\SLm%KA9xhw for your mattermost user on any given mattermost server.
Install Mattermost Android App and sign into your server
Wipe Mattermost Android App Data and open Mattermost Android App again
Mattermost will prompt you to add a new server and sign into it
When typing parts of the password SwiftKey Keyboard will suggest the next part up to the next special character, e.g. you type Hp! and JTqSzp will appear in the keyboards suggest row (this is just an example and might not work 100% but Swiftkey did just suggest parts of my password after uninstalling and reinstalling Mattermost Android App)
this did not happen to me on any other app before - Mattermost being the 1st time I did notice Swiftkey did store a previously entered password

Expected behavior
The keyboard app does NOT store parts of the password in the 1st place nor does it suggest parts of it.
Observed behavior (that appears unintentional)
The keyboard app did save parts of the password and suggested parts when it noticed I was typing my password again.
Possible fixes
Set flags for the password field so keyboard apps know not to store entered characters (I have no idea if and how this works on android).

@c0deright Thank you, would you be open to sharing these reports via<EMAIL_ADDRESS>Please see more on our Responsible Disclosure Policy here https://mattermost.com/security-vulnerability-report/.

Have you tried any other keyboard? Gboard maybe? Cause there is nothing we do in the app to store anything on any keyboard, so I just want to confirm

Ok, I did dig a bit deeper and it seems to be an issue outside of Mattermost, I think.
The password field has a eye symbol to the right that is crossed out by default, resulting in entered characters to be replaced with •. In this mode your 5 chars long password looks like ••••• and the keyboard does not save the string - everything is fine.
When the crossed out eye symbol is tapped, the characters entered aren't replaced by • and the keyboard does save the string - this is the issue I stumbled over because of my complicated and long password I wanted to make sure to enter correctly.
I guess it's more of an Android thing then. Using Android 9.

is it ok to close this issue then?

Yeah, I guess.


make modeshape dependency explicit

The fcrepo-module-auth-rbacl module depends on fcrepo-kernel-modeshape, but that dependency is not made explicit (hence the fact that it will not currently compile).

Please include a link to the related ticket that provoked this compilation failure.

@awoods: I thought it was https://jira.duraspace.org/browse/FCREPO-2242, but that code doesn't actually change how the dependencies on kernel-modeshape work. Furthermore, Jenkins seemed to build it just fine w/o this current PR.
Still, I would maintain that the current maven configuration on fcrepo-module-auth-rbacl is wrong and that this change fixes it. Perhaps the best approach is to create a new ticket for this.

Sure, if this is indeed a problem (or will be some day) and not actually related to an existing ticket... a new ticket makes sense.

https://jira.duraspace.org/browse/FCREPO-2265


pull request

it's been a while. would you still consider merging pull requests or any update to this?

@dev7060 would still be happy to merge any pull requests you have!


Document iptables and container-selinux config for RancherD install on SELinux

https://github.com/rancher/docs/issues/2916

If the same configuration applies to RHEL 8 as well, I think it will be helpful to call that out. I'll update the PR.


Automated cherry pick of #2060: support get model distinct field

Cherry pick of #2060 on release/2.10.0.
#2060: support get model distinct field

Codecov Report

:exclamation: No coverage uploaded for pull request base (release/2.10.0@dc53e5a). Click here to learn what that means.
The diff coverage is 0%.

@@               Coverage Diff                @@
##             release/2.10.0   #2078   +/-   ##
================================================
  Coverage                  ?   7.59%
================================================
  Files                     ?     410
  Lines                     ?   63401
  Branches                  ?       0
================================================
  Hits                      ?    4813
  Misses                    ?   58177
  Partials                  ?     411

Flag
Coverage Δ

#aFlag
7.59% <0%> (?)

Impacted Files
Coverage Δ

pkg/compute/models/vpcs.go
2.12% <0%> (ø)

pkg/compute/models/routetables.go
2.85% <0%> (ø)

pkg/cloudcommon/db/modelbase.go
1.79% <0%> (ø)

pkg/compute/models/guests.go
0.68% <0%> (ø)

pkg/compute/models/networks.go
0.85% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc53e5a...647dd4a. Read the comment docs.

/lgtm

/approve


Example page is 404

I think it's because index.html in /docs is named index.prod.html, but shouldn't.
Not working
https://ajnsit.github.io/purescript-concur/
Working
https://ajnsit.github.io/purescript-concur/index.prod.html

Thanks for the report! Fixed in https://github.com/ajnsit/purescript-concur/commit/c0cc87247cd33c68f6c8ba4fe1b5f78267b01d6c.


stack view should have an option to remove the elements from DOM on collapsing.

[ ] bug
[x] feature request
[ ] enhancement

Expected behavior
The stack-view removes the children elements from the DOM on stack-view collapse
This could be an optional param
Actual behavior
This option is not available
Reproduction of behavior
n/a
Environment details

Angular version: 2.0.X

**Clarity version:0.9.4

OS and version:

Browser: chrome, firefox, IE11, edge

@youdz : Please add to backlog if this is something we want to do. Close with explanation if not.

Actually, this is a duplicate of #897 (which hasn't been written yet) and part of #903, so I'll close it as duplicate. But yes, we'll be working on this soon.


Cannot collect data when Apache is running in a different port

When running NGINX as a proxy for Apache, NGINX will take the default ports. If you choose to keep scraping from the normal endpoint data may be stale due to caching or may result in different issues (at least in my setup) like: apache_exporter[643684]: 2021/06/22 08:49:29 http: Accept error: accept tcp [::]:9117: accept4: too many open files; retrying in 1s.
scrape_uri cannot be used to collect from a different port (like 81) because it complains: first path segment in URL cannot contain colon

hi @andbuitra
You could have nginx proxy requests (withotu cache) to Apache no?

@Lusitaniae I just checked and it actually is not caching the server-status but the exporter is not collecting the data. When restarted it collects it and after a random amount of time it stops. Before it used to show the "too many open files" but I increased the openfile limit in the systemd unit. Now it just doesn't do it.
I've enabled debug mode and upgraded to 0.9.0 from 0.8.0 just to check. I'll see how long it lasts

Worth looking specifically into what process is keeping too many open files.
See https://serverfault.com/questions/707490/how-to-determine-which-processes-have-most-inodes-opened
I would not expect apache exporter or nginx to cause that.
Do you also have application server running in the same server? Or is Apache creating too many processes?

 The exporter is still running on the screen:
INFO[0000] Starting apache_exporter (version=0.9.0, branch=HEAD, revision=5f55c8eafd0c5e51237736b361db395b2b2b5e09)  source="apache_exporter.go:367"
INFO[0000] Build context (go=go1.16.5, user=root@8855f7ef9546, date=20210606-11:51:22)  source="apache_exporter.go:368"
INFO[0000] Starting Server: :9117                        source="apache_exporter.go:369"
INFO[0000] Collect from: http://localhost/server-status/?auto  source="apache_exporter.go:370"
INFO[0000] listening and wait for graceful stop          source="apache_exporter.go:374"

@andbuitra Have you found a solution?
I will close the issue for now, feel free to update if you need


[move-prover] Setting up infrastructure for analysis/transformation tests.

This sets up the infrastrucure for testing analysis phases (which create annotations) and transformations.

A printer for stackless bytecode functions.
The printer supports a mechanism to dump annotations as produced by analysis phases.
A test driver which executes baseline tests in stackless-bytecode-generator/tests/<subdir> where each subdir represents a set of tests related to a particular pipeline. These tests are based on printing out the initial bytecode and the bytecode+annotations for the result of running a pipeline.

This new feature is demonstrated on a basic test in stackless-bytecode-generator/tests/lifetime.
This PR also adds a new analysis for reaching definitions, and tests it using the above mechanism, but nothing is yet done with the result of this analysis. It also adds some code to stackless_control_graph.rs for tracking predecessors blocks in addition to successor blocks, which will allow backward analysis (in a future PR).
Motivation
Improve prover architecture.
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
New infrastructure for tests of bytecode analysis and some concrete tests.
Related PRs
NA

@emmazzz @sblackshear FYI.
This is currently based on #3401  so ignore the diff for that one.

:umbrella: The latest upstream changes (presumably 3a45ce2a94ddbcb0f9fd57f2d31c21fab6ab9d4c) made this pull request unmergeable. Please resolve the merge conflicts.

:umbrella: The latest upstream changes (presumably 02bd6b0192b834d6e9778cc82d30c2b44e3ba263) made this pull request unmergeable. Please resolve the merge conflicts.

@bors-libra r=shazqadeer

:pushpin: Commit cbb3efc has been approved by shazqadeer

:hourglass: Testing commit cbb3efcf89e10b2b604a54259984c390679715da with merge 0c3d67de63823efcc89894a4b90c7e068b4c3201...

:sunny: Test successful - checks-actions_land_blocking_test, checks-circle_commit_workflow
Approved by: shazqadeer
Pushing 0c3d67de63823efcc89894a4b90c7e068b4c3201 to master...


cpu.x86: use push and pop when emitting %prologue and %epilogue

A fun trick they use in llvm is to replace some add/sub rsp instructions with push and pops. The latter instructions take one byte each while the former takes four. We can use that too. It reduces the code size by 0.87% :) So, not so much but it was an easy optimization.

Merged (awhile ago), thanks!!


[material-ui][Autocompelte] Highlighted index is broken when inserting new item

Steps to reproduce
Link to live example: https://codesandbox.io/p/sandbox/mui-autocomplete-bug-94thpd?file=%2Fsrc%2FApp.js
Steps:

As soon as the component appears, press the bottom arrow to select the first item (Pulp Fiction)
Wait 3 seconds for an additional item to appears at the top (The Godfather)
Press the top arrow to select the new item

Current behavior
The last item (The Shawshank Redemption) is now selected
Expected behavior
The first item (The Godfather) should be selected
Context
No response
Your environment

npx @mui/envinfo
  Don't forget to mention which browser you used.
  Output from `npx @mui/envinfo` goes here.

Search keywords: autocomplete

It's a bug. I also observed that it only occurs when the option is added to the beginning of the list.

hey, is this bug still open?

hey, is this bug still open?

No. A PR was opened to fix it - #41306


Using this fork of rosetta drone effectively disables the rear LED on the mini 2

Describe the bug
Immediately after tapping open in rosetta drone, the rear led turns off and won't turn back on unless the firmware is downgraded and upgraded with DJI Assistant.
To Reproduce
Steps to reproduce the behavior:

Connect the remote to the phone and allow rosetta drone to connect to it
Tap open in the app. The rear LED on the drone should almost immediately turn off and it'll stay off even across cold reboots and battery pulls.

Expected behavior
The rear LED should be on / flashing to show the current status of the drone
Smartphone (please complete the following information):

Device: Google Pixel 4a 5G
OS: Android 11

Additional context
The only fix I could find is reinstalling the firmware completely. This can be done by installing DJI Assistant from here, changing upgrade_enable in C:\Program Files (x86)\DJI Product\DJI Assistant 2 (Consumer Drones Series)\DJIEngine\defaultsettings.ini to 1, then downgrading the firmware to any older version then upgrading it if you want. Clicking refresh on the current firmware version doesn't work.

Sounds like a parameter is set wrong?
Might be related to mavic air 2 logic. Tbh I cant say i ever noticed that and never had to refresh either.
Try just changing the color in the DJI Fly app next time, I usually switch to rosetta after takeoff, maybe it only happens if u start through it?

Changing the color changes the front LED perfectly fine but the rear LED stays off. I haven't tried switching to Rosetta drone after takeoff but I'll try it and see what happens.

lmk how it goes. Alternatively can spoof as mini 1, but then the UX SDK controls dont let u select certain stuff and we would need custom UI elements (sdk calls still succeed)

(Also i think just "Write default parameters" in Dronehacks should do the job as well, refresh shouldnt be needed, probably fucks with a bitmask)

I'm not sure what's going on but now it's crashing with this error when I tap open in the app. It's happening on my Pixel and a Lenovo tablet with an Android 10 GSI. I don't know if this is related but on the main window it says Model Not Available and Unknown Aircraft.
AndroidRuntime: Process: sq.rogue.rosettadrone, PID: 18105
AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{sq.rogue.rosettadrone/sq.rogue.rosettadrone.MainActivity}: java.lang.NullPointerException: Attempt to invoke interface method 'java.lang.Object java.util.List.get(int)' on a null object reference
AndroidRuntime:        at sq.rogue.rosettadrone.DroneModel.initFlightController(DroneModel.java:264)
AndroidRuntime:        at sq.rogue.rosettadrone.DroneModel.<init>(DroneModel.java:219)
AndroidRuntime:        at sq.rogue.rosettadrone.MainActivity.onCreate(MainActivity.java:589)
ActivityTaskManager:   Force finishing activity sq.rogue.rosettadrone/.MainActivity
ActivityTaskManager:   Force finishing activity sq.rogue.rosettadrone/.ConnectionActivity
ActivityManager: Showing crash dialog for package sq.rogue.rosettadrone u0

It turns off the LED even if I connect after takeoff. Also now that I know about dronehacks I was able to find the parameter that messes it up. g_config.misc_cfg.forearm_lamp_ctrl is 255 by default but rosetta drone changes it to 1 and I guess that's not a valid value.

It's no problem at all! I like messing around with the drone anyways so this is fun to me. I'm not sure if you already knew this but apparently you can connect to the drone thru USB after takeoff and read parameters while it's hovering lmao.

Lol what? Viable info mate!

Yeah you just gotta get it into that usb mode where it exposes the serial ports, unplug the cord from the computer and let the controller connect, then just take off and reconnect usb afterwards. I'll send a video in a bit

we've been looking at getting usb comms working after launch of payloads, so this is very interesting

Btw did u have issues with video feed? I had to rework it all because its super buggy on my S21.
Works ok on my S6 Edge Plus but still sometimes loses a keyframe.
As far i can tell, need to wait for a SDK update, so curious about other phones (tho should always be in VLOS anyway ;) )

Yeah I've been having problems. The video is just completely black at first but switching between video and photo mode a few times in the app gets it working sometimes but other times it'll just show random blocks of pixels. Also I could never get it to stream video anywhere else but I couldn't get it working with a phantom I had access to for a bit either so it might just be a me problem lol.

Here's the video: https://photos.app.goo.gl/iUEn9adkHKLW6A5W7

Yeah I've been having problems. The video is just completely black at first but switching between video and photo mode a few times in the app gets it working sometimes but other times it'll just show random blocks of pixels. Also I could never get it to stream video anywhere else but I couldn't get it working with a phantom I had access to for a bit either so it might just be a me problem lol.

I can reproduce those issues on my S6, that gives me hope that I can fix that stuff.
It would be easy if i can request a keyframe, but that is just missing from the android sdk (its available in SDK v5 and IOS SDK v4) so all i can do is wait or waste hours... lol
Rn its as stable is i was able to get it.
https://github.com/m4xw/rosettadrone_mini2/commit/e896debc275cf43edc379c1ea897c16471688d18 was before i rewrote it to use ffmpeg for decoding, notsure if that changes things. would need to cherrypick the other changes tho

@lmore377 can you check if camera still works in the app when doing this?

Here's the video: https://photos.app.goo.gl/iUEn9adkHKLW6A5W7

Gotta go to the store for a bit but I'll try it when I get back. Here's the full quality upload since Google photos is being ass https://mega.nz/file/xtUCyZCa#_VVH7TDc9CpKNsRI54J1qLeZ06lDp9LW9azM09mXm1I

@m4xw The camera immediately cuts out when plugging in USB in both dji fly and rosetta but telemetry comes through fine and it comes back when you disconnect USB. DJI Fly shows these errors when its plugged in.

Also, in rosetta the stream comes back fine with some minor glitchiness so I'm thinking the video stream is still active but no data is being pushed through it when it disables the camera.

And one more thing, when USB is plugged in all battery info disappears in dji fly

Also, in rosetta the stream comes back fine with some minor glitchiness so I'm thinking the video stream is still active but no data is being pushed through it when it disables the camera. Is there any way to manually request a single parameter from the drone instead of having to refresh the entire list every time? And is there any way to get parameters remotely? I want to see what (if anything) is different between the usb being connected and disconnected.

Its because its not getting a full frame. Thats how the encoding works. If you add correct data on borked data, its gonna get funky

And one more thing, when USB is plugged in all battery info disappears in dji fly

@m4xw The camera immediately cuts out when plugging in USB in both dji fly and rosetta but telemetry comes through fine and it comes back when you disconnect USB. DJI Fly shows these errors when its plugged in.

Yea... thanks for confirming

If you start recording before you plug the drone in it'll keep recording perfectly fine. https://mega.nz/file/owNAnJCA#um2rHyIobABQftNXP_XppWN9Fm6rUvnhXAe_fLJbG_I


New object is created on destination instead of null when using Adapt<>

I have the following classes:
ScheduleWrapper
 public Guid ID { get; set; }
 public Guid? CompanyID { get; set; }
 public CItyWrapper City {get;set;}
 public CItyWrapper City1 {get;set;}

Schedule
 public Guid ID { get; set; }
 public Guid? CompanyID { get; set; }
 public Company Company { get; set; }
 public CIty Start{get;set;}
 public CIty End {get;set;}

When I try to adapt ScheduleWrapper to Schedule, on Schedule an empty Company object will be created instead of null.
Other configurations are:
config.Default.PreserveReference(true);
config.Default.IgnoreNullValues(true);
config.NewConfig<Schedule, ScheduleWrapper>().TwoWays()
                .Map(dest => dest.City, src => src.Start)
                .Map(dest => dest.City1, src => src.End);

I noticed this behavior also in other places where the explicit mapping is not defined.
Is there a setting to not create empty objects on destination?

I migrated from AutoMapper, and there I had the "AllowNullDestinationValues" property on the MapperConfiguration.

@andrerav

A little too late to the party but I just hit the same problem.
This is how I fixed it
        config.NewConfig<ProductCategory, SaveProductCategoryDto>()
            .MaxDepth(1);
        config.NewConfig<SaveProductCategoryDto, ProductCategory>()
            .IgnoreMember((member, side) => side == MemberSide.Destination
                                            && member.Name.Equals(nameof(ProductCategory.Company)))
            .IgnoreNullValues(true)
            .MaxDepth(1);

where ProductCategory is a data entity, retrieved from DB that has a navigational prop to Company, not required and not needed when creating a new ProductCategory.
Best,
Simo


how can I use libev with grpc together

My applications is heavily using libev, how can I adapter grpc to work with libev?

"It depends"
The easiest way would be to put grpc work on a separate thread.
Stepping up from there, you could have a different grpc_endpoint that uses libev... and every time there's IO on that endpoint follow up with a poll to a completion queue (the synchronous c++ api's won't work).
If you're ambitious, an iomgr port to libev ought to be possible. If you wanted to do that work, we'd probably take it as a contribution and make it an option for users.

cc @murgatroid99
Has this become more feasible after the libuv abstractions?

does this need more documentation or feature support in grpc?


[nbl] Add new extractor

Please follow the guide below

You will be asked some questions, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your pull request (like that [x])
Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

[x] Searched the bugtracker for similar pull requests
[x] Read adding new extractor tutorial
[x] Read youtube-dl coding conventions and adjusted the code to meet them
[x] Covered the code with tests (note that PRs without tests will be REJECTED)
[x] Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[ ] Bug fix
[ ] Improvement
[x] New extractor
[ ] New feature

Description of your pull request and other information
This is an extractor for game VODs, condensed game VODs, and highlight clips from the website of the National Basketball League of Australia (https://nbl.com.au). At this point it supports video watch URLs at the path /tv/[category]/[id]/. All tests are passing on 2.7.18 and 3.9.11 (I wasn't able to run 2.6.* on my dev machine).
I had plans to include the latter part of the url as the display_id, but I must be misunderstanding the named capture groups in _VALID_URL, as this does not seem to be getting added to the info dict.
I'd like to support the embed url format as well, e.g https://ott.nbl.com.au/en-int/embed/1303323, as this is used in og:video tags and might be a simplification for the extractor, but I didn't want to add that functionality at this time with my limited knowledge of what's available in ytdl.

One concern I had was disambiguation, as obviously there's other organisations with the initials NBL. The Australian league seems to be the most prominent one and other regions' instances of NBL seem to have more fitting initialisms, e.g NBLC. Of course there's no problem with renaming the extractor to avoid conflicts.

One concern I had was disambiguation, as obviously there's other organisations with the initials NBL. ...

In this context, and I know little of basketball, it seems that the name you chose is uncontroversial, as the former US NBL won't be streaming anything from 70 years ago.

There doesn't seem to be any geo restriction with the media, I've successfully downloaded full games to a server hosted in the Netherlands.
The website requires login for normal users to view content, but the streaming service at ott.nbl.com.au does not, once a video ID is retrieved it can be used to download that content without authentication.

I'm investigating allowing more lenient URLs and retrieving the title information from the embed page instead, please stand by.


#1618 [0.x] remove circular dependency in 0.X version

Hello,
This PR fixes circular dependency in 0.x version.
I don't want to upgrade to 1.x because of some libs not upgraded yet.
The circular dependency is annoying with React Native because of the yellow box warnings.
I used npx madge --circular src to check it.

Q
A

Fixed Issues?
Fixes #1618

Patch: Bug Fix?
Yes

Major: Breaking Change?
No

Minor: New Feature?
No

Tests Added + Pass?
Yes

Any Dependency Changes?
No

Refactor code to remove circular dependency.
Thanks,
Julien Usson

Looks good, gonna merge it and release a patch version tomorrow. Would be good to test against it in master - rollup detects circular deps, not sure though if it's possible to fail the build automatically when such situation occurs.

I've released this as 0.16.1


Chat features

Public room chat and private is working,
You can now : create and join public room.
see the messages previously sent on this rooms.
click on someone else profile to get redirected to its page.
you can also create a private room to send message to someone else,
the send messages buttons are working, but they do no change the url, its a bti more complex than it seems, ill do it later
you cant see messages from block user, you can send message from block user.
You should be able to have a list of rooms you are in.
Also changed a bit the app function so that you are reddirected to login page
To do next
Add a delete room button to quit receiving messages
Translations !!!
Is_Invite settings in messsages so that we could send duel requests
Refreshing is kinda working but it closes the chat as the chat is not a part of url, shall we change this ?

#59 btw