Error after update to 4.89

The issue tracker is for reporting product deficiencies. "How do I?" questions should be posted to the discussion forum at https://groups.google.com/group/google-apps-manager. When in doubt, start at the discussion forum and return here only when instructed to do so.
Please confirm the following:

I have upgraded to the latest GAM release from https://git.io/gamreleases and I still have this issue. CONFIRMED
I am typing the command as described in the GAM Wiki at https://github.com/jay0lee/gam/wiki
CONFIRMED

Full steps to reproduce the issue:

Update from 4.85 to 4.89
run gam info user
Get "Select the authorized scopes be entiering a number." (Never got this screen before)
Select "c) Continue to authorization" and receive...

WARNING: [Errno 2] No such file or directory: 'E:\\GAM\\client_secrets.json'
ERROR: To use GAM you need to create an API project. Please run:
gam create project

If I try to create a new project, after the browser authorization I get the following stack trace...

Traceback (most recent call last):
  File "gam.py", line 14609, in <module>
  File "gam.py", line 13904, in ProcessGAMCommand
  File "gam.py", line 7902, in doCreateProject
  File "gam.py", line 7845, in _getLoginHintProjectId
  File "gam.py", line 7639, in getCRMService
  File "gam.py", line 7632, in _run_oauth_flow
  File "site-packages\google_auth_oauthlib\flow.py", line 440, in run_local_server
AttributeError: 'NoneType' object has no attribute 'replace'
[6552] Failed to execute script gam

Expected outcome (what are you trying to do?):  I expected that upgrading GAM wouldn't break my working installation.
Actual outcome (what errors or bad behavior do you see instead?):
See stack trace above

In what directory is 4.85 installed?
In what directory is 4.89 installed? It would appear to be E:\GAM
If they are not the same directory, copy client_secrets.json,
oauth2service.json and oauth2.txt from the 4.85 directory to the 4.89
directory.
Update your system PATH to point to the 4.89 directory.
Ross
On Fri, Jul 26, 2019 at 10:42 AM tmairegasnighto<EMAIL_ADDRESS>wrote:

The issue tracker is for reporting product deficiencies. "How do I?"
questions should be posted to the discussion forum at
https://groups.google.com/group/google-apps-manager. When in doubt, start
at the discussion forum and return here only when instructed to do so.
Please confirm the following:

I have upgraded to the latest GAM release from
https://git.io/gamreleases and I still have this issue. CONFIRMED
I am typing the command as described in the GAM Wiki at
https://github.com/jay0lee/gam/wiki
CONFIRMED

Full steps to reproduce the issue:

Update from 4.85 to 4.89
run gam info user
Get "Select the authorized scopes be entiering a number." (Never
got this screen before)
Select "c) Continue to authorization" and receive...

WARNING: [Errno 2] No such file or directory: 'E:\GAM\client_secrets.json'
ERROR: To use GAM you need to create an API project. Please run:
gam create project

If I try to create a new project, after the browser authorization I
get the following stack trace...

Traceback (most recent call last):
File "gam.py", line 14609, in
File "gam.py", line 13904, in ProcessGAMCommand
File "gam.py", line 7902, in doCreateProject
File "gam.py", line 7845, in _getLoginHintProjectId
File "gam.py", line 7639, in getCRMService
File "gam.py", line 7632, in _run_oauth_flow
File "site-packages\google_auth_oauthlib\flow.py", line 440, in run_local_server
AttributeError: 'NoneType' object has no attribute 'replace'
[6552] Failed to execute script gam
Expected outcome (what are you trying to do?): I expected that upgrading
GAM wouldn't break my working installation.
Actual outcome (what errors or bad behavior do you see instead?):
See stack trace above
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/jay0lee/GAM/issues/982?email_source=notifications&email_token=ACCTYL33X2DWTW5KZCAGCXTQBMZQDA5CNFSM4IHF35T2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HBYQQSA,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACCTYL4HBSOS632YV2BD7B3QBMZQDANCNFSM4IHF35TQ
.

--
Ross Scroggs
<EMAIL_ADDRESS>

Yep, I realized the same thing after I posted and that fixed the issue.
The installer should probably default to the same directory as the previous install.


[BUG] Invalid drawable declarations generated when decompiling Instagram <IP_ADDRESS>.120

Information

Apktool Version (apktool -version) - 2.6.0
Operating System (Mac, Linux, Windows) - Mac
APK From? (Playstore, ROM, Other) - Play Store

Facebook uses some unusual build system for their apps. Resources end up packaged inside the apk in a single folder named r, with some files referenced by more than one entry inside resources.arsc. While apktool does undo most of it correctly, it generates a file res/values/drawables.xml like this:
<?xml version="1.0" encoding="utf-8"?>
<resources>
    <item type="drawable" name="chalk_brush">R||AQAEH1e24i3A|512|512|160</item>
    <item type="drawable" name="circle">R||AQAAoKYEg-_S|256|255|160</item>
    <item type="drawable" name="direct_dark_mode_inbox_reply_pill_background_gray">@drawable/direct_inbox_reply_pill_background_gray_night</item>
    <!-- more proper resource aliases -->
    <item type="drawable" name="direct_thread_gifs_place_holder_item_shhmode">@drawable/direct_thread_gifs_place_holder_item_night</item>
    <item type="drawable" name="fr_frame">R||AQAAkUF_KkOv|1024|1024|160</item>
    <item type="drawable" name="fr_lut">R||AQAAyPlUeUNj|1024|32|160</item>
    <item type="drawable" name="frbw_frame">R||AQAAk2w7mYkq|1024|1024|160</item>
    <item type="drawable" name="frbw_lut">R||AQAB6iBI8j8n|1024|32|160</item>
    <item type="drawable" name="frbw_overlay">R||AQACuRw-mWCq|1024|1024|160</item>
    <item type="drawable" name="in_call_notif_tv_icon">R||AQAGKjMnng39|120|122|160</item>
    <item type="drawable" name="inline_searchbox_background_shhmode">@drawable/inline_search_box_background_night</item>
    <item type="drawable" name="input_background_shhmode">@drawable/input_background_night</item>
    <item type="drawable" name="instagram_add_filled_24">I1800B7</item>
    <item type="drawable" name="instagram_add_outline_24">I18007B</item>
    <!-- a whole lot more of these nonsensical lines -->
    <item type="drawable" name="ll_font">R||AQACveI2bHlA|160|96|160</item>
    <item type="drawable" name="ll_leaks">R||AQABjmVlm-FX|1024|1024|160</item>
    <item type="drawable" name="ll_lut">R||AQADDu-9mCiw|1024|32|160</item>
    <item type="drawable" name="ll_noise">R||AQADKNuSWsE5|1024|1024|160</item>
    <item type="drawable" name="marker">R||AQABJEwQfEtS|256|256|160</item>
    <item type="drawable" name="media_additional_reactors_circle_shhmode">@drawable/media_additional_reactors_circle_night</item>
    <item type="drawable" name="meme_emphasis">R||AQACINpbVTEL|1301|371|160</item>
    <item type="drawable" name="neon_bulb_asset">R||AQABxh689A9n|256|256|160</item>
    <item type="drawable" name="ps_lut">R||AQABHygJLIb3|1024|32|160</item>
    <item type="drawable" name="radial_asset">R||AQADhGZJhKUF|256|256|160</item>
    <item type="drawable" name="rainbow_brush">R||AQAAeJIP-Nxa|256|256|160</item>
    <item type="drawable" name="textedit_background_error_shhmode">@drawable/textedit_background_error_night</item>
    <item type="drawable" name="textedit_background_shhmode">@drawable/textedit_background_night</item>
    <item type="drawable" name="unified_inbox_message_mask_no_border_shhmode">@drawable/unified_inbox_message_mask_no_border_night</item>
    <item type="drawable" name="unified_inbox_message_mask_shhmode">@drawable/unified_inbox_message_mask_night</item>
    <item type="drawable" name="unified_inbox_my_message_mask_shhmode">@drawable/unified_inbox_my_message_mask_night</item>
    <item type="drawable" name="vhs_glitch_0">R||AQADL3Xyljfu|1024|1024|160</item>
    <item type="drawable" name="vhs_glitch_1">R||AQABaRWCTW4_|1024|1024|160</item>
    <item type="drawable" name="vhs_lut">R||AQABd-1upOoH|1024|128|160</item>
    <item type="drawable" name="APKTOOL_DUMMY_bd" />
    <item type="drawable" name="APKTOOL_DUMMY_105" />
    <item type="drawable" name="APKTOOL_DUMMY_199" />
    <item type="drawable" name="APKTOOL_DUMMY_19a" />
    <item type="drawable" name="APKTOOL_DUMMY_1d4" />
    <item type="drawable" name="APKTOOL_DUMMY_1d5" />
    <!-- and some more of these too — not sure why they're here at all -->
</resources>

Obviously, building this as is results in an app that crashes when it first tries to make use of a resource that decoded incorrectly.
I don't know Android internals nearly well enough, but might it be that there's some newer kind of resource reference in resources.arsc, and these are meant to be @drawable/something references?
Stacktrace/Logcat
10-04 00:33:21.495 22922 22922 D AndroidRuntime: Shutting down VM
10-04 00:33:21.497 22922 22922 E AndroidRuntime: FATAL EXCEPTION: main
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Process: com.instagram.android, PID: 22922
10-04 00:33:21.497 22922 22922 E AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.instagram.android/com.instagram.mainactivity.MainActivity}: android.view.InflateException: Binary XML file line #34: Binary XML file line #35: Error inflating class com.instagram.common.ui.base.IgSimpleImageView
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3654)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3806)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:83)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2267)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.os.Handler.dispatchMessage(Handler.java:107)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.os.Looper.loop(Looper.java:237)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.app.ActivityThread.main(ActivityThread.java:8167)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at java.lang.reflect.Method.invoke(Native Method)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.view.InflateException: Binary XML file line #34: Binary XML file line #35: Error inflating class com.instagram.common.ui.base.IgSimpleImageView
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.view.InflateException: Binary XML file line #35: Error inflating class com.instagram.common.ui.base.IgSimpleImageView
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: java.lang.reflect.InvocationTargetException
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at java.lang.reflect.Constructor.newInstance0(Native Method)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at java.lang.reflect.Constructor.newInstance(Constructor.java:343)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.view.LayoutInflater.createView(LayoutInflater.java:854)
...
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.view.LayoutInflater.inflate(LayoutInflater.java:481)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at X.1Zj.A0P(:23)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at androidx.appcompat.app.AppCompatActivity.setContentView(:268435463)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at com.instagram.base.activity.BaseFragmentActivity.onCreate(:14)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at com.instagram.mainactivity.MainActivity.onCreate(:349)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.app.Activity.performCreate(Activity.java:7963)
...
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.app.ActivityThread.main(ActivityThread.java:8167)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at java.lang.reflect.Method.invoke(Native Method)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.content.res.Resources$NotFoundException: Drawable com.instagram.android:drawable/instagram_arrow_left_outline_24 with resource ID #0x7f0804ab
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.content.res.Resources$NotFoundException: Drawable com.instagram.android:drawable/instagram_arrow_left_pano_outline_24 with resource ID #0x7f0804ae
10-04 00:33:21.497 22922 22922 E AndroidRuntime: Caused by: android.content.res.Resources$NotFoundException: File I180158 from drawable resource ID #0x7f0804ae
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.content.res.ResourcesImpl.loadDrawableForCookie(ResourcesImpl.java:999)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.content.res.ResourcesImpl.loadDrawable(ResourcesImpl.java:735)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.content.res.Resources.getDrawableForDensity(Resources.java:965)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at X.11v.getDrawableForDensity(:65)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.content.res.Resources.getDrawable(Resources.java:880)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at X.1Xt.A00(:33)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at X.1Xu.get(:18)
10-04 00:33:21.497 22922 22922 E AndroidRuntime: 	at android.content.res.ResourcesImpl.loadDrawable(ResourcesImpl.java:715)
...
10-04 00:33:21.498 22922 22922 E AndroidRuntime: 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:496)
10-04 00:33:21.498 22922 22922 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1100)
10-04 00:33:21.498 22922 22922 E AndroidRuntime: Caused by: java.io.FileNotFoundException: I180158
10-04 00:33:21.498 22922 22922 E AndroidRuntime: 	at android.content.res.AssetManager.nativeOpenNonAsset(Native Method)
10-04 00:33:21.498 22922 22922 E AndroidRuntime: 	at android.content.res.AssetManager.openNonAsset(AssetManager.java:992)
10-04 00:33:21.498 22922 22922 E AndroidRuntime: 	at android.content.res.ResourcesImpl.loadDrawableForCookie(ResourcesImpl.java:983)
10-04 00:33:21.498 22922 22922 E AndroidRuntime: 	... 58 more

Steps to Reproduce

apktool d instagram.apk
apktool b instagram

Frameworks
If this APK is from an OEM ROM (Samsung, HTC, LG). Please attach framework files
(.apks that live in /system/framework or /system/priv-app)
It is not.
APK
If this APK can be freely shared, please upload/attach a link to it.
https://www.apkmirror.com/apk/instagram/instagram-instagram/instagram-instagram-207-0-0-39-120-release/instagram-207-0-0-39-120-android-apk-download/download/
Questions to ask before submission

Have you tried apktool d, apktool b without changing anything? Yes, that's exactly what I did.
If you are trying to install a modified apk, did you resign it? Yes.
Are you using the latest apktool version? Yes.

The drawable declarations are actually being generated correctly, in the sense that they are being directly copied from the resources.asrc table. For example, for me the install always fails with
Caused by: android.content.res.Resources$NotFoundException: Drawable com.instagram.android:drawable/instagram_facebook_circle_filled_24 with resource ID #0x7f08069e
When I look up the generated entry for that drawable in values/drawables, I find an apparently nonsensical entry:
    <item type="drawable" name="instagram_facebook_circle_filled_24">I18006C</item>
However, that row is actually in the resources.asrc table. It looks like this:
ID 		       name                                                     default 0x7f08069e	instagram_facebook_circle_filled_24    I18006C
I don't know what "I18006C" is actually supposed to mean. But there are some xml entries that are decipherable. For the entries that look like this one:
<item type="drawable" name="theme_halloween_preview_icon_light">R||AQAC7IVDw74H|150|150|240</item>
The AQAC7IVDw74H is actually a base64url encoded string, which instagram parses to the url https://lookaside.facebook.com/ras/v2/?id=AQAC7IVDw74H which downloads a nice picture of a jack-o-lantern. So they clearly have some custom code for downloading resources on the fly.
So, these resources are being decoded by apktool correctly. And you can look at the resources.asrc fille of the recompiled apk and see that they are also being reencoded correctly too.
This begs the question: why is the recompiled program crashing then? I have no idea.

I appreciate the research you did as I read my email on this ticket @MartinDupont

I've found the solution.
After running apktool d instagram.apk, delete assets/drawables.bin, then run apktool b instagram, and it runs perfectly. (Caveat: I did this with instagram-215-0-0-27-359 with c libraries for a different architecture, but it should work the same). (Caveat 2: With this version, you need to manually delete a line in /res/values/styles.xml, because apktool for some reason generates a doubled item. No big deal).
I don't know what drawables.bin is, but I don't think it's a standard part of an android build. What I think is happening, is that apktool decodes files stored under /r to the /res folder, and upon recompiling, puts them back in the /res folder, and not the original folder, and in addition, it doesn't re-mangle the names. This doesn't normally cause problems for most apps, because the references are still consistent. However, what's probably happening is that drawables.bin is some library which is used to decode the funny-looking drawable declarations, and because we haven't changed it during compile/recompile, it still assumes the resources are sitting under /r with the original names. Which is why it crashes when it tries to load those resources. I don't know why the app still runs when drawables.bin is deleted. This explanation may also be wrong.
In any case, I don't think that apktool has a responsibility to deal with this. Instagram is clearly using a very strange, nonstandard hacky way of getting dynamic resources in their app, and it's clearly a unique case.
This could be solved if apktool was changed so that decompiling and recompiling returned things to exactly the way they were before, but from looking at the apktool code, I think this is too much effort for such a strange edge case.
I vote for closing the ticket

The original problem was that there were multiple references inside resources.arsc to the same file (like @drawable/thing1 and @drawable/thing2 both actually being /res/drawable/thing.png) and apktool couldn't deal with that. I'm not sure whether the problems with current versions of apktool and instagram are caused by this same issue.

I should have been a little more rigorous before. I reran the process with Apktool 2.6.0, Ubuntu 18.04 and Instagram <IP_ADDRESS>.120. I can again get it to recompile and run by just deleting drawables.bin after decompiling.
In any case, the "invalid" drawable declaration that has been generated isn't the problem, as the app runs just fine after being recompiled with those drawables, and they are a faithful copy of what's in the resources.arsc file. The Stacktrace that you get was the same one that I used to get, before I deleted the drawables.bin.
What makes you think that the duplicate references inside resources.arsc are causing an error? And what are the duplicate references? I haven't found any yet

Duplicating here from the chat. I worked around this by decompiling with --no-res, extracting the /r directory from the apk into /unknown in the project, and adding all the files under /r to unknownFiles in apktool.yml. This results in a fully working rebuilt apk.

@grishka Can you give examples of this? What does your unknownFiles section look like? Folder structure? I'm having difficulty understanding what this section of the YML should look like since it's not a standard YML map.

unknownFiles:
  DebugProbesKt.bin: '8'
  LICENSE_OFL: '8'
  LICENSE_UNICODE: '8'
  billing.properties: '0'
  firebase-annotations.properties: '0'
  firebase-common.properties: '0'
  firebase-components.properties: '0'
  firebase-iid-interop.properties: '8'
  firebase-iid.properties: '8'
  firebase-measurement-connector.properties: '8'
  firebase-messaging.properties: '8'
  play-services-auth-api-phone.properties: '0'
  play-services-auth-base.properties: '0'
  play-services-auth.properties: '0'
  play-services-base.properties: '0'
  play-services-basement.properties: '0'
  play-services-clearcut.properties: '8'
  play-services-flags.properties: '8'
  play-services-location.properties: '8'
  play-services-phenotype.properties: '8'
  play-services-places-placereport.properties: '8'
  play-services-safetynet.properties: '8'
  play-services-stats.properties: '8'
  play-services-tasks.properties: '0'
  play-services-vision-common.properties: '0'
  play-services-vision.properties: '0'
  org/assertj/core/internal/cglib/util/words.txt: '8'
  r/--.xml: "0"
  r/-.xml: "0"
  r/-0.png: "0"
  r/-1.xml: "0"
  r/-2.xml: "0"
  r/-3.png: "0"
  r/-4.9.png: "0"
  r/-5.xml: "0"
  r/-6.xml: "0"
  r/-7.webp: "0"
  r/-8.png: "0"
  r/-9.xml: "0"
  r/-_.xml: "0"
  r/-a.xml: "0"
  r/-b.glsl: "0"
  r/-c.png: "0"
  r/-d.xml: "0"
  r/-e.json: "0"
  r/-f.json: "0"
  r/-g.xml: "0"
  r/-h.png: "0"
  r/-i.png: "0"
  r/-j.xml: "0"
...

The resources themselves appear like they do in the original apk:

The XML files are binary as well (this particular one must be a state list drawable):


Data portal slow to load despite api caching in cloudfront

Underlying issue

Data portal front end points at https://api.dev.corpora.cziscience.com/ instead of http://api.cellxgene.dev.single-cell.czi.technology/

Do we want to do caching in cloudfront?
I'm thinking about the case when we add new datasets to the database.  If we cache at the cloudfront level then we won't see the new datasets, right?  (unless we have a process to clear the cache after each new dataset is added).

This is an issue @Bento007 has raised concerns about as well. We've discussed invalidating the cache when we make changes in the db. Currently the cache only lasts an hour which should help somewhat. It may be worth organizing a larger group discussion to determine the best way to approach this.


⚠ Bofrost.de has degraded performance

In 4d65abf, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1434 ms

Resolved: Bofrost.de performance has improved in 76048d4 after 22 minutes.


Scroll in an iframe inside a step

Trying to use a iframe inside one of my slides for a quick demo, I realized that the scroll doesn't work properly.  The scroll bar appears when needed, but it doesn't move the content of the frame source.
This is the code I'm using:
<div class="step" data-x="7850.666666666667" data-y="1638.4" data-scale="1" data-rotate="90"

Do you have any idea if this is a problem of my usage, or iframes are not so well supported by impress.js?
Thanks in advance.

Closing due to lack of response. If the issue is still happening, please provide more info.


[connectivity]add test driver.

Add test driver to the plugin.

lgtm modulo the nits above

looks ready to merge


DirectML 1.10 is 2x as slow as CUDA for onnxruntime on NVidia Quadro P5000

I am using onnxruntime to run stable diffusion float16 onnx. And I am getting times of 1.5 seconds per cycle for DirectML and 0.7 seconds per cycle when I switch to CUDA runtime.
I have Nvidia 1080 GTX equivalent (actually is is Quadro P5000).
Is this as expected? I am using latest DirectML.dll 1.10 from the nuget package.

Are you using ONNX Runtime Nightly? For other hardware the difference between the official release and nightly is close to 2x. Note though that to my knowledge a P5000 does not have the features to do proper FP16, your card is faster in FP32 because the chip is crippled in FP16. My laptop GTX 1070 on CUDA does 0.55s per iteration on FP32 hence outperforming your P5000 despite how weak it is in comparison.

Are you using ONNX Runtime Nightly? For other hardware the difference between the official release and nightly is close to 2x. Note though that to my knowledge a P5000 does not have the features to do proper FP16, your card is faster in FP32 because the chip is crippled in FP16. My laptop GTX 1070 on CUDA does 0.55s per iteration on FP32 hence outperforming your P5000 despite how weak it is in comparison.

Hello, I am not using the nightly build. I will try that. (I assumed that the problem bust be in the DirectML implementation rather than the onnxruntime since CUDA is so fast).
I'm pretty sure the FP16 version was faster on DirectML than the FP32 version. (BTW there are more reasons to use FP16 such as smaller file sizes and load times).
Have you tried DirectML on your GTX 1070? Can you get similar times to CUDA?

I maintain a repo focused on ONNX FP16 here: https://github.com/Amblyopius/AMD-Stable-Diffusion-ONNX-FP16. Mostly to support users with AMD cards (as they don't have the CUDA option) but happy to support Nvidia. My main card is a 6700XT so haven't tried DirectML on the GTX 1070. My understanding is that we're getting an updated ONNX Runtime in February, until then nightly is the way to go. You can always check the repo for what is the best (known to me) state of affairs.

I did a pip install of ort-nightly-directml and swapped over the DirectML.dll files but now I get an error:
Non-zero status code returned while running Trilu node. Name:'/text_model/Trilu' Status Message: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\MLOperatorAuthorImpl.cpp(1866)\onnxruntime.dll!00007FFC6FBE2A90: (caller: 00007FFC6FBE1F93) Exception(3) tid(ad0) 80070057 The parameter is incorrect.
I guess I need a new onnxruntime.dll from somewhere.

I was typing away and then you updated that you were using c#. You can get the nightly for DirectML in the same place: https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly

I was typing away and then you updated that you were using c#. You can get the nightly for DirectML in the same place: https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ORT-Nightly

Thanks! It worked!!!! 🥳🎉🎉🎉🥳 It's a miracle.
For anyone else with this problem:
I didn't need to replace the DirectML.dll file
I just needed to get the 1.14 onnx runtime developer release.
Now my DirectML times are about 0.42 seconds (compared to 0.38 for CUDA - an I'm not going to quibble about 10%). That's good enough for me 😁 (With the released runtime it was 0.9 seconds so over 2x improvement!) Now I can delete all those 2GB CUDAnn files.

Cool, happy to help! To my knowledge (but for clarity: not associated with MS or the team) the next official release is planned for February. So then you can go to the mainline and get the benefits without having to use the dev version.

It's good to be ahead of the curve. Hopefully by then they'll be able to shave off that extra 10-20% difference to match CUDA.


Cannot install under ember-cli 0.2.0

When I run ember install:addon ember-cli-materialize in a newly created ember 0.2.0 project.
The first problem encountered is jQuery incompatibility:
 conflict Unable to find suitable version for jquery
    1) jquery ^1.11.1
    2) jquery >=2.1.1
    3) jquery >= 1.7.0 < 2.2.0

Afterwards, no matter what I choose:
module.js:355
  Module._extensions[extension](this, filename);
                               ^
Error: Module did not self-register.
    at Error (native)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/private/tmp/blap/node_modules/ember-cli-sass/node_modules/broccoli-sass/node_modules/node-sass/lib/index.js:181:15)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
<EMAIL_ADDRESS>postinstall: `node scripts/build.js`
Exit status 1
Error<EMAIL_ADDRESS>postinstall: `node scripts/build.js`
Exit status 1
    at EventEmitter.<anonymous> (/private/tmp/blap/node_modules/ember-cli/node_modules/npm/lib/utils/lifecycle.js:213:16)
    at EventEmitter.emit (events.js:110:17)
    at ChildProcess.<anonymous> (/private/tmp/blap/node_modules/ember-cli/node_modules/npm/lib/utils/spawn.js:14:12)
    at ChildProcess.emit (events.js:110:17)
    at maybeClose (child_process.js:1008:16)
    at Process.ChildProcess._handle.onexit (child_process.js:1080:5)

It sounds like a general problem of node-sass 1.* with node >= 0.12: https://github.com/sass/node-sass/issues/653
You should open a ticket on https://github.com/aexmachina/ember-cli-sass
Temporarily you can downgrade node to 0.10.

I've also seen the jquery issue but don't run into the sass issue seen here with 0.2.0.
The project should probably be upgraded from the 0.2.0 beta version of ember cli to 0.2.0. WDYT?

I will do in the next days.

Opened https://github.com/aexmachina/ember-cli-sass/issues/30

@sgasser check out the comment on the linked issue. It seems like the dependency on ember-cli-sass needs to be simply bumped...

Thanks! Update solves the problem


Allow spaces in fenced code attribute

Currently things after the first space in the fenced code attribute is ignored, so this
```foo bar
lorem ipsum
```

is transformed into

lorem ipsum

While this is fine for “standard” syntax (a la CommonMark, GFM, etc.), the approach makes it impossible to extend the syntax without modifying the parser.
I propose that instead of just parsing until the first space, the lang attribute should contain all content from the last opening tick/wave to the next newline. For the above example, this means that lang will become foo bar instead of just foo. The HTML render can then do either of the followings:

Output it verbatim (with a language- prefix)

lorem ipsum

Split the attribute with space, and prefix each of the items with language- (doesn’t make sense to me)

lorem ipsum

Output up to the first space, discard the rest. This makes the output identical as before.

The benefit for this is that custom renderers can treat those extra content anyway they like. This would make it possible to implement, for example, a file name header above the code block, i.e. render this
```python hello.py
print('Hello!')
```

into something like

hello.py

  print 'Hello!'

As said, new code allows clients to get the full info string.


rpi-base: Added missing HiFiBerry

- What I did
Added missing support for HiFiBerry AMP100, AMP3, AMP4 Pro, DAC+ ADC Pro, DAC+ DSP, DAC+ HD, Digi Pro.
- How I did it
Add right lines to rpi-base.inc. Changes were tested and works fine with the branch.
Connected also with issue #1168

@agherzan What is wrong with DCO? The commit message has required formatting.

@agherzan What is wrong with DCO? The commit message has required formatting.

DCO - Developer Certificate of Origin, basically you add Signed-off-by: tag to indicate that the work came from you.

DCO - Developer Certificate of Origin, basically you add Signed-off-by: tag to indicate that the work came from you.

I know that. Strange because the commit had "Signed-off-by" section. I did rebase, removed it and added it again.

It's an edge case on the CI side. It falls apart when the incoming branch is the same as the destination branch (in name). "master" and "master" in this case. Don't worry about it.

@kluszon Can you please cleanup the merge commits?

@agherzan I removed any other changes. Please merge it.


(Chore) Remove users not in Delius from Dev and Test

The users  “tester.testy”, “bernard.beaks” and “panesar.jaspal” do not exist in Delius. Now we call Delius to check if a user can access the offender for a assessment / placement request, we need to ensure they exist in Delius, otherwise the call will fail.

Gonna close this for now


Travis CI: The sudo tag is now deprecated in Travis

sudo: required no longer is.  The sudo command is now always available in Travis CI and there is no way to turn it off.

@cclaus
Pulled this to the wrong branch (master).
Reverted and tried to pull to the right branch (dev_fall_2019) but GitHub refused to restore the original pull request.
So redid it by manual edit in the dev_fall_2019 branch.
Will be merged with master after completion of some other issues.
Thanks for the contribution!


Implement at_coroutine_exit

This is a draft for at_coroutine_exit as it is found in libunifex.
Im addressing currently the following TODOs

[x] unhandled_stop should invoke the callback
[x] fix completion_signatures for __die_on_stop

I have fixed unhandled_stopped, but It needs an intrusive change to __with_awaitable_senders_base. Do you have any better ideas on how to get the type erased stop handler?

Do you have any better ideas on how to get the type erased stop handler?

Maybe we take a page from libunifex and bundle together the parent promise's coroutine_handle and stopped callback into a continuation_handle<> struct. Then with_awaitable_senders::continuation() would return a continuation_handle<> instead of a coroutine_handle<>. Thoughts?

Eventually we'll have to make continuation_handle normative and add it to P2300, but I'd like to iterate on its design a bit. But not in this PR. I think this one is ready to go, once tests pass.
Thank you!

Hi @maikel,
In test\exec\test_at_coroutine_exit.cpp, the definition of the REQUIRE_TERMINATE macro is protected by #ifdef STDEXEC_HAS_FORK, but there are test definitions that use REQUIRE_TERMINATE and aren't protected by an #ifdef. (The invocations are protected, but not the definitions.)
This leads to compilation errors when STDEXEC_HAS_FORK is not defined, as for example on MinGW.


bump mattn/go-sqlite3 to v1.11.0

Make sure these boxes checked before submitting your pull request.

[x] Do only one thing
[x] No API-breaking changes
[x] New code/logic commented & tested

For significant changes like big bug fixes, new features, please open an issue to make an agreement on an implementation design/plan first before starting it.
What did this pull request do?
Bumped a minor version of a dependency to latest.

Codecov Report

Merging #2565 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2565   +/-   ##
=======================================
  Coverage   79.42%   79.42%
=======================================
  Files          24       24
  Lines        3466     3466
=======================================
  Hits         2753     2753
  Misses        612      612
  Partials      101      101

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 836fb2c...1648742. Read the comment docs.


How to track user online/offline based on when they connect via faye?

Sounds like this is a question for the Faye issue tracker, not sync.


2764-FE0F is describd as "heavy black heart"

I noticed there is a "name": "HEAVY BLACK HEART", in emoji_pretty.json, but the actual image(2764-fe0f.png) from a spot check are red.

Ahhh, I see on https://emojipedia.org/red-heart/ it explains why.

Ahhh, I see on https://emojipedia.org/red-heart/ it explains why.