Revert "Set ulimit for image build."

Revert bumping the ulimit in our build command. The build nodes have had their default ulimit increased within /usr/share/containers/containers.conf.
This reverts commit 8473f52536cd9f168ab47d393fd42aacd098666e, reversing changes made to c2d531b75f71d28730a1bf8a751998e94f2c5434.

/assign @jstuever


sota_sanity.bbclass is no longer compatible with oe-core master (4.2 Mickledore)

meta-updater/classes/sota_sanity.bbclass", line 100, in defaultsota_check_sanity_eventhandler(e=<bb.event.SanityCheck object at 0x7fa32faedf30>):
         if bb.event.getName(e) == "SanityCheck":
    >        sanity_data = copy_data(e)
             if e.generateevents:
NameError: name 'copy_data' is not defined

copy_data function was removed in:
https://git.openembedded.org/openembedded-core/commit/?id=d3eb4531aae28a07cb7e52ed5fe1102445d2effd

Looks like bb.data.createCopy(e.data) will do the same thing? I don't know that code too well, I don't think I touched it myself.


Document properties

The properties file contains the properties:
stopWordListName = data/CN.nw.wordlist.txt
endWordListName = data/CN.endlist.txt
forbiddenCharListName = data/CN.charlist.txt
stopThreshold = 50
forbiddenThreshold = 800
minAV = 5
minCount = 3
minDocumentCount = 5
terminologyThreshold = 0.6
Could you document what each parameter is? I think the first 3 are obvious, but the others could use some explanation.
Thanks!

Also, if there are other properties squirreled away somewhere that can affect the results, that would be useful as well!

I will document the parameters, but what matters most is using a Chinese word segmenter that works well on your data.

Agreed. Bad segmentation is impossible to recover from.

Updated README.md. Thank you!


👥 Google Workspace and Microsoft 365 groups

Create:

[ ] Google Workspace<EMAIL_ADDRESS>[ ] Microsoft 365<EMAIL_ADDRESS>

Request via ServiceNow REQ1349667

Google Workspace: RITM1652760
Microsoft 365: RITM1652759

Both RITMs closed complete, have verified sending!


[material-ui-codemod] 404 while trying to install version v1.0.0-beta.35

While running npm install on a project with dev dependency
"material-ui-codemod": "1.0.0-beta.35"
i get 404 for this package.
I created separate clean npm project and try to install it and there is:
npm ERR! code ENOVERSIONS npm ERR! No valid versions available for material-ui-codemod
After this i checked github repo and for tags 1.0.0-beta.41 and bellow there is a npm issue with following 404 for npm registry:

Is that because old packages are not longer set for build or something went wrong during the deployment?
Does anybody have any idea?

Ok, it looks like this version is no longer available. I will update it in the repo.


Archive successfully created, but fastlane reports build error

Trying to build and deploy a beta version of an app to iTunesConnect using fastlane. The lane declaration looks like:
before_all do
    cocoapods
    increment_build_number
    cert
    sigh
end

lane :beta do
    ipa(
	workspace: "[APP NAME].xcworkspace",
   	scheme: "[APP NAME]-demo"
    )
    # commented out `deliver` to see if it is causing errors, determined that it's not.
    #deliver(
    #	skip_deploy: true,
    #	beta: true
    #)
end

Running fastlane beta results in
// lots of compiling files, warnings, etc.
▸ Copying [APP NAME]/[APP NAME]-source.sqlite
▸ Processing [APP NAME]-Info.plist
▸ Generating '[APP NAME]-demo.app.dSYM'
▸ Running script 'Copy Pods Resources'
▸ Touching [APP NAME]
▸ Signing /Users/JeremyYL/Library/Developer/Xcode/DerivedData/[APP NAME]-aojamicklltpjsbgcavnjdsoxrjo/Build/Intermediates/ArchiveIntermediates/[APP NAME]-demo/InstallationBuildProductsLocation/Applications/[APP NAME]-demo.app
▸ Touching [APP NAME]
** ARCHIVE FAILED **

[11:25:02]: A build error occured. This can have many reasons, usually
[11:25:02]: it has something to do with code signing. The `ipa` action
[11:25:02]: uses `shenzhen` under the hood: https://github.com/nomad/shenzhen
[11:25:02]: The command that was used by fastlane:
[11:25:02]: krausefx-ipa build -w "[APP NAME].xcworkspace" -s "[APP NAME]-demo" -m "/Users/JeremyYL/Developer/[PROJECT FOLDER]/Distribution_[PROVISIONING PROFILE NAME].mobileprovision" --verbose | xcpretty
[11:25:02]: -------------------------------------------------------
[11:25:02]: Variable Dump:
[11:25:02]: {:DEFAULT_PLATFORM=>:ios, :ENVIRONMENT=>nil, :PLATFORM_NAME=>:ios, :LANE_NAME=>"ios beta", :BUILD_NUMBER=>46, :CERT_FILE_PATH=>"/tmp/cert/2Y5RBGP8SW-R58UK2EWSO.cer", :CERT_CERTIFICATE_ID=>"2Y5RBGP8SW", :SIGH_PROFILE_PATH=>"/Users/JeremyYL/Developer/[PROJECT FOLDER]/Distribution_[PROVISIONING PROFILE NAME].mobileprovision", :SIGH_UDID=>"5226c81a-3e67-48e6-9763-cf6628755803"}
[11:25:02]: A build error occured, this is usually related to code signing. Take a look at the error above
[11:25:03]: fastlane finished with errors
/Users/JeremyYL/.rvm/gems/ruby-2.2.0/gems/fastlane-1.3.2/lib/fastlane/actions/ipa.rb:94:in `rescue in run': A build error occured, this is usually related to code signing. Take a look at the error above (RuntimeError)
// ruby stacktrace

No extra info on what the error is. Stranger still, the archive doesn't actually fail; each time I run this a new archive appears in the Projects organizer's Archives tab. I've even uploaded one to iTunesConnect to see if it was built correctly, and iTC doesn't seem to have any issues with it.
Any tips on how to get a little more info on what's going on here? I tried to look and see if the ipa action had more verbose output, but it didn't seem like it.

Can you run
krausefx-ipa build -w "[APP NAME].xcworkspace" -s "[APP NAME]-demo" -m "/Users/JeremyYL/Developer/[PROJECT FOLDER]/Distribution_[PROVISIONING PROFILE NAME].mobileprovision" --verbose | xcpretty

and
krausefx-ipa build -w "[APP NAME].xcworkspace" -s "[APP NAME]-demo" -m "/Users/JeremyYL/Developer/[PROJECT FOLDER]/Distribution_[PROVISIONING PROFILE NAME].mobileprovision" --verbose

yourself and see if you get any useful output?

Running the first command results in similar output to running fastlane beta:
// Lots of output
▸ Processing [APP NAME]-Info.plist
▸ Generating '[APP NAME]-demo.app.dSYM'
▸ Running script 'Copy Pods Resources'
▸ Touching [APP NAME]
▸ Signing /Users/JeremyYL/Library/Developer/Xcode/DerivedData/[APP NAME]-aojamicklltpjsbgcavnjdsoxrjo/Build/Intermediates/ArchiveIntermediates/[APP NAME]-demo/InstallationBuildProductsLocation/Applications/[APP NAME]-demo.app
▸ Touching [APP NAME]
** ARCHIVE FAILED **

Running the second give a bit more detail, but nothing that helps me very much. The final part that runs is the touch command:
Touch /Users/JeremyYL/Library/Developer/Xcode/DerivedData/[APP NAME]-aojamicklltpjsbgcavnjdsoxrjo/Build/Intermediates/ArchiveIntermediates/[APP NAME]-demo/BuildProductsPath/Release-iphoneos/[APP NAME]-demo.app.dSYM
    cd /Users/JeremyYL/Developer/[PROJECT FOLDER]
    export PATH="/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/Applications/Xcode.app/Contents/Developer/usr/bin:/Users/JeremyYL/.rvm/gems/ruby-2.2.0/bin:/Users/JeremyYL/.rvm/gems/ruby-2.2.0@global/bin:/Users/JeremyYL/.rvm/rubies/ruby-2.2.0/bin:/usr/local/heroku/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/usr/texbin:/Users/JeremyYL/.scripts:/usr/local/bin:/Users/JeremyYL/.rvm/bin"
    /usr/bin/touch -c /Users/JeremyYL/Library/Developer/Xcode/DerivedData/[APP NAME]-aojamicklltpjsbgcavnjdsoxrjo/Build/Intermediates/ArchiveIntermediates/[APP NAME]-demo/BuildProductsPath/Release-iphoneos/[APP NAME]-demo.app.dSYM

** ARCHIVE FAILED **

Both commands resulted in an archive being built that is recognized by Xcode in it's Organizer.
Perhaps touch is exiting with an error code? If you need more context, I can get you more of the console output, but there is quite a bit to copy over. I wasn't sure how much would actually be helpful.

@Ziewvater Thanks for the information. You'll even see the xcodebuild command when executing the command without the xcprety. Can you try running the xcodebuild command only? It will be printed out right after running the command.

xcodebuild command I found was
xcodebuild -sdk iphoneos -workspace "[APP NAME].xcworkspace" -scheme "[APP NAME]-demo" -configuration 'Release' clean build archive

The archive still reports as failing, but the archive appears in Xcode's Organizer. The console printout seems to be the same as when running through fastlane. Is there anything I can look for that would help better diagnose this issue?

I'm seeing the exact same thing as reported above. Here's a slightly redacted version of what I see.
krausefx-ipa build -s "[REDACTED SCHEME NAME]" -c "Release" -m "[REDACTED].mobileprovision" -i "iPhone Distribution: [REDACTED] ([REDACTED])" --verbose
....
Touch /Users/richmischook/Library/Developer/Xcode/DerivedData/[REDACTED SCHEME NAME]-hiifyztwiozhldbqaanwwdzpjsbr/Build/Intermediates/ArchiveIntermediates/[REDACTED SCHEME NAME]/BuildProductsPath/Release-iphoneos/[REDACTED APP NAME].app.dSYM
cd /Users/richmischook/development/github/[REDACTED APP NAME]
export PATH="/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/Applications/Xcode.app/Contents/Developer/usr/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/richmischook/gradle-2.4/bin/"
/usr/bin/touch -c /Users/richmischook/Library/Developer/Xcode/DerivedData/[REDACTED SCHEME NAME]-hiifyztwiozhldbqaanwwdzpjsbr/Build/Intermediates/ArchiveIntermediates/[REDACTED SCHEME NAME]/BuildProductsPath/Release-iphoneos/[REDACTED APP NAME].app.dSYM
** ARCHIVE FAILED **
As reported by @Ziewvater the archive IS created and shows up in Organiser and indeed Crashlytics App sees it.
Further info:
richmischook$ krausefx-ipa --version
krausefx-ipa 0.14.2
richmischook$ deliver --version
deliver 0.12.0
richmischook$ fastlane --version
fastlane 1.3.2

UPDATE: turns out my Scheme was building my 'Acceptance Tests' target - which was un-signable due to bundle id conflict. Removing that led to new issues. It seems that 'fastlane deploy' is ignoring the 'embed' parameter for ipa as specified in the Fastfile. In the 'krausefx-ipa build' command that is generated it is referencing - I think - the NAME of the mobile provisioning file specified in Xcode but perhaps expecting it to exist in the project directory. So the following error is generated:
▸ Touching [REDACTED APP NAME].app.dSYM
error: Unable to copy '[REDACTED PROVISIONING PROFILE FILE NAME].mobileprovision' to '/var/folders/j1/m7n83v850wj1yf40zj7f0jv00000gn/T/gOgsPzbffM/Payload/[[REDACTED APP NAME]].app/embedded.mobileprovision'
If I call 'krausefx-ipa build ....' directly and specify the mobile provisioning profile to use with the -m switch - it works. Alternatively if I rename the .mobileprovision file pulled down by SIGH it works.

Maybe I should be using SPACESHIP to pull down the Profile I need and write it out to the file name @KrauseFx?

Thanks @rmischook, that was my problem too. I had to uncheck the test target under Run in the build section of my scheme.

@samsonjs @rmischook unchecking the test target under Run in the build section of my scheme fixed it for me too. Thanks!

Great to hear this issue is resolved now. Feel free to submit a new one if you run into any more problems.

@KrauseFx I had this issue too - out of interest, why does Fastlane/shenzhen uses the 'run' process to create the IPA? I'd have assumed it would use the archive one?

@craigmarvelley no reason, that's just how shenzhen works right now. I'm working on a new tool that will replace shenzhen soon :+1:

Same issue here! I was also expecting an archive operation. @KrauseFx, when you say you are working now in a new tool, do you expect it within weeks time or months time to be released? I am setting up a new CI server and I would love to see this in there :)
Thank you so much for your awesome work!

@angelolloqui You can already give the new tool a try here. Let me know of anything that should be improved before the initial release :+1:

I just tried and I get an error:
Results at '/var/folders/30/1czz094d4wg6mqy86n3pkj_c0000gp/T/864E0865-D95C-4856-A8F2-B5B8318D0FFB-9526-00001BA6C8548725/TheCorner.ipa'
Moving exported product to '/Users/Angel/Library/Developer/Xcode/Archives/2015-08-10/TheCorner.ipa'
** EXPORT SUCCEEDED **
/Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/error_handler.rb:74:in `handle_package_error': Error packaging up the application (RuntimeError)
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/runner.rb:78:in `block in package_app'
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/runner.rb:162:in `call'
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/runner.rb:162:in `execute_command'
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/runner.rb:77:in `package_app'
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/runner.rb:11:in `run'
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/manager.rb:8:in `work'
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/commands_generator.rb:46:in `block (2 levels) in run'
	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in `call'
	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in `call'
	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:153:in `run'
	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:428:in `run_active_command'
	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:68:in `run!'
	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/delegates.rb:15:in `run!'
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/commands_generator.rb:63:in `run'
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/lib/gym/commands_generator.rb:19:in `start'
	from /Library/Ruby/Gems/2.0.0/gems/gym-0.0.1/bin/gym:6:in `<top (required)>'
	from /usr/bin/gym:23:in `load'
	from /usr/bin/gym:23:in `<main>'

but the nice thing about your tool vs shenzhen is that I got a better message just before this error:
### Checking original app
+ /usr/bin/codesign --verify -vvvv /var/folders/30/1czz094d4wg6mqy86n3pkj_c0000gp/T/864E0865-D95C-4856-A8F2-B5B8318D0FFB-9526-00001BA6C8548725/TheCorner.app
Program /usr/bin/codesign returned 1 : [/var/folders/30/1czz094d4wg6mqy86n3pkj_c0000gp/T/864E0865-D95C-4856-A8F2-B5B8318D0FFB-9526-00001BA6C8548725/TheCorner.app: a sealed resource is missing or invalid
file missing: /private/var/folders/30/1czz094d4wg6mqy86n3pkj_c0000gp/T/864E0865-D95C-4856-A8F2-B5B8318D0FFB-9526-00001BA6C8548725/TheCorner.app/archived-expanded-entitlements.xcent
]
Codesign check fails : /var/folders/30/1czz094d4wg6mqy86n3pkj_c0000gp/T/864E0865-D95C-4856-A8F2-B5B8318D0FFB-9526-00001BA6C8548725/TheCorner.app: a sealed resource is missing or invalid
file missing: /private/var/folders/30/1czz094d4wg6mqy86n3pkj_c0000gp/T/864E0865-D95C-4856-A8F2-B5B8318D0FFB-9526-00001BA6C8548725/TheCorner.app/archived-expanded-entitlements.xcent

So at least it is providing me with valuable feedback... I will continue checking why this happens later this week (I have no time now).
Thanks!

@angelolloqui yes, that was the goal of gym: You get a minimal output when everything works fine and get as much relevant information as possible if the build fails.


Error: Failed to create session. for mobile chrome browser

browsers.json
"chrome-mobile": {
    "default": "mobile-75.0",
    "versions": {
      "mobile-75.0": {
        "image": "selenoid/chrome-mobile:75.0",
        "port": "4444",
        "path": "/wd/hub"
      }
    }
  }

config.js
{
                //hostname: 'selenoid.mobile',
                maxInstances: 1,
                browserName: 'chrome-mobile',
                browserVersion: 'mobile-75.0',
                'selenoid:options': {
                    enableVnc: true,
                    enableLog: true,
                },
            }

Services in config file
services: [
            [
                'selenoid-standalone', {
                pathToBrowsersConfig: './browsers.json',
                customSelenoidContainerName: 'wdio-selenoid',
            },
            ],
        ],

package:
{
    "devDependencies": {
        "wdio-selenoid-standalone-service": "^1.0.0"
    }
}

can some one help what am missing here? am working using webdriverIO

@vania-pooh sorry, am not getting you.
if I want to run mobile tests what setup I can do ...my os is Mac
can you please guide me?

@RollerIn no, that's impossible. Will work only under Linux. Docker for Mac does not forward virtualization to containers.

@vania-pooh ,I am really in confusion mode, I have pulled amd64/ubantu image and used same as above config ..but still getting issue , dont mind can you suggest me what aim missing here ...
Request failed with status 400 due to invalid argument: Requested environment is not available
[0-0] 2022-02-22T11:40:49.864Z ERROR webdriver: invalid argument: Requested environment is not available

do I need to pass anything here additionally
services: [
            [
                'selenoid-standalone', {
                pathToBrowsersConfig: './browsers.json',
                customSelenoidContainerName: 'wdio-selenoid',
            },
            ],
        ],

@RollerIn you are trying to request a browser that is missing in browsers.json.

Can anyone guide me what steps need to take care from scratch to set up for android mobile setup to run on solenoid?
Really worrying to run mobile browser on Selenoid.
I have pulled amd64/ubantu and did above setup but still not able to achieve it.

@RollerIn
  "chrome-mobile": {
    "default": "mobile-75.0",

Browser name should be just chrome. Only version can contain mobile. This is hard-coded in Appium.

Thank you @vania-pooh , after I have change to chrome , I can see Selenoid UI with but failed to create session

Error log:
 request failed due to response error: session not created
ERROR webdriver: Request failed with status 500 due to session not created: wait: http://<IP_ADDRESS>:4444/wd/hub does not respond in 30s
ERROR webdriver: session not created: wait: http://<IP_ADDRESS>:4444/wd/hub does not respond in 30s

@RollerIn increase session startup timeout to 2 minutes with -session-attempt-timeout 2m flag. 30 seconds is too small for Android.

@vania-pooh even after increasing 2m,3m,5m,,no use
even tried manually on seleniodUI to create sessions for both android and mobile chrome..but failed to create sessions

log:
TimeoutError: Timeout awaiting 'request' for 120000ms when running "http://<IP_ADDRESS>:4444/wd/hub/session" with method "POST" and args "{"capabilities":{"alwaysMatch":{"browserName":"chrome","browserVersion":"mobile-75.0","selenoid:options":{"enableVnc":true,"enableLog":true}},"firstMatch":[{}]},"desiredCapabilities":{"browserName":"chrome","browserVersion":"mobile-75.0","selenoid:options":{"enableVnc":true,"enableLog":true}}}"
still am I missing anything is here

@vania-pooh Android images only work under Linux with virtualization support enabled.

sorry asking again, if possible can you share what exactly I need to do , am new to this and not aware on "Linux with virtualization support enabled." ..can I get any reference link to understand step by step

@RollerIn either take a Linux hardware server or a virtual machine with nested virtualization enabled. Such machines can be e.g. launched in Google Cloud (https://cloud.google.com/compute/docs/instances/nested-virtualization/enabling). In AWS I think only *.metal instances will work.

I use Linux with ubuntu where nested virtualisation is enabled and I am trying to work with mobile-chrome with WebdriverIO.
Below is the error I came across while trying to run in mobile-chrome,
" [mobile_chrome mobile-86.0 Android #0-0] An unknown server-side error occurred while processing the command. Original error: URI and package arguments are required ".
I am using the capabilities as mentioned in selenoid ui and my mobile-chrome image is already pulled from docker and consumed by selenoid UI. Also I could see session getting created but not proceeding further with the tests. Not sure whether this is a bug or some missing configuration which is not updated in 'selenoid ui capabilities' for JavaScript.
Attached screenshots below for reference.

Note : I am not using any selenoid related service from wdio as it seems to be very slow and not working due to lack of maintenance and I am connecting to solenoid directly by mentioning the port 4444 referring to the localhost. I made my tests working with web browsers but struck with mobile-chrome which is part of requirement.
@vania-pooh Any help would be appreciated. Thanks !!

@HarikrishnanVK you need to pass app capability with APK URL.

@vania-pooh I am trying to test web app with mobile-chrome image provided by selenoid. And it's already in place. Not sure which apk url I need to pass. I am not testing any native app here with apk extension. I assume that mobile-chrome image of selenoid should have chrome configured inside the mobile and it should be able to pick the test without the help of appium service.
Can I get some working set of capabilities for running this. I guess the capabilities provided in selenoid ui isn't enough. Also Could not find one in the official documention.

@HarikrishnanVK what Docker image are you using with Selenoid?

@vania-pooh selenoid chrome-mobile . Version 86.  https://hub.docker.com/layers/selenoid/chrome-mobile/86.0/images/sha256-e42762253bc2608e3ccfae7318481e2660a57f566b63a7c1e5e73036951ae2de?context=explore

@HarikrishnanVK browserName should be always chrome but browserVersion can be any value, e.g. mobile-86.0.

It works. Thanks. Additionally I also changed the Json title of the browser from browsers.json as selenoid picks the name of its browser from browsers.json file.

This sets the limitation of having only either web chrome or mobile chrome in the browsers.json but can not have both.

Guess that's why document suggests to have the name as 'mobile-chrome' for mobile execution in browsers.json. But some how, the logic expects only the 'chrome' keyword for the execution to be triggered on chrome browser.
Ideally we should have a unique identifier to identify the browsers of desktop/mobile environment. This could be a bug.

@HarikrishnanVK you can put all versions under the same chrome key in browsers.json.

As an alternative, I tried to set device resolution using 'screenResolution' capability but that didn't had any effect on selenoid UI looking at the emulator. It just behaves like desktop browser.
Resolution capability used in selenoid options:

Android emulator image in Selenoid UI which remains unchanged after passing the capability:

Expected one should be like Android Studio emulator image that runs in local:

From the above images, you can see the difference between selenoid and android studio emulators. Test fails as selenoid is not setting the resolution according to the capability.

@HarikrishnanVK you can rebuild image from source code to use any device and skin. https://github.com/aerokube/images/blob/master/selenium/automate_android.sh


[Bug][Workflow instance] task instance loss

Sometimes the task cannot be executed correctly, it is shown that the task instance is killed, there is no obvious error when viewing the master log, and the task is not submitted to the worker. This problem does not often occur, and it runs normally after re-running. May I ask why this is caused?

任务有时候不能正确执行，查看任务实例显示被kill掉，查看master日志并没有什么明显错误，并且任务也没有被提交到worker里，这个问题不是经常出现，并且重跑后运行正常，请问下这是什么原因导致的。

which version?

no response from user, I will close this issue temporarily,  please re-open it when issue update


Release latest changes as version 1.1

Requires digging into how the Apple Store and Google Play work.

Apple version is out.  Been having trouble with the Android version.  At this point need some testing to verify that the latest fix works.

Done.


Exclude web-app folder from plugin.

Ran into some issues with assets from the redis plugin overriding assets from other plugins, I think it can all be excluded.

Yep, that should have been excluded, thanks for the PR!

published as 2.0.1

Thanks, do I need to do anything special to backport to 1.6.4? I branched off of the tag but when I compare there's no button to open a PR.

I think there needs to be a branch in this repo at tag 1.6.4 for me to PR onto it.

Created a grails_2 branch from 1.6.4.


arch_native_linux is actually called native_arch_linux

arch_native_linux is actually called native_arch_linux down below in the .bazelrc where it's actually being consumed.

@googlebot I signed it!

Please merge with #37850 since both touch the same file

I merged it and will close the other one.


Is it possible to get normalized bbox coordinates from the inference_detector apis

I get the following results from the inference_detector API. Just wanted to enquire whether is it possible to get normalized bbox coordinates using some inbuilt function or do I need to write a custom function for the same.
array([[377.76096  , 254.91911  , 602.107    , 533.1789   ,   0.8144508]],
dtype=float32),
array([[2.5211449e+02, 4.9718927e+02, 3.4354926e+02, 6.1962537e+02,
1.3233066e-01]], dtype=float32),
array([[5.5296722e+02, 3.0450262e+02, 7.8253992e+02, 5.9926660e+02,
1.2978064e-01],
[2.5386581e+02, 4.9311365e+02, 3.3906073e+02, 6.1097095e+02,
3.2496080e-01]], dtype=float32)
Thanks,
Chandan Verma

You mean normalizing the bboxes to [0, 1]? There is currently no builtin options and you need to implement it.

Hi, @hellock
Is there any function like this in mmdet now?


Tapping the "1 Reply" text sometimes doesn't show the original message

Unsure why this even happens. It only happens sometimes. But we can investigate it whenever

this should be fixed now, was a null exception


how to get left and right boundaries of the current road

the snippet in the method Observer()in graph_observer.py:
    # the distance between the ego agent and boundaries of the road
    distance = []
    num_lines = len(list(world.ego_agent.road_corridor.lane_corridors))
    distance.append(Distance(world.ego_agent.GetPolygonFromState(world.ego_agent.state),
                    world.ego_agent.road_corridor.lane_corridors[num_lines-1].right_boundary))
    distance.append(-Distance(world.ego_agent.GetPolygonFromState(world.ego_agent.state),
                    world.ego_agent.road_corridor.lane_corridors[0].left_boundary))
    distance.sort(key=abs)
    obs[-2] = self._normalize_value(distance[0], [0, 3*num_lines])

I want to know a little about the snippet:

does world.ego_agent.road_corridor simply hold the road that ego agent is currently in or all roads in the map?
does barkhave an interface to provide the number of lanes of the road ego agent is currently in?

its the road_corridor specific to the ego agent
road_corridor.lane_corridors are all lanes within a road.

I will close this issue for now due to inactivity. Feel free to re-open this issue if it still persists!


My transaction history is lost after change browser

When i changed my browser to chrome, my transaction history is lost,  same jsonfile, only account name is different.

This is duplicated to #37

This is one of the limitation of decentralized wallet. We dont keep any of users' data on any centralized storage (servers or cloud) so your transaction history is only available on one browser even if you are using the same address.
In theory, we can fetch all of the transactions of the address from the chain and filter them but that's too complicated for little usage value.
I'm going to close this issue. If you do think that we should support it, please reopen this issue. Thank you.


Tests won't run

I was trying to run the tests so I can pull a fix for my other issue. But I can't get the tests to run. When I do I get:

1..8
not ok 1 py3.6 can package requests with default options
# (in test file test.bats, line 20)
#   `sls package' failed
# npm WARN<EMAIL_ADDRESS>No description
# npm WARN<EMAIL_ADDRESS>No repository field.
#
# +<EMAIL_ADDRESS># added 44 packages in 11.518s
#
#   Serverless Error ---------------------------------------
#
#   Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#
#   Stack Trace --------------------------------------------
#
# ServerlessError: Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#     at plugins.forEach (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:81:17)
#     at Array.forEach (native)
#     at PluginManager.loadPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:68:13)
#     at PluginManager.loadServicePlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:109:10)
#     at PluginManager.loadAllPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:64:10)
#     at service.load.then (/home/winstonewert/n/lib/node_modules/serverless/lib/Serverless.js:64:28)
#     at <anonymous>
# From previous event:
#     at runCallback (timers.js:781:20)
#     at tryOnImmediate (timers.js:743:5)
#     at processImmediate [as _immediateCallback] (timers.js:714:5)
# From previous event:
#     at __dirname (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:25:46)
#     at Object.<anonymous> (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:43:4)
#     at Module._compile (module.js:569:30)
#     at Object.Module._extensions..js (module.js:580:10)
#     at Module.load (module.js:503:32)
#     at tryModuleLoad (module.js:466:12)
#     at Function.Module._load (module.js:458:3)
#     at Function.Module.runMain (module.js:605:10)
#     at startup (bootstrap_node.js:158:16)
#     at bootstrap_node.js:575:3
#
#   Get Support --------------------------------------------
#      Docs:          docs.serverless.com
#      Bugs:          github.com/serverless/serverless/issues
#      Forums:        forum.serverless.com
#      Chat:          gitter.im/serverless/serverless
#
#   Your Environment Information -----------------------------
#      OS:                     linux
#      Node Version:           8.2.1
#      Serverless Version:     1.19.0
#
#
#   Serverless Error ---------------------------------------
#
#   Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#
#   Stack Trace --------------------------------------------
#
# ServerlessError: Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#     at plugins.forEach (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:81:17)
#     at Array.forEach (native)
#     at PluginManager.loadPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:68:13)
#     at PluginManager.loadServicePlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:109:10)
#     at PluginManager.loadAllPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:64:10)
#     at service.load.then (/home/winstonewert/n/lib/node_modules/serverless/lib/Serverless.js:64:28)
#     at <anonymous>
# From previous event:
#     at runCallback (timers.js:781:20)
#     at tryOnImmediate (timers.js:743:5)
#     at processImmediate [as _immediateCallback] (timers.js:714:5)
# From previous event:
#     at __dirname (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:25:46)
#     at Object.<anonymous> (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:43:4)
#     at Module._compile (module.js:569:30)
#     at Object.Module._extensions..js (module.js:580:10)
#     at Module.load (module.js:503:32)
#     at tryModuleLoad (module.js:466:12)
#     at Function.Module._load (module.js:458:3)
#     at Function.Module.runMain (module.js:605:10)
#     at startup (bootstrap_node.js:158:16)
#     at bootstrap_node.js:575:3
#
#   Get Support --------------------------------------------
#      Docs:          docs.serverless.com
#      Bugs:          github.com/serverless/serverless/issues
#      Forums:        forum.serverless.com
#      Chat:          gitter.im/serverless/serverless
#
#   Your Environment Information -----------------------------
#      OS:                     linux
#      Node Version:           8.2.1
#      Serverless Version:     1.19.0
#
not ok 2 py3.6 can package requests with zip option
# (in test file test.bats, line 27)
#   `sls package' failed
# npm WARN<EMAIL_ADDRESS>No description
# npm WARN<EMAIL_ADDRESS>No repository field.
#
# +<EMAIL_ADDRESS># added 44 packages in 1.289s
#
#   Serverless Error ---------------------------------------
#
#   Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#
#   Stack Trace --------------------------------------------
#
# ServerlessError: Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#     at plugins.forEach (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:81:17)
#     at Array.forEach (native)
#     at PluginManager.loadPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:68:13)
#     at PluginManager.loadServicePlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:109:10)
#     at PluginManager.loadAllPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:64:10)
#     at service.load.then (/home/winstonewert/n/lib/node_modules/serverless/lib/Serverless.js:64:28)
#     at <anonymous>
# From previous event:
#     at runCallback (timers.js:781:20)
#     at tryOnImmediate (timers.js:743:5)
#     at processImmediate [as _immediateCallback] (timers.js:714:5)
# From previous event:
#     at __dirname (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:25:46)
#     at Object.<anonymous> (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:43:4)
#     at Module._compile (module.js:569:30)
#     at Object.Module._extensions..js (module.js:580:10)
#     at Module.load (module.js:503:32)
#     at tryModuleLoad (module.js:466:12)
#     at Function.Module._load (module.js:458:3)
#     at Function.Module.runMain (module.js:605:10)
#     at startup (bootstrap_node.js:158:16)
#     at bootstrap_node.js:575:3
#
#   Get Support --------------------------------------------
#      Docs:          docs.serverless.com
#      Bugs:          github.com/serverless/serverless/issues
#      Forums:        forum.serverless.com
#      Chat:          gitter.im/serverless/serverless
#
#   Your Environment Information -----------------------------
#      OS:                     linux
#      Node Version:           8.2.1
#      Serverless Version:     1.19.0
#
#
#   Serverless Error ---------------------------------------
#
#   Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#
#   Stack Trace --------------------------------------------
#
# ServerlessError: Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#     at plugins.forEach (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:81:17)
#     at Array.forEach (native)
#     at PluginManager.loadPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:68:13)
#     at PluginManager.loadServicePlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:109:10)
#     at PluginManager.loadAllPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:64:10)
#     at service.load.then (/home/winstonewert/n/lib/node_modules/serverless/lib/Serverless.js:64:28)
#     at <anonymous>
# From previous event:
#     at runCallback (timers.js:781:20)
#     at tryOnImmediate (timers.js:743:5)
#     at processImmediate [as _immediateCallback] (timers.js:714:5)
# From previous event:
#     at __dirname (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:25:46)
#     at Object.<anonymous> (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:43:4)
#     at Module._compile (module.js:569:30)
#     at Object.Module._extensions..js (module.js:580:10)
#     at Module.load (module.js:503:32)
#     at tryModuleLoad (module.js:466:12)
#     at Function.Module._load (module.js:458:3)
#     at Function.Module.runMain (module.js:605:10)
#     at startup (bootstrap_node.js:158:16)
#     at bootstrap_node.js:575:3
#
#   Get Support --------------------------------------------
#      Docs:          docs.serverless.com
#      Bugs:          github.com/serverless/serverless/issues
#      Forums:        forum.serverless.com
#      Chat:          gitter.im/serverless/serverless
#
#   Your Environment Information -----------------------------
#      OS:                     linux
#      Node Version:           8.2.1
#      Serverless Version:     1.19.0
#
ok 3 # skip (can't dockerize on linux if not root & not in docker group) py3.6 can package requests with zip & dockerizePip option
ok 4 # skip (can't dockerize on linux if not root & not in docker group) py3.6 can package requests with dockerizePip option
not ok 5 py2.7 can package requests with default options
# (in test file test.bats, line 52)
#   `sls package' failed
# npm WARN<EMAIL_ADDRESS>No description
# npm WARN<EMAIL_ADDRESS>No repository field.
#
# +<EMAIL_ADDRESS># added 44 packages in 1.291s
#
#   Serverless Error ---------------------------------------
#
#   Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#
#   Stack Trace --------------------------------------------
#
# ServerlessError: Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#     at plugins.forEach (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:81:17)
#     at Array.forEach (native)
#     at PluginManager.loadPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:68:13)
#     at PluginManager.loadServicePlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:109:10)
#     at PluginManager.loadAllPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:64:10)
#     at service.load.then (/home/winstonewert/n/lib/node_modules/serverless/lib/Serverless.js:64:28)
#     at <anonymous>
# From previous event:
#     at runCallback (timers.js:781:20)
#     at tryOnImmediate (timers.js:743:5)
#     at processImmediate [as _immediateCallback] (timers.js:714:5)
# From previous event:
#     at __dirname (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:25:46)
#     at Object.<anonymous> (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:43:4)
#     at Module._compile (module.js:569:30)
#     at Object.Module._extensions..js (module.js:580:10)
#     at Module.load (module.js:503:32)
#     at tryModuleLoad (module.js:466:12)
#     at Function.Module._load (module.js:458:3)
#     at Function.Module.runMain (module.js:605:10)
#     at startup (bootstrap_node.js:158:16)
#     at bootstrap_node.js:575:3
#
#   Get Support --------------------------------------------
#      Docs:          docs.serverless.com
#      Bugs:          github.com/serverless/serverless/issues
#      Forums:        forum.serverless.com
#      Chat:          gitter.im/serverless/serverless
#
#   Your Environment Information -----------------------------
#      OS:                     linux
#      Node Version:           8.2.1
#      Serverless Version:     1.19.0
#
#
#   Serverless Error ---------------------------------------
#
#   Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#
#   Stack Trace --------------------------------------------
#
# ServerlessError: Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#     at plugins.forEach (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:81:17)
#     at Array.forEach (native)
#     at PluginManager.loadPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:68:13)
#     at PluginManager.loadServicePlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:109:10)
#     at PluginManager.loadAllPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:64:10)
#     at service.load.then (/home/winstonewert/n/lib/node_modules/serverless/lib/Serverless.js:64:28)
#     at <anonymous>
# From previous event:
#     at runCallback (timers.js:781:20)
#     at tryOnImmediate (timers.js:743:5)
#     at processImmediate [as _immediateCallback] (timers.js:714:5)
# From previous event:
#     at __dirname (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:25:46)
#     at Object.<anonymous> (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:43:4)
#     at Module._compile (module.js:569:30)
#     at Object.Module._extensions..js (module.js:580:10)
#     at Module.load (module.js:503:32)
#     at tryModuleLoad (module.js:466:12)
#     at Function.Module._load (module.js:458:3)
#     at Function.Module.runMain (module.js:605:10)
#     at startup (bootstrap_node.js:158:16)
#     at bootstrap_node.js:575:3
#
#   Get Support --------------------------------------------
#      Docs:          docs.serverless.com
#      Bugs:          github.com/serverless/serverless/issues
#      Forums:        forum.serverless.com
#      Chat:          gitter.im/serverless/serverless
#
#   Your Environment Information -----------------------------
#      OS:                     linux
#      Node Version:           8.2.1
#      Serverless Version:     1.19.0
#
not ok 6 py2.7 can package requests with zip option
# (in test file test.bats, line 59)
#   `sls package' failed
# npm WARN<EMAIL_ADDRESS>No description
# npm WARN<EMAIL_ADDRESS>No repository field.
#
# +<EMAIL_ADDRESS># added 44 packages in 1.387s
#
#   Serverless Error ---------------------------------------
#
#   Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#
#   Stack Trace --------------------------------------------
#
# ServerlessError: Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#     at plugins.forEach (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:81:17)
#     at Array.forEach (native)
#     at PluginManager.loadPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:68:13)
#     at PluginManager.loadServicePlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:109:10)
#     at PluginManager.loadAllPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:64:10)
#     at service.load.then (/home/winstonewert/n/lib/node_modules/serverless/lib/Serverless.js:64:28)
#     at <anonymous>
# From previous event:
#     at runCallback (timers.js:781:20)
#     at tryOnImmediate (timers.js:743:5)
#     at processImmediate [as _immediateCallback] (timers.js:714:5)
# From previous event:
#     at __dirname (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:25:46)
#     at Object.<anonymous> (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:43:4)
#     at Module._compile (module.js:569:30)
#     at Object.Module._extensions..js (module.js:580:10)
#     at Module.load (module.js:503:32)
#     at tryModuleLoad (module.js:466:12)
#     at Function.Module._load (module.js:458:3)
#     at Function.Module.runMain (module.js:605:10)
#     at startup (bootstrap_node.js:158:16)
#     at bootstrap_node.js:575:3
#
#   Get Support --------------------------------------------
#      Docs:          docs.serverless.com
#      Bugs:          github.com/serverless/serverless/issues
#      Forums:        forum.serverless.com
#      Chat:          gitter.im/serverless/serverless
#
#   Your Environment Information -----------------------------
#      OS:                     linux
#      Node Version:           8.2.1
#      Serverless Version:     1.19.0
#
#
#   Serverless Error ---------------------------------------
#
#   Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#
#   Stack Trace --------------------------------------------
#
# ServerlessError: Serverless plugin "serverless-python-requirements" not found. Make sure it's installed and listed in the "plugins" section of your serverless config file.
#     at plugins.forEach (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:81:17)
#     at Array.forEach (native)
#     at PluginManager.loadPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:68:13)
#     at PluginManager.loadServicePlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:109:10)
#     at PluginManager.loadAllPlugins (/home/winstonewert/n/lib/node_modules/serverless/lib/classes/PluginManager.js:64:10)
#     at service.load.then (/home/winstonewert/n/lib/node_modules/serverless/lib/Serverless.js:64:28)
#     at <anonymous>
# From previous event:
#     at runCallback (timers.js:781:20)
#     at tryOnImmediate (timers.js:743:5)
#     at processImmediate [as _immediateCallback] (timers.js:714:5)
# From previous event:
#     at __dirname (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:25:46)
#     at Object.<anonymous> (/home/winstonewert/n/lib/node_modules/serverless/bin/serverless:43:4)
#     at Module._compile (module.js:569:30)
#     at Object.Module._extensions..js (module.js:580:10)
#     at Module.load (module.js:503:32)
#     at tryModuleLoad (module.js:466:12)
#     at Function.Module._load (module.js:458:3)
#     at Function.Module.runMain (module.js:605:10)
#     at startup (bootstrap_node.js:158:16)
#     at bootstrap_node.js:575:3
#
#   Get Support --------------------------------------------
#      Docs:          docs.serverless.com
#      Bugs:          github.com/serverless/serverless/issues
#      Forums:        forum.serverless.com
#      Chat:          gitter.im/serverless/serverless
#
#   Your Environment Information -----------------------------
#      OS:                     linux
#      Node Version:           8.2.1
#      Serverless Version:     1.19.0
#
ok 7 # skip (can't dockerize on linux if not root & not in docker group) py2.7 can package requests with zip & dockerizePip option
ok 8 # skip (can't dockerize on linux if not root & not in docker group) py2.7 can package requests with dockerizePip option

Hmm. That is odd. What happens if you cd example && npm i .. ? (that's what the 1st setup step of the tests do.
Feel free to push the branch and open a PR so our CircleCI will run the tests for you.

Ok, I figured out:

You need serverless installed globally
You also need to run npm install on the root of the repo, the test running npm install .. isn't sufficient.

I don't have a pull request to try, as the first thing I did was attempt to get the test running.

Yup. Definitely need sls installed globally, the later I didn't realize, I don't think it does in CI :man_shrugging:

I can see it running the npm install when I look at the CI logs, so that's explains that.

Oh, you're right, circle.yml does that before bats gets called.


Make schema/dataset property part of table scan specification

Is your feature request related to a problem? Please describe.
Use case:
When having multiple datasets or schemas, the mono-configuration of the warehouse.yml expects that all tables are available in one schema/dataset. The idea is that the property should be part of each table YML specification.
The below is my attempt to change it to a placeholder schema.
(soda-sql)seba% soda scan warehouse.yml ./tables/code_to_division_mapping.yml

google.api_core.exceptions.NotFound: 404 Not found: Table umg-de:sodasql.code_to_division_mapping was not found in location US

(job ID: xxxxxxxxx)

     -----Query Job SQL Follows-----

    |    .    |    .    |    .    |
   1:SELECT
   2:  COUNT(*)
   3:FROM `sodasql.code_to_division_mapping`
    |    .    |    .    |    .    |
  | Test error for "row_count > 0": name 'row_count' is not defined
  | Executed 1 queries in 0:00:01.690635
  | Scan summary ------
  | 1 measurements computed
  | 1 tests executed
  | 1 of 1 tests failed:
  |   Test test(row_count > 0) failed with measurements null
  | Errors occurred!
  |   [error] Exception during aggregation query
  |   [test_execution_error] Test "row_count > 0" failed
  | Exiting with code 1
  | Starting new HTTPS connection (1): collect.soda.io:443
  | https://collect.soda.io:443 "POST /v1/traces HTTP/1.1" 200 0

Describe the solution you'd like
property schema/dataset should be at table.yml specification and not at warehouse.yml.
Soda SQL Version: 2.1.2

@vijaykiran are you Ok if I propose a PR?

@sebastianvillarroel  Sorry for the delay .. can you please write a short summary how it would change the yml ? I'd say this should be "optional" config .. not to cause any issues with existing ymls.


OpenAISwift 1.4.0 multiple errors with new Xcode 15 release

I installed 1.4.0 due to the issues I was having with the previous version specifically with ChatMessage.swift and getting the operation could not be completed error. Now Xcode 15 just got released and now nothing works. building my project, I get 9           " Cannot find type 'ChatError' in scope", "cannot find ChatMessage in scope", "cannot find ChatConservation in scope" errors.

related to #122


Awaiting for initialisation of next-i18next instance

Hi @isaachinman 👋
Referring to the next issue https://github.com/isaachinman/next-i18next/issues/374#issuecomment-529196313 specifically this comment:

Finally took some time to dig into this. The issue of awaiting the initialisation of the i18next instance is a very trivial one, and I can provide an easy, non-breaking way for users to do that if they want to. However, I think that that race condition is causing a very low percentage of the error reports we are seeing here.

Would you mind on sharing the approach of awaiting the initialisation of next-i18next instance, please ?
We're having some issues with it and we haven't found a way to await it or initialise it synchronously
Thanks 🙏

Hi @carloscuesta. So, after doing some stuff with user options and plugins, etc, next-i18next ultimately just calls i18n.init.
The .init function returns a promise, and I believe there's also some sort of callback option as well.
There is already a boolean flag on the i18n instance called isInitialized, which you could observe and await.
Or, we can simply assign the promise back onto the i18n object, so that you can await that:
next-i18next source code:
config.use.forEach(x => i18n.use(x))
const isInitializedPromise = i18n.init(config)
i18n.isInitializedPromise = isInitializedPromise

User's code:
const server = express()
await nextI18next.i18n.isInitializedPromise
server.use(nextI18NextMiddleware(nextI18next))

There's also a possible argument that the init promise should be on the i18n object by default... Could get @jamuhl's opinion on that.
Let me know if this makes sense.

@isaachinman no reason I know of...so this looks ok to me.

@jamuhl So you'd accept a PR to put that promise on the object itself?

@isaachinman
config.use.forEach(x => i18n.use(x))
const isInitializedPromise = i18n.init(config)
i18n.isInitializedPromise = isInitializedPromise

that looks like code in next-i18next - not sure what you like to put on i18next?
if it's, adding the Promise inside init to this, pseudo code:
init: (options) => {
   //...
   const deferred = defer();
   this.isInitializedPromise = deferred;
   // ...
}

would be ok...just that Promise will be overridden on each init call! or is this only about adding it to the typings?

@isaachinman
Your example looks right and is easy to understand follow and implement 👍
Awaiting the initialisation of i18n ensures that the translations are loaded ? 🤔

will be overridden on each init call

Good point. We'll keep this logic in the next-i18next source then.
@carloscuesta You can read the documentation here. Please feel free to submit a PR with this functionality. Must include at least basic test coverage. Thanks!

Any update?


Inital commit to add TensorFlow processor #84

Current TensorFlow SCDF Processor project is at https://github.com/tzolov/tensorflow-spring-cloud-stream-app-starters
I have added only the core tensorflow processor not the `twitter-sentiment processor
Resolves https://github.com/spring-cloud-stream-app-starters/app-starters-release/issues/84.

Thanks, @tzolov!
Hi, @artembilan @sobychacko @ericbottard - please review this at your convenience. The goal for this PR is to gather feedback and whenever it is ready, I'd like for it to be contributed to its own repo. Possibly spring-cloud-stream-app-starters/tensorflow - TBD.

Actually, this commit includes only the static descriptor files. The actual project will be submitted as a PR in https://github.com/spring-cloud-stream-app-starters/tensorflow.

Because the  https://github.com/spring-cloud-stream-app-starters/tensorflow is empyt Github doesn't allow me to fork it. There is no button or workaround.
Perhaps you can add an empty README to the project.

@tzolov just added empty readme, repo is now forkable.

This PR does not make sense without any original project.
Should be mark "On hold" or [DON NOT MERGE YET]
@tzolov ,
please, consider to stop PR'ing from your master.
It is pretty easy to perform git checkout -b TensorFlowProcessor to create a new branch and switch to it directly.
That way it would be easier for us to review and merge on the our local side.
And after merging you will need only git pull upstream master to your local copy.
Right now you will have to do git reset --hard upstream/master with some other possible cleaning and resolving conflicts work.

@artembilan: It was my fault - I suggested submitting a PR without a new repo - sorry for the confusion. @tzolov is going to drop the remaining part of PR in https://github.com/spring-cloud-stream-app-starters/tensorflow.

@sabbyanandan ,
thank you for taking responsibility, but that isn't an error or a fault at all, really!
I just suggest to mark the PR somehow to let other to know do not merge it by accident.
The way you suggested (this PR) we have finally that repo and it is ready for forking and pushing.
Otherwise I don't know how things would go.
E.g. I asked once about repo for the AWS Kinesis Binder, but nothing happened 😉

here is the PR: https://github.com/spring-cloud-stream-app-starters/tensorflow/pull/1

also should we discuss the review (and eventual rework) here or on the tensorflow#PR itself?

Thank you for update, @tzolov !
Well, we will discuss here and there, but already in the scope of particular changes.

I've marked this one with the [DO NOT MEGE YET].
I'm fully sure that we should merge this one until that in tensorflow

@artembilan i see and completely relate with your point.  This PR shouldn't be merged until the tensorflow processor PR get resolved.

Updated the tensorflow processor version.
LGTM
@tzolov you covered pretty much everything needed for the docs and other metadata generation. Great work!
Here is the CI plan for the Tensorflow app starter: https://jenkins.spring.io/view/Seeds/job/spring-scst-app-starters-tensorflow-master-ci/
Merging.

Great! Tanks guys!
Currently the tensorflow project contains two processors: the core tensorflow-processor and twitter-sentiment-processor, but this PR covers only the tensorflow-processor i believe?
Would the twitter-sentiment-processor be picked somehow automatically (doesn't seem so) or
should i open another PR for the twitter-sentiment-processor?
@sabbyanandan @sobychacko @artembilan WDYT?


convert managed policy to inline, add ScheduleKeyDeletion permission

Description of changes:

convert arn:aws:iam::aws:policy/AWSKeyManagementServicePowerUser into inline policy
add ScheduleKeyDeletion into inline policy

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@Julian-Chu Ignore those failures, i'm just doing some tests :)


Fix docs search and add proper secret management for Algolia

Removed plain text algolia secret setup in docusaurus config and store them instead in Vercel env variables.
The original config was pointing to wrong endpoint in Algolia, leading to using outdated index. This PR should also fix the long running docs search issue.

Okay we might need to sunset the PR Test Github action here, for various reasons:

It is not fully simulating the build environment of Vercel, here failing because it does not have access to the env variables set in Vercel
Vercel is already doing the build test with the test deployment

@sukoneck @bertmiller shall I submit a PR to sunset it?


Builder Block Quarries not working on servers. Working only in single player.

rftools-1.1x-6.01.jar
rftoolsctrl-1.1x-1.6.8.jar
rftoolsdim-1.1x-5.02.jar
Are used on the server side. Using FTB Beyond Server with FTB Beyond Client.
Forge Version 1.10.2-<IP_ADDRESS>6
FTB Beyond 1.10.2-1.10.0
rftools.cfg.txt

You need to be more specific. What is not working? RFTools Builder is used a LOT by many many peoples on servers (including me yesterday even) and it works fine. So if it is not working for you something else must be going on. I need more details. Logs, your setup and what is not working about it

RFTools Builder as Quarry not working on server. Looks like it scans 1 Y value and immediately quits

fml-server-latest.log
latest.log

Ah, are the chunks claimed? This is a problem in FTB Utilities where it doesn't allow fake players to work

Nope, area is unclaimed.

Can you show screenshots of the setup and gui?

I'll do one better, a GIF
https://i.imgur.com/nEjkvwL.gifv

I just talked to the guys on Minecolonies about the same issue, they told me the following:
"If you're on a server you'll have to tell RFtool to give the machine the players UUID as own UUID.
Because that's how forge intended it. (That's what Lex told us)"

That doesn't make any sense. A Builder should be able to work even if a player didn't place it (if placed with some automatic block placer for example). Also many people are using this on servers with no problems. The Builder is one of the most widely used quarries atm

Don't shoot the messenger XD... I'm just copy pasting.

How would I go about doing this?

Updated to latest version and still nothing is working.

Latest version of what?

As far as I know this is a bug in Minecolonies. Please report to them

After some further research, RFTools Builder ~ Quarry is not working because the server is in offline mode. The server must be in offline mode for Bungeecord to work.

Issue is fixed, looks like an incorrect NBT value.

what?.. how do i fix a incorrect NBT value?


Renaming the Follow button to "Follow back" when followed by user

Hi there! 👋 I’d like to suggest a small improvement:
Rename the “Follow” button to “Follow back” when the user is already followed by someone, similar to how Instagram handles it? I think this would make it more intuitive for users to follow each other back.
Before:

After:

Before:

After:

The button is already shown as Follow Back on a post from someone who follows you but who you don't follow as a result of the experiment in #3439, which has now been rolled out completely.
So I think this is a good suggestion as it would mean the button would appear as Follow Back on the user's profile and in other places too.

hey @gaearon @surfdude29 thanks for the feedback, makes sense to me! just commited the adjustments


Remove Cypress

Description
This issue to track the work needed for the Cypress removal.
Summary
We have added a significant amount of Playwright tests to the codebase, so I think we're in a good position to start removing Cypress.
I'd propose we proceed by going through each Cypress file and check if the test cases are written in Playwright. Add the tests in if they aren't.
Potential impediments
There is a significant discrepancy I've found and I'm afraid it might complicate the migration work. It's that Cypress has a .task() API that allows us to execute a Node command while Playwright doesn't. With this API, we can choose to work with new user data or certified user data in each test case, but we don't have this flexibility with Playwright.

Cypress:
https://github.com/freeCodeCamp/freeCodeCamp/blob/0d123e0ba99df8fc175cbec4f05c3ca995c509a9/cypress/e2e/default/top-contributor.ts#L3
Playwright:
https://github.com/freeCodeCamp/freeCodeCamp/blob/0d123e0ba99df8fc175cbec4f05c3ca995c509a9/.github/workflows/e2e-playwright.yml#L132

IMHO, the ability to write tests with new user data is important, as we need to have test coverage for flows such as completing a new step/challenge, claiming a certifications, etc.
A couple approaches off I have so far:

Keep using certified user data, but mock APIs when needed, either manually or using HAR files.
1.1 Mocking manually:
https://github.com/freeCodeCamp/freeCodeCamp/blob/0d123e0ba99df8fc175cbec4f05c3ca995c509a9/e2e/email-sign-up.spec.ts#L113-L128
1.2 Using HAR files: https://playwright.dev/docs/mock#mocking-with-har-files
Run exec() within Playwright tests when needed. Example: https://stackoverflow.com/questions/71397181/node-execute-command-asynchronously-and-read-results-on-completion
Split the Playwright CI flow into one that runs with seed and one that runs with seed:certified-user

I'm still unsure which approach I prefer:

(1) is okay, but we lose some confidence as the APIs aren't tested
(2) should work, but I have never executed a Node command directly in tests, so I'm not sure if there will be any problems coming from this
(3) is probably the ideal way to setup and execute our tests. But the changes will be quite involved as we will need to specify which test suites should be run by which CI flow

Action items
(I'm only focusing on files that are blocking my PR (#52525) for now. I'll get back to the list when I have more brain capacity. Other maintainers are welcome to help with the todo list 🙂)
### Migrate and remove Cypress test files [TBU]
- [ ] `learn/responsive-web-design/show-cert-from-superblock.ts` - prioritized to unblock component migration
- [ ] `settings/certifications.ts` - prioritized to unblock component migration
- [ ] `settings/portfolio.ts` - prioritized to unblock component migration

### Remove Cypress
- [ ] Remove Cypress commands
- [ ] Remove the entire `cypress` directory
- [ ] Remove Cypress from the dependency list

Thanks for tracking this.
1: No thank you 🙂
2: This should work, provided:
a) Tests are not run in parallel
b) The order of the tests is predictable (or, we have to seed and mock for every test)
3: Seems like the ideal solution, and, provided we can specify a directory to run for playwrite (i know we can), then it is more like having two e2e folders, and should not require much more work than answering a quick question: "Does this test require the mocked Camper or new Camper"
For 3, we might even have 3 directories:

e2e/new/
e2e/mocked/
e2e/independent

I agree that 3) is the way to go for now.
In the future we can seed multiple users and login to whichever user is appropriate for the test, but that's a little way off.


Change clang version for github build action

Reading the script, "15" should hit a supported version of clang on macOS.
This also changes the target branch names that should trigger the actions to explicitly be the default branch name.

Cool. The checks triggered on this PR, and the macOS one passed with the changes =)


UAA startup issues: keyManager must be set, could not resolve placeholder 'encryption.active_key_label'

SECURITY NOTICE: If you have found a security problem in the UAA, please do not file a public github issue. Instead, please send an email to<EMAIL_ADDRESS>Thanks for taking the time to file an issue. You'll minimize back and forth and help us help you more effectively by answering all of the following questions as specifically and completely as you can.
What version of UAA are you running?
What output do you see from curl <YOUR_UAA>/info -H'Accept: application/json'
as-is...
(503) Server Unavailable.

See \uaa\build\reports\tests\ uaa-server.log for details.
after manual source/resource changes...
{
    "app": {
        "version": "4.23.0-SNAPSHOT"
    },
    "links": {
        "uaa": "http://localhost:8080/uaa",
        "passwd": "/forgot_password",
        "login": "http://localhost:8080/uaa",
        "register": "/create_account"
    },
    "zone_name": "uaa",
    "entityID": "cloudfoundry-saml-login",
    "commit_id": "46c122d",
    "idpDefinitions": {},
    "prompts": {
        "username": [
            "text",
            "Email"
        ],
        "password": [
            "password",
            "Password"
        ]
    },
    "timestamp": "2018-10-23T17:25:19-0600"
}

How are you deploying the UAA?
I am deploying the UAA locally only using gradlew per instructions
What did you do?
PS C:\path> git clone git://github.com/cloudfoundry/uaa.git
PS C:\path> cd uaa
PS C:\path\uaa> .\gradlew run

...then from a different window...
PS C:\path> $headers = @{
>>     Accept = "application/json"
>> }
PS C:\path> curl http://localhost:8080/uaa/info -H $headers

What did you expect to see? What goal are you trying to achieve with the UAA?
(200) OK.

What did you see instead?
(503) Server Unavailable.

See \uaa\build\reports\tests\ uaa-server.log for details.
Manual Changes to Successfully Launch UAA Locally
These manual changes were inspired by other community-posted issues and solutions that are closed, but do not seem to be fully resolved (ex: Issue 694, Issue 717,  Topic 24918012)

Are these the right changes to get UAA up and running, as-is?  (obviously UAA needs to be properly configured... this is just a starting point)
The instructions don't mention that such changes are required.

Change 1.
Copy this block from \uaa\src\main\resources\required_configuration.yml to uaa\src\main\resources\uua.yml
encryption:
  active_key_label: CHANGE-THIS-KEY
  encryption_keys:
  - label: CHANGE-THIS-KEY
    passphrase: CHANGEME

Change 2.
Register keyManager as shown below in \uaa\src\main\webapp\WEB-INF\spring\saml-idp.xml:
    <bean id="idpMetadataManager"
          class="org.cloudfoundry.identity.uaa.provider.saml.idp.NonSnarlIdpMetadataManager"
          depends-on="spMetaDataProviders" destroy-method="destroy">
        <constructor-arg name="configurator" ref="spMetaDataProviders"/>
        <property name="refreshCheckInterval" value="${login.saml.metadataRefreshInterval:0}"/>
        <property name="generator" ref="zoneAwareIdpMetadataGenerator"/>
        <property name="keyManager" ref="idpZoneAwareSamlKeyManager"/>
    </bean>

While the above two changes do allow for startup...
...the better workaround (as mentioned by @BloodWorkXGaming in Issue 717) is to add an extra slash (to create a triple slash) to these lines in uaa\build.gradle instead.
if (activeProfiles.contains('debug')) {
    jvmArgs = "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 -DLOGIN_CONFIG_URL=file:///" + new File(".").absolutePath + "/uaa/src/main/resources/required_configuration.yml"
} else {
    jvmArgs = "-DLOGIN_CONFIG_URL=file:///" + new File(".").absolutePath + "/uaa/src/main/resources/required_configuration.yml"
}

Closing due to inactivity.


SourceHut will blacklist the Go module mirror

Description
SourceHut will disable git access for the Go Module Mirror on February 24th, 2023.
https://sourcehut.org/blog/2023-01-09-gomodulemirror/
GItea uses git.sr.ht/~mariusor/go-xsd-duration.
Gitea Version
Not revelant.
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
No response
Operating System
No response
How are you running Gitea?
Not revelant.
Database
None

We don't use proxy.golang.org, we use goproxy.io so this shouldn't be an issue. @jolheiser please confirm

$ go mod why git.sr.ht/~mariusor/go-xsd-duration
# git.sr.ht/~mariusor/go-xsd-duration
code.gitea.io/gitea/routers/api/v1/activitypub
github.com/go-ap/activitypub
git.sr.ht/~mariusor/go-xsd-duration

So whilst I think our CI should continue to work - the issue is likely that anyone else building from source will be negatively affected.

We could add it to the Makefile

I guess as long as we don't override any personal setting that a user has set this would probably be fine.

There is still some ongoing discussion between the Go team and Drew, so hopefully this can be resolved before action is needed yet.
As mentioned, CI should continue to work, and past that there's a chance users won't be affected if the transitive dep doesn't change, as the module proxy should still serve the last updated version regardless.

We could add it to the Makefile

I'd rather we vendor/incorporate the affected module than to mess with GOPROXY. The official proxy has certain undocumented behaviours that other proxies might not have, for example fixing the broken hugo releases.

Update 2023-01-31: Russ Cox of the Go team reached out to us to address this problem. After some discussion, an acceptable plan was worked out. [...] Consequently, we have cancelled our plans to disable Go traffic to git.sr.ht. No action is required by users to continue receiving service. Thanks Russ!

Thanks for the update! I'll close this for now, then.


Bump resolvers

Maybe we could drop cabal.project-8.8.2 and build for ghc-8.8.2 with the default one if we update the index state.

@jneira good idea. I actually tried that, but forgot to bump the index state.


Add a trivial review label

This adds a new trivial review label to the configuration, which if it is present, makes it so that the PR doesn't block on reviews. Other preconditions may still prevail (e.g. CI must pass), but the requirement on reviewers in particular is relaxed.
Fixes #34.

See it in action there: https://github.com/bnjbvr/octotest/pull/2

See it in action there: https://github.com/bnjbvr/octotest/pull/2


mbedTLS compiled with clang incorrectly rejects certificates (broken validation)

Summary
mbedTLS will reject certificates that are valid, instead returning MBEDTLS_ERR_X509_CERT_VERIFY_FAILED.
This only happens if mbedTLS is built with Clang and only since 0.27.0.
System information
Mbed TLS version: 0.27.0
Operating system and version: Linux glibc
Configuration: default
Compiler and options: clang version 12.0.1
Steps to reproduce
This can be reproduced with the ssl_server and ssl_client1 test programs, which the following script does:
rm -rf mbedtls-2.27.0
wget 'https://github.com/ARMmbed/mbedtls/archive/refs/tags/v2.27.0.tar.gz' -O - | tar -xz
cd mbedtls-2.27.0
make -j4 programs

./programs/ssl/ssl_server &>/dev/null &
sleep 1
./programs/ssl/ssl_client1
killall ssl_server

Expected behavior
When ran with CC=gcc or if the script is edited to build 0.26.0 or earlier

  . Seeding the random number generator... ok
  . Loading the CA root certificate ... ok (0 skipped)
  . Connecting to tcp/localhost/4433... ok
  . Setting up the SSL/TLS structure... ok
  . Performing the SSL/TLS handshake... ok
  . Verifying peer X.509 certificate... ok
  > Write to server: 18 bytes written

GET / HTTP/1.0

  < Read from server: 156 bytes read

HTTP/1.0 200 OK
Content-Type: text/html

mbed TLS Test Server

Successful connection using: TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256

ssl_msg.c:4090: mbedtls_ssl_handle_message_type() returned -30848 (-0x7880)
ssl_msg.c:5543: mbedtls_ssl_read_record() returned -30848 (-0x7880)

Actual behavior
When ran with CC=clang:

  . Seeding the random number generator... ok
  . Loading the CA root certificate ... ok (0 skipped)
  . Connecting to tcp/localhost/4433... ok
  . Setting up the SSL/TLS structure... ok
  . Performing the SSL/TLS handshake...ssl_tls.c:2620: x509_verify_cert() returned -9984 (-0x2700)
ssl_msg.c:4714: mbedtls_ssl_fetch_input() returned -29312 (-0x7280)
ssl_msg.c:4065: ssl_get_next_record() returned -29312 (-0x7280)
ssl_cli.c:3101: mbedtls_ssl_read_record() returned -29312 (-0x7280)
 failed
  ! mbedtls_ssl_handshake returned -0x7280

Last error was: -29312 - SSL - The connection indicated an EOF

We've had a previous report of a bug with Clang 12: https://github.com/ARMmbed/mbedtls/issues/4786 . We think that it's due to a Clang bug but we haven't completed the investigation so it's still possible that the problem is undefined behavior in Mbed TLS. Do the unit tests pass for you? Can you try with an older Clang version or with reduced optimization?

The test suite shows a failure for test_suite_dhm, at test_suite_mpi it hangs indefinitely.
Other options tested using ssl_server + ssl_client1 like before:

Compiler
-O1
-O2 (default)

clang 12.0.1
:heavy_check_mark:
:x:

clang 11.0.0
:heavy_check_mark:
:heavy_check_mark:

I'm inclined to say this is indeed a duplicate of the bug you linked.

I have reproduced these results on the latest development_2.x and with clang-13. As with #4786, annotating mpi_mul_hlp with noinline fixes the issue, so it seems likely to be the same.

Closing as a duplicate of https://github.com/ARMmbed/mbedtls/issues/4786


Detect and show an error when two notes have the same metadata

Different notes with the same YAML metadata for this plugin should cancel the whole operation. Users need to know when this happens so that they can fix the inconsistencies. Usually this will happen after a note was copy/pasted

Done for title and slug


InternalServerError from APNs

We are using Pushy in our production environment and it works great! Besides we have a staging environment where tests are performed prior to launching new versions. Staging environment is not always used and most of the time it is in idle state.
Today when we started to use staging, we got InternalServerError errors (stack trace given below). I guess it is related to long idle state of pushy, not sure though. We didn't have any choice but to restart the app.
In which cases APNs may return such an error? Should pushy be closing and reconnecting in such cases? Or is it a duty of our side to reconnect upon such an error?
Using pushy v0.9, java 1.8.0_102 (Oracle)
2017-01-06 05:23:27,715  WARN [nioEventLoopGroup-2-4] c.r.p.a.ApnsClient APNs server reported an internal error when sending ... .
2017-01-06 05:23:27,715 ERROR [nioEventLoopGroup-2-4] o.g.g.a.ApnsResponseListener Failed to send notification id=117_963041067
java.util.concurrent.ExecutionException: com.relayrides.pushy.apns.ApnsServerException: {"reason":"InternalServerError"}
    at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54)
    at org.grouvi.gpns.apns.ApnsResponseListener.operationComplete(ApnsResponseListener.java:59)
    at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:507)
    at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:486)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:427)
    at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:129)
    at com.relayrides.pushy.apns.ApnsClient.handleServerError(ApnsClient.java:965)
    at com.relayrides.pushy.apns.ApnsClientHandler$ApnsClientHandlerFrameAdapter.onDataRead(ApnsClientHandler.java:173)
    at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onDataRead(DefaultHttp2ConnectionDecoder.java:229)
    at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readDataFrame(DefaultHttp2FrameReader.java:411)
    at io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:245)
    at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:155)
    at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:118)
    at io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:335)
    at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:395)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
    at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
    at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1069)
    at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:902)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.relayrides.pushy.apns.ApnsServerException: {"reason":"InternalServerError"}
    ... 37 common frames omitted

I encountered the exact same problem (against Apple's sandbox environment). I had a retry mechanism causing it to retry sending the push. It continued to fail for 48 hours before I restarted the server software. After that, there were no problems anymore. Notably, the same internal error failure was encountered for both different topics used (for two different apps). Also using Pushy 0.9, but Oracle Java 1.8.0_60.

Huh… I still think this is an upstream problem, but @markusrydh, can you be more specific about what you did to make this happen? Can you reproduce it?

The (development) server was more or less idle (no push for a day), then the server sent a VoIP-push to the client. Immediately it got a InternalServerError and all pushes continued to get that until restarted.
I googled a bit and it seems that other libraries using the HTTP/2 token-based approach is getting the same problem. It seems that a connection close/reopen seems to do the trick. Although it might be a problem on the Apple side, it would be nice if there was a workaround. I am currently thinking about having to setting up a new connection if I get repeated InternalServerErrors, but of course it would be nice if the library did this behind the scenes. Or maybe a "force close" option, to make the connection immediately close and then re-open the connection (as it would for a normal broken connection). It would be a more lightweight option than disconnect().
Just a reference to some of the issues I found in node-apn that seems to be related:
https://github.com/node-apn/node-apn/issues/463
https://github.com/node-apn/node-apn/issues/461
https://github.com/node-apn/node-apn/issues/449

And no, I have not been able to reproduce yet.

Thanks for the links; I've shared some information there that I hope will be helpful to the node-apn community. For our part, I still think the best bet is to file a bug report with Apple. It sounds like emerging consensus around reproduction steps is:

Open a connection and send a push notification with an authentication token.
Keep the connection open, but mostly idle for more than an hour (the expiration period of an authentication token). Pushy will do this automatically by sending PING frames.
Send another notification with the initial authentication token.
The server will respond with an HTTP/500.

Seem right? If so, I'll file a bug report with Apple with the information we have so far.

I've reported this to Apple as rdar://30168506.

@jchambers Point 2 is questionable. I suspected before that idle connections caused issues but we got the issue in our prod environment where we usually do not have that long idle times. Anyways, your steps seem reasonable to me.

Acknowledged; will update the ticket upstream.

Another observation:
To get rid of InternalServerError exceptions we are restarting the app and when stopping we see plenty of error logs with ClientNotConnectedException (stack trace below). I am pretty sure that those exceptions are for old push notifications, not for those that were queued when jvm was terminating. According to logs, notifications were not accepted and neither rejected by APNS. So it seems listener threads hung up waiting for the future result.
I have suspicions that this may be related to the issue. Not sure though.
I have to mention that we use custom event loop group with 4 threads and all error logs were from the same thread, in stack trace below - nioEventLoopGroup-2-2
2017-01-30 08:31:24,300 ERROR [nioEventLoopGroup-2-2] o.g.g.a.ApnsResponseListener Failed to send notification id=67087_1985304421
 java.util.concurrent.ExecutionException: com.relayrides.pushy.apns.ClientNotConnectedException: Client disconnected unexpectedly.
        at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54)
        at org.grouvi.gpns.apns.ApnsResponseListener.operationComplete(ApnsResponseListener.java:59)
        at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:507)
        at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:486)
        at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:427)
        at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:129)
        at com.relayrides.pushy.apns.ApnsClient$2$2.run(ApnsClient.java:495)
        at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
        at io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:73)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.relayrides.pushy.apns.ClientNotConnectedException: Client disconnected unexpectedly.
        ... 9 common frames omitted

We recently upgraded our servers from Pushy 0.7.2 to 0.9.2 (and API tokens - hurrah!) and we just ran into this situation on our development environment over the weekend.  I noticed in the node thread that someone put in a hack to reset every hour.  Do folks have a workaround for the issue while we wait for Apple to fix their end?

@bazi My apologies—I completely missed your comment from the end of January. Thanks for the additional data! I'm not sure what to make of it, but will look into it.
@ksquaredkey Not sure. I saw your post on the mailing list, and I think that's the right place to ask. I'll see what I can learn through other channels.

@bazi I think what you saw ("notifications were not accepted and neither rejected by APNS") may be related to or the same as #435; does that seem plausible to you?

Yes, I do agree that this issue and #435 are related.
But I have to note that I can do ApnsClient.disconnect() and reconnect again as opposed to mentioned issue where reporter says he can't disconnect and reconnect.
We have implemented regular reconnection mechanism which is triggered every 30 minutes or on InternalServerError, whichever comes first, with some thresholding. And this resolves many of our problems.
I had concluded to myself that it is an upstream issue, which you had already stated in your replies. Btw, I really appreciate your efforts in solving such issues. Thanks for great pushy!!!

We've been seeing this a lot recently as well. Any chance the reconnecting mechanism could be included in Pushy itself?

Any chance the reconnecting mechanism could be included in Pushy itself?

As mentioned earlier, I'd really rather not if at all possible. I acknowledge that this is a real problem for real users, but I still think the right thing to do—for now at least—is to do exactly what @bazi is doing:

We have implemented regular reconnection mechanism which is triggered every 30 minutes or on InternalServerError, whichever comes first, with some thresholding.

I'm still trying as hard as I can to push for an upstream fix, which seems like The Right Thing To Do. If I get a clear sign that no fix is coming, we'll talk about adding a workaround directly to Pushy.

By "upstream fix" I imagine you mean Apple?

If I get a clear sign that no fix is coming, we'll talk about adding a workaround directly to Pushy.

Makes sense. I'm not holding my breath for Apple doing anything about it though :/

By "upstream fix" I imagine you mean Apple?

Correct. I do have reason to believe they're looking into the issue now (and maybe weren't before), so this isn't a "wait forever" situation.

That's great news :)

Funny, I'm back to this issue again, myself.  We just tried implementing the disconnect/connect and that does not help the issue.  The ApnsClient continues to send the expired token on this new connection and Apple continues to send us the InternalServerFailure response.
The workarounds folks suggested to me on the mailing list (last month) of destroying and creating new ApnsClient instances does not work for us.  Our server is Scala based and all of our infrastructure is immutable.  It would be a major rewrite to support nuking the ApnsClients itself.
Digging through the source, I figured out the crux of the issue (Apple's bug notwithstanding) is the invalidation of the authentication token "token) in the AuthenticationTokenSupplier not being invalidated.  The only way it gets invalidate (as of 0.9.3) is on the receipt and processing of an "ExpiredProviderToken" response.  Changing connections will not help.  Near as I can tell, Jon's recommendation in the Radar of a GOAWAY will not help.  The issue is there is only one way to clear that token - the ExpiredProviderToken.  (well, ok, two ways, nuke the object containing it)
So, it seems to me that we just need a mechanism to invalidate the token prior to trying an > 1 hour idle time send and let a new token be created. I did not want to get into all sorts of timeout management, etc inside pushy, and others might not care for the added hopes so I propose:
Adding a public method to ApnsClient .invalidateAuthToken() that will call .invalidateToken() on the AuthenticationTokenSupplier instance.
Calling this will then case a new Auth Token to be created on the next Push Request send().  This would be just a minimal change to Pushy but still allow my code to be in control of working around Apple's bug.
I will probably do a PoC of this as a Fork of Pushy but was hoping for some feedback on whether folks thought this was A Really Bad Idea.  I'm stuck between Apple's bug and a ship date - I cannot wait for Apple to fix their end.

This is a thoughtful and helpful question, and I feel bad for the brevity of the reply, but here it goes: I think this is fixed in #451.

On reflection, that was way too brief. In #451, we moved token-handling into the ApnsClientHandler and out of the ApnsClient. That means that reconnecting will invalidate/re-create the authentication token.

Thanks Jon!  I was looking on the 0.9.3 branch yesterday.  Looking at master, I see what you are talking about - looks good.  Given the level of changes that represents in master, am I correct in assuming that that code is still a while away from coming out in a new release?  Just checking as I think I'm going to still have to make my local mod for now and throw a ticket on our board to remove it when the new version of Pushy comes out.

I was actually planning on doing a release tomorrow-ish, but it may take a few days to move through the Sonatype tubes to Maven Central since we're moving to a new group ID (com.relayrides to com.turo).

oh very nice!!!  You made my day (which happens to be my birthday - thanks for the present ;-)

Folks, with apologies for the long delay, I've just received a reply from Apple requesting more information. They write:

Are you still seeing the issue mentioned in the description? If you do, would you please provide some more details with a reproduction?

Bundle ID.
Time frame + time zone of the reproduction.
The IP address of your service will be helpful.
Detail information about pushes (token, message ID, timestamp) around the reproduction time frame is also helpful.

I'd happily accept an answer to the first question ("are you still seeing this issue?") here. If you still experiencing the issue and if you're comfortable sharing bundle IDs, timestamps, IPs, and push notification details, please email me directly at<EMAIL_ADDRESS>Thanks!

Well, it has been half a year that I opened this issue. Since then we had applied our custom fixes (like periodic reconnects), and there were couple of pushy releases as well.
In short, we are not facing this issue anymore.
I want to note that, as I see it, there were other issues mentioned here. Please provide your current status regarding your issues... @markusrydh @ksquaredkey @NachoSoto @eager

I had this problem, and found this thread. @bazi could you please share any graceful way to do periodic reconnects from pushy client?

@StarWindMoonCloud This is actually very timely. Did you experience this problem recently? With Pushy? If so, could you please email me the information requested by Apple (please see https://github.com/relayrides/pushy/issues/408#issuecomment-318142876).
Thank you!

Sent @jchambers

Thanks, @StarWindMoonCloud. I've passed the information along to Apple. We'll see what happens next.

@jchambers Any update from Apple? It happened again at my side today.
I'm worried Apple might be slow, since the thread started over half an year ago...

I'm worried Apple might be slow, since the thread started over half an year ago...

…and there's the understatement of the month ;)
No reply yet, but it's still the weekend. In my experience, they usually stay focused on a problem once they've acknowledged it. Let's give them a few days, but after that, it's probably reasonable to use one of the workarounds mentioned in this thread (like closing/reopening a connection when you get an InternalServerError).

To follow up, I have received no additional information from Apple about this issue since last month.

I had this problem, and found this thread. @bazi could you please share any graceful way to...

@StarWindMoonCloud sorry for late response. Here are steps to do a re-connection:

first of all gracefully disconnect clients by ApnsClient.disconnect().await()
build a brand new client and do connect using that client: new ApnsClientBuilder().setSigningKey( signingKey ).build()

Make sure that during re-connection phase you pause all your notification sending operations. Release all your references to disconnected client(s) and use newly created client(s).

Now that we've moved to a pooled connection model, callers don't have direct control over connections and we may need to take additional steps to address the issue within Pushy itself. Does anybody know if this is still A Thing in practice, though?

I do still get InternalServerErrors from Apple using an admittedly old version of Pushy (0.9.2). Attaching a log. It is not an uncommon error, and I am using a manual disconnect/connecting with new connection to get it to work. Don't know if newer version of pushy would change this in any way? Will update to newer version, but maybe I can't use the latest version if I cannot restart connection?
Pushy_internalservererror.log

Don't know if newer version of pushy would change this in any way?

I'd say it's very unlikely. The problem has been reported across a wide variety of APNs libraries on a variety of platforms (a popular node.js APNs library had the same problem, for example), and so it appears to be beyond our control.

Will update to newer version, but maybe I can't use the latest version if I cannot restart connection?

That's an oversight on my part, and we should have a new release in a couple days that works around the problem. I'd recommend holding off until then.

This should be fixed in #529, which will ship soon.

Fanstastic, will simplify our code quite a bit! Thanks Jon!


Support Mathjax for tex rendering

Motivation
@zmx0142857 came up with an idea, rendering tex with Mathjax, which is a lot faster than original LaTeX rendering. He authored manimlib/mathjax directory. The svg files generated by different compilers are often slightly different in shape, but the one generated by Mathjax has a more complicated structure, so @TonyCrane improved SVGMobject class for better svg parsing. I refactored MTex class to support Mathjax rendering.
Usage
To use Mathjax, user may have nodejs installed, and run npm install at the directory manimlib/mathjax.
Proposed changes

A manimlib/mathjax/index.js, tex2svg.js, package.json: support Mathjax renderer
M manimlib/utils/tex_file_writing.py: implement tex_to_svg_using_mathjax() function, which uses node command to generate svg files of tex
M .gitignore: filter out nodejs-related files
M manimlib/mobject/svg/mtex_mobject.py: add use_mathjax config option (default False) to allow users select tex renderer; add generate_plain_tex_file config option (default False) to allow users generate another svg without additional \color commands to prevent the shape of svg from being altered by them

Test
Code:
from manimlib import *

class MathjaxTest(Scene):
    def construct(self):
        tex_string = """
\\mathbf{V}(x_0, x_1, \\cdots, x_{n-1})^{\\mathsf T} \\times \\mathbf{v} =
\\begin{bmatrix}
    1 &1 &1 &\\cdots & 1\\\\
    x_0 &x_1 &x_2 &\\cdots & x_{n-1}\\\\
    \\vdots &\\vdots &\\vdots &\\ddots & \\vdots\\\\
    x_0^{n-1} &x_1^{n-1} &x_2^{n-1} &\\cdots & x_{n-1}^{n-1}\\\\
\\end{bmatrix} \\times \\mathbf{v}
        """

        mtex_mathjax_mob = MTex(tex_string, use_mathjax=True).set_fill(MAROON)
        mtex_mathjax_mob.shift(2 * UP)
        self.add(mtex_mathjax_mob)

        mtex_mob = MTex(tex_string).set_fill(TEAL)
        mtex_mob.shift(2 * DOWN)
        self.add(mtex_mob)

Result:

Some additional commits

M manimlib/utils/simple_functions.py: refactor choose with lru_cache and scipy's func (borrowed from community edition); delete clip_in_place func (unused)
M manimlib/utils/color.py: delete get_shaded_rgb func (unused)
D manimlib/utils/string.py: delte unused functions