RocketMQ 4.9.2 unsupport subscribe 2 topic with 1 consumer group in 2 process

I subscribe 2 topic with 1 consumer group in 2 process
when a process upload the SubscrptionData to broker, the broker will remove SubscriptionData uploaded in other process
code in ConsumerGroupInfo#updateSubscription
subscribe 2 topic with 1 consumer group in 1 process is ok
is this a bug?

Are you subscribing to different topics in the two processes?

Inconsistent subscription relationships can cause this problem

Are you subscribing to different topics in the two processes?

yes, 2 diff topics

A lot of same problem: Inconsistent subscription relationships. Look at this:  https://rocketmq.apache.org/zh/docs/4.x/bestPractice/21subscribe/


enable use of build script before pod install

Describe the solution you'd like
appcenter build automatically installs pods from a Podfile in the source tree.  Our Podfile references pods by relative path from Capacitor:
def capacitor_pods
  pod 'Capacitor', :path => '../../node_modules/@capacitor/ios'
  pod 'CapacitorCordova', :path => '../../node_modules/@capacitor/ios'
end

These pods are not in the source tree; npm install must be called first for those dependencies to be installed.
We have tried using appcenter-post-clone.sh as a solution to run npm install however it looks like pod install is automatically called by appcenter prior to that build script.
Unless I'm missing something, this makes it impossible to use appcenter for builds using Capacitor without adding node_modules to the Git repository (we don't want to do that.)
Describe alternatives you've considered
We could add node_modules to Git, but don't want to have 3rd party dependencies in our Git repository.
Additional context
Build output:
Hosted Agent successfully assigned.
Configuring connection settings for Hosted Agent
Starting Hosted Agent
##[section]Starting: Job
##[section]Starting: Initialize job
Current agent version: '2.148.2'
Prepare build directory.
Set build variables.
Download all required tasks.
Downloading task: ShellScript
Downloading task: CmdLine
Downloading task: CocoaPods
Downloading task: DeleteFiles
Downloading task: Xcode
Downloading task: CopyFiles
Downloading task: PublishBuildArtifacts
Start tracking orphan processes.
##[section]Finishing: Initialize job
##[section]Starting: Checkout
==============================================================================
Task         : Get sources
Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
Version      : 1.0.0
Author       : Microsoft
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
==============================================================================
Syncing repository: prod-client-app (ExternalGit)
##[command]git version
git version 2.21.0
##[command]git lfs version
git-lfs/2.7.1 (GitHub; darwin amd64; go 1.12)
##[command]git init "/Users/vsts/agent/2.148.2/work/1/s"
Initialized empty Git repository in /Users/vsts/agent/2.148.2/work/1/s/.git/
##[command]git remote add origin https://github.com/....git
##[command]git config gc.auto 0
##[command]git config --get-all http.https://github.com/....git.extraheader
##[command]git config --get-all http.proxy
##[command]git remote set-url origin<EMAIL_ADDRESS>##[command]git remote set-url --push origin<EMAIL_ADDRESS>##[command]git lfs install --local
Updated git hooks.
Git LFS initialized.
##[command]git config remote.origin.lfsurl<EMAIL_ADDRESS>##[command]git config remote.origin.lfspushurl<EMAIL_ADDRESS>##[command]git fetch --tags --prune --progress --no-recurse-submodules origin
...
remote: Total 127 (delta 28), reused 108 (delta 15), pack-reused 0
Receiving objects: 100% (127/127), 372.84 KiB | 1.10 MiB/s, done.
Resolving deltas: 100% (28/28), done.
From https://github.com/.../...
 * [new branch]      appcenter2 -> origin/appcenter2
 * [new branch]      master     -> origin/master
##[command]git lfs fetch origin 50d0318bcbb6d5058290c6e564d9a52b2818fa6a
fetch: Fetching reference 50d0318bcbb6d5058290c6e564d9a52b2818fa6a
##[command]git checkout --progress --force 50d0318bcbb6d5058290c6e564d9a52b2818fa6a
HEAD is now at 50d0318 PROD-52: add appcenter post-clone script
##[command]git submodule sync --recursive
##[command]git submodule update --init --force --recursive
##[command]git remote set-url origin https://github.com/<obscured>.git
##[command]git remote set-url --push origin https://github.com/<obscured>.git
##[command]git config --unset-all remote.origin.lfsurl
##[command]git config --unset-all remote.origin.lfspushurl
##[section]Finishing: Checkout
##[section]Starting: Tag build
==============================================================================
Task         : Shell Script
Description  : Run a shell script using bash
Version      : 2.1.3
Author       : Microsoft Corporation
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613738)
==============================================================================
[command]/bin/bash /Users/vsts/agent/2.148.2/scripts/emit-tags.sh continuous,xcode
##[section]Async Command Start: Add Build Tag
Build '3' has following tags now: normal, continuous
##[section]Async Command End: Add Build Tag
##[section]Async Command Start: Add Build Tag
Build '3' has following tags now: normal, continuous, xcode
##[section]Async Command End: Add Build Tag
##[section]Finishing: Tag build
##[section]Starting: Set Xcode path
==============================================================================
Task         : Command Line
Description  : Run a command line with arguments
Version      : 1.1.3
Author       : Microsoft Corporation
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613735)
==============================================================================
[command]/bin/bash -c sudo xcode-select --switch /Applications/Xcode_10.1.app/Contents/Developer
##[section]Finishing: Set Xcode path
##[section]Starting: Pod install
==============================================================================
Task         : CocoaPods
Description  : CocoaPods is a dependency manager for Swift and Objective-C Cocoa projects. This task runs 'pod install'.
Version      : 0.142.2
Author       : Microsoft Corporation
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613745)
==============================================================================
[command]/usr/local/lib/ruby/gems/2.6.0/bin/pod install --repo-update
Updating local specs repositories
  $ /usr/local/bin/git -C /Users/vsts/.cocoapods/repos/master fetch origin --progress
...
  Resolving deltas: 100% (16427/16427), completed with 2893 local objects.
  From https://github.com/CocoaPods/Specs
     42b70470f77..ee357c424c6  master     -> origin/master
  $ /usr/local/bin/git -C /Users/vsts/.cocoapods/repos/master rev-parse --abbrev-ref HEAD
  master
  $ /usr/local/bin/git -C /Users/vsts/.cocoapods/repos/master reset --hard origin/master
  Checking out files: 100% (2599/2599), done.
  HEAD is now at ee357c424c6 [Add] TJNetwork 0.0.1

CocoaPods 1.7.0.beta.2 is available.
To update use: `gem install cocoapods --pre`
[!] This is a test version we'd love you to try.

For more information, see https://blog.cocoapods.org and the CHANGELOG for this version at https://github.com/CocoaPods/CocoaPods/releases/tag/1.7.0.beta.2

[!] The version of CocoaPods used to generate the lockfile (1.6.1) is higher than the version of the current executable (1.5.3). Incompatibility issues may arise.
Analyzing dependencies
Fetching podspec for `Capacitor` from `../../node_modules/@capacitor/ios`
[!] No podspec found for `Capacitor` in `../../node_modules/@capacitor/ios`
##[error]/usr/local/lib/ruby/gems/2.6.0/bin/pod failed with return code: 1
##[error]The 'pod' command failed with error: /usr/local/lib/ruby/gems/2.6.0/bin/pod failed with return code: 1
##[section]Finishing: Pod install
##[section]Starting: Xcode build (not signed)
==============================================================================
Task         : Xcode
Description  : Build, test, or archive an Xcode workspace on macOS. Optionally package an app.
Version      : 4.130.0
Author       : Microsoft Corporation
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613730)
==============================================================================
##[section]Finishing: Xcode build (not signed)
##[section]Starting: Checkout
==============================================================================
Task         : Get sources
Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
Version      : 1.0.0
Author       : Microsoft
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
==============================================================================
Cleaning any cached credential from repository: prod-client-app (ExternalGit)
##[section]Finishing: Checkout
##[section]Starting: Finalize Job
Start cleaning up orphan processes.
##[section]Finishing: Finalize Job
##[section]Finishing: Job

Having a similar issue. We are using react-native-unimodules from expo, which references node_modules in the Podfile.
Like @greensopinion I have tried using the post-clone build script, which says it is run immediately after clone and before doing anything else, but this is not the case.

@greensopinion @Maximell Are your apps React Native? If so, that is a known issue. At one point, we attempted to try to move the script earlier in the process, but that ended up breaking many existing apps, who built the scripts with assumption of when it is currently executed. So we reverted that effort. I'd still like to see how we can resolve this, but haven't been able to prioritize it.

Hi @DennisPan, yes, my app is in React-Native.
I was actually able to resolve this issue, with some help from Jorge in AppCenter support (thanks!). Despite having builds queue automatically from git push, I had to go into the build configuration and push Save and Build for it to recognize my appcenter-post-clone.sh file (placed next to my .xcodeproj file).
Still working through other build issues, but npm install is happening before pod install now! 🎉

I have the same error:
##[error]The process '/usr/local/lib/ruby/gems/2.6.0/bin/pod' failed with exit code 1
##[error]The 'pod' command failed with error: The process '/usr/local/lib/ruby/gems/2.6.0/bin/pod' failed with exit code 1
@maximell could you share your appcenter-post-clone.sh ?

@manolobattista93 sure! It is really long and complicated... ;)
#!/usr/bin/env bash

# Install node_modules
npm install

Hope that helps!

@greensopinion @Maximell Are your apps React Native? If so, that is a known issue. At one point, we attempted to try to move the script earlier in the process, but that ended up breaking many existing apps, who built the scripts with assumption of when it is currently executed. So we reverted that effort. I'd still like to see how we can resolve this, but haven't been able to prioritize it.

No, it's not React Native.  We're using JavaScript/Vue in a WebView, facilitated by the Capacitor Framework

@Maximell thanks :) I solved with another solution, adding on appcenter-post-clone.sh:
# Force the 1.5.3 version of cocoapods
echo "uninstalling all cocoapods versions"
sudo gem uninstall cocoapods --all
echo "installing cocoapods version 1.5.3"
sudo gem install cocoapods -v 1.5.3

This remove the cocoapods version installed on my App Center and force the specified version, 1.5.3 in my case.

Combining the contributions in this thread has helped me achieve a flawless build process in AppCenter for my React Native 0.60 project. My ios/appcenter-post-clone.sh now looks like this:
#!/usr/bin/env bash

# Force the 1.7.5 version of cocoapods, as opposed to the version installed by AppCenter
echo "Uninstalling all cocoapods versions"
sudo gem uninstall cocoapods --all
echo "Installing cocoapods version 1.7.5"
sudo gem install cocoapods -v 1.7.5

# Upgrade Node to a version expected by React Native 0.60
set -ex
brew uninstall node@6
NODE_VERSION="8.10.0"
curl "https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}.pkg" > "$HOME/Downloads/node-installer.pkg"
sudo installer -store -pkg "$HOME/Downloads/node-installer.pkg" -target "/"

# Run Yarn
yarn

Please mind the version numbers so that they correspond to your Podfile and package.json. And after adding this script, go to your Build settings in AppCenter, confirm that your post clone script is recognized, then hit Save & Build.
Thanks gents.

Glad to hear it @jorisw! Feel free to make a PR with the script that worked for you in our collection of community driven build scripts.

Hi @DennisPan, yes, my app is in React-Native.
I was actually able to resolve this issue, with some help from Jorge in AppCenter support (thanks!). Despite having builds queue automatically from git push, I had to go into the build configuration and push Save and Build for it to recognize my appcenter-post-clone.sh file (placed next to my .xcodeproj file).
Still working through other build issues, but npm install is happening before pod install now! 🎉

I had similar issue for flutter app. I was pulling my hair out trying to figure out why my post-build wasn't getting run before pod install. Need "flutter pub get" to be run in order to generate the file needed by pod install. @Maximell's suggestion to press "Save and build" worked! Now it's finally recognizing and using it.

Having a similar issue in Flutter. The build server says you need to run pod repo update in order to satisfy latest dependencies. I am still figuring out how to tell the build script to run pod repo update. It does pod install automatically on 'flutter pub get' but I need to run pod repo update.
Please let me know if anyone figures this out.


pandas_ta.ema(df.close) returns different to df.ta.ema() when EMA length is greater than DataFrame length

Which version are you running? The lastest version is on Github. Pip is for major releases.
import pandas_ta as ta
print(ta.version)

0.3.14b0
Do you have TA Lib also installed in your environment?
$ pip list

No.  Also I get no difference in result when calling with talib=True or talib=False
Have you tried the development version? Did it resolve the issue?
$ pip install -U git+https://github.com/twopirllc/pandas-ta.git@development

Unsure.  I get install errors when trying to install the dev version.  Last line:
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

Describe the bug
When using the ema() function, you can call it either through:
ta.ema(df)
or
df.ta.ema()
The bug occurs when the length passed to ema() exceeds the length of the DataFrame.  In the first way of calling, None is returned, but in the second way, it returns the original DataFrame, unaltered.
To Reproduce
import pandas as pd
import pandas_ta as ta

df = pd.DataFrame(range(100), columns = ['close'])

# correct behaviour
ta.ema(df, 101) # returns None

# incorrect behaviour
df.ta.ema(101) # returns df

Expected behavior
a None is workable.  Arguably it should be a Series() of NaN, but None works well enough

Hello @bigtonylewis
Incremental rows with indicator lengths greater than ohlcv rows has been fixed and available on development version. Here is the relevant test:
@pytest.mark.parametrize("talib", [False, True])
def test_study_all_incremental_rows_talib(df, all_study, talib):
    MAX_ROWS = 90
    df = df.iloc[:MAX_ROWS]   # Trim for this test

    for i in range(0, MAX_ROWS):
        _df = df.iloc[:i]
        _df.ta.study(all_study, cores=0, talib=talib)
        # Break when max columns reached
        if _df.shape[1] - df.shape[1] == ALL_COLUMNS:
            assert _df.shape[1] > df.shape[1]
            break

I get install errors when trying to install the dev version.

The message is not an error from this library and a web search revealed a potential solution for:
pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

Kind Regards
KJ

Hello @bigtonylewis,
I assume by no response that the solution provided was sufficient. Thus I will be closing this issue in a few days.
Kind Regards,
KJ

Hi KJ,
I'm still getting the same issue:
$ ipython
Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.24.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pandas as pd
   ...: import pandas_ta as ta
   ...: print(ta.version)
0.4.12b0

In [2]: df = pd.DataFrame(range(100), columns = ['close'])

In [3]: assert ta.ema(df, 101) is None

In [4]: assert df.ta.ema(101).equals(df) == False, ".ta.ema() returned the original dataframe when EMA length is > D
   ...: F length"
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[4], line 1
----> 1 assert df.ta.ema(101).equals(df) == False, ".ta.ema() returned the original dataframe when EMA length is > DF length"

AssertionError: .ta.ema() returned the original dataframe when EMA length is > DF length

@bigtonylewis
I do not consider this a bug. Actially a feature as you have described in your #795 post.

I have this workaround in my code, but think it should be in the library, as I'm sure it would benefit others.

😎

I disagree.

Calling the function two different ways on the same data yields two different results: df.ta.ema(101) != ta.ema(df, 101)
The fact that it returns the original dataframe could never be correct behaviour
It runs the risk of passing any sanity checks (e.g. an empty Series) and being considered legitimate / correct.

It seems very obvious to me.  What am I not understanding?

Oh, maybe you're relating this too much with #795.  That one is all about returning a NaN vs a Series of NaN, which is for convenience; both are semantically correct, but one is easier to handle for my use case.
This is about df.ta.ema(length) returning a clearly wrong result when length > len(df)

@bigtonylewis

Oh, maybe you're relating this too much with https://github.com/twopirllc/pandas-ta/issues/795.

Probably. 😅  To correct my misunderstandings, do you have resources and examples that I can learn from?
Let's fix it! ✅  Fork, checkout the development branch, edit (including tests) and make a PR. 😎


Google OSS-Fuzz

https://github.com/google/oss-fuzz
https://github.com/google/oss-fuzz/tree/master/projects/inchi
https://bugs.chromium.org/p/oss-fuzz/issues/list?q=inchi

Update requested and made to connect oss-fuzz to the github InChI repository
https://github.com/google/oss-fuzz/pull/11598/files
Note that the chromium bugs list just shows the public bugs (so >90 days). Complete list (with new items from 1.07b) are being worked on


Fhv/e2e authenticate instructions

Introduction and Explanation
An attempt to remedy the authentication confusions when reading the Docs.

@denis-cord I have fixed the last bits that you suggested.


Provide way to edit the original Python code (file) associated with an Interactive Window cell

This is a reopen for https://github.com/microsoft/vscode-jupyter/issues/13130
Now that we are integrated for the Jupyter Data Viewer, we also need to support the Data Viewer in the interactive window. This means that we need a way to export code back to the original Python file. The current behaviour is that we can only export the code to a new notebook, which will not be executable.
Thanks!
CC @DonJayamanne

I agree, the notebook format is so unfriendly to source control that we use the interactive window extensively so we can use the jupytext format. Another option here from my perspective would be to render jupytext python files in the notebook editor as opposed to pairing it to an interactive window. (Assuming this would make the wrangler just work)
This (or something similar) is an open ask here: https://github.com/microsoft/vscode-jupyter/issues/1240

@yinghawl @jabbera
You can use the DW functionality with Interactive window today.
This issue is about modifying the Python file from within the DW extension. Is that something you need?
I'd like to keep the comments in this issue related to the  topic, else discussing things like jupytext, etc tend to pollute this issue as they are not really related to this.

@DonJayamanne Sorry, I don't want to derail the conversation, but It's never worked for me so I thought we were talking about data wrangler working period.

Yup, I am encountering the same issue as @jabbera

@jabbera @yinghawl Please file a separate issue in the DW extension
This issue is not for discussion of enabling that feature
To my knowledge this issue does not block enabling that (this is more of an API issue).

Given that this issue has been open for a while and there hasn't been any follow up since then i'm closing this issue.
@pwang347 please feel free to create this issue again when you do plan on starting work on this, else lets close this as there's nothing blocking here for now.


Metro-Police

The game can be downloaded from old-games.ru. I recommend patching it to 1.6 version from patches-scrolls de because without it the game crashes.
On modern Windows the game needs FaultTolerantHeap to run well.
If you have a DRM related issue, create a folder named "LASERLOK" and put an empty file "LASERLOK.IN" and the game should run.
The color issue is from the use of palleted textures. It is easily solvable by enabling it in the .ini, but the screenshots were taken before I set it, so please ignore that part.

The game sets transparency on the ladders as expected:

However, with palleted texture support in DDrawCompat and with dgVoodoo2:

On an emulated Voodoo2:

The loading screen text should "scroll", however it just comes up as a text. On dgVoodoo2 it does fine:

The game water is darker than normal:

For testing: you can use the savefiles from here.
The "texture2" save should get you to the water bug in point 3, and the "423432423" save should get you to the transparency bug in point 1.
Thank you

The game still consistently crashes while loading a saved game or starting a new game, at about a quarter of the progress bar. That's with FaultTolerantHeap. Without it, it crashes before the progress bar even starts.

I randomly decided to hook the game to DxWnd, with DirectX7 version hook and Handle Exceptions, and the game runs in mission. Works without FaultTolerantHeap too.

I figured it out based on the discussion on DxWnd forums. It was an issue with the wrong directory names in the RIP. I just replaced the whole Scene directory with the one from the CD install (which I couldn't get working with the empty laserlok.in file), and now it works.

The game sets transparency on the ladders as expected:

It sets the wrong color key when palettized textures are used. Those black pixels have a palette index of 0x7F but the game sets the color key to 0xFF. Both map to the same black color in the palette, so I suppose it only looks "correct" on some buggy drivers that match the color key with the looked-up color rather than the index. It also looks wrong with the Direct3D software RGB renderer (you can try it on PCem with a non-3D capable GPU), so I don't want to touch it. It should be patched in the game.

The loading screen text should "scroll", however it just comes up as a text. On dgVoodoo2 it does fine:

It's a matter of performance. The game draws each letter one by one with BltFast as quickly as it can. I recorded it with OBS at 60 FPS on a 144Hz monitor. With DDrawCompat, the whole text is rendered in 1-2 frames (~5 with debug logs enabled). With dgVoodoo it takes ~12.

The game water is darker than normal:

Darker than what? To me, it looks about the same as on Voodoo 2 in that particular save.

Darker than what? To me, it looks about the same as on Voodoo 2 in that particular save.

Here, it looks:

I don't get it. That still looks pretty dark. Where is the "non-dark" version? And with what hardware or wrapper?

I don't get it. That still looks pretty dark. Where is the "non-dark" version? And with what hardware or wrapper?

I am talking about the water. By dark, I meant to say it's more bluer, and more specifically, non transparent too.
Its a Voodoo 3

I think the non-transparent version matches the Voodoo 2 and the transparent version matches the Voodoo 3. Of course they look different, since the transparent version has colors mixed with the bottom of the pool...
As for why it's transparent in one case and not the other, I have no idea. It must be an engine bug, probably it skips alpha blending when it shouldn't or vice versa. Maybe related to all the memory faults. With RGB565 texture format it's always transparent, but with palettized textures it depends on who-knows-what.

Ok, I think I figured out the transparency issue thanks to a hint from the DX6 SDK:

D3DTBLEND_MODULATE
Modulate texture-blending mode is supported. In this mode, the RGB values of the texture are multiplied with the RGB values that would have been used with no texturing. Any alpha values in the texture replace the alpha values in the colors that would have been used with no texturing; if the texture does not contain an alpha component, alpha values at the vertices in the source are interpolated between vertices.

DDrawCompat always emulates palettized textures via the A8R8G8B8 format, so the texture itself always has alpha values. Therefore, the alpha was taken from the palette rather than the vertices. The only thing I don't understand yet is why the transparency still worked in the other save.
I replaced it with X8R8G8B8, which breaks the palettized texture support in FF8 instead. But it turns out that it was possible to fix by removing the D3DPTEXTURECAPS_ALPHAPALETTE capability. In this case, the game falls back to using color keying instead of palettes with alpha.
I added these changes, along with the ladder "fix", so that the Voodoo 3 behavior can be reproduced more accurately in this game:
ddraw.zip (diff.txt compared to v0.5.4)
I don't know if this breaks any other palettized texture games though. If it turns out that no other games need D3DPTEXTURECAPS_ALPHAPALETTE support, and that this alternative method of color key matching works fine everywhere, then I suppose it could stay this way by default in the next release. I really don't want to add more unnecessary config options.

Since the issue is fixed from my side, and probably no further engagement is needed for this game, I am closing the thread from my side.


take care of swift 3

hi brad thanks for your objective c i worked with it alot please take care of swift 3 and ios 10 in this version thanx for your gr8 help

I'm have been working on a fork of GPUImage2 converted to Swift 3 for iOS 10, so I may put it up later.

I'll be doing this as a "swift3" branch, since many people will need the Swift 2.2 code until Swift 3 is in a stable Xcode. Don't worry about porting this over, I'll take care of that while working on any renaming that might be needed to match the Swift 3 API guidelines.

There is now a "swift3" branch that should have the iOS and Mac projects and sample applications converted to Swift 3:
https://github.com/BradLarson/GPUImage2/tree/swift3
I'll most likely have to keep this in a separate branch until Xcode 8 can deploy to the App Store.

Thanx brad alot
Best regards
Mina michel

On Jun 21, 2016, at 9:03 PM, Brad Larson<EMAIL_ADDRESS>wrote:
There is now a "swift3" branch that should have the iOS and Mac projects and sample applications converted to Swift 3:
https://github.com/BradLarson/GPUImage2/tree/swift3
I'll most likely have to keep this in a separate branch until Xcode 8 can deploy to the App Store.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Awesome. Thanks again, Brad.


Queued tasks not running

Dear Airflow Maintainers,
Before I tell you about my issue, let me describe my environment:
Environment

1.7.0 (and a few unrelated fixes I submitted recently)
Running a Scheduler with CeleryExecutor
Example code to reproduce the bug (as a code snippet in markdown)
TimeOperator upstream of two PythonOperators (which then trigger four notifications)
Operating System: Linux
Python Version: 3.4.3(2.7.6 installed as well but it should be using v3)

Now that you know a little about me, let me tell you about the issue I am having:
Description of Issue

After upgrading to 1.7.0 I'm seeing tasks queued that are never run

[2016-04-11 13:36:22,579] {jobs.py:709} INFO - Done queuing tasks, calling the executor's heartbeat
[2016-04-11 13:36:22,579] {jobs.py:712} INFO - Loop took: 0.209075 seconds
[2016-04-11 13:36:22,580] {models.py:256} INFO - Finding 'running' jobs without a recent heartbeat
[2016-04-11 13:36:22,580] {models.py:262} INFO - Failing jobs without heartbeat after 2016-04-11 13:34:07.580924
[2016-04-11 13:36:27,375] {jobs.py:536} INFO - APB: process_events
[2016-04-11 13:36:27,378] {jobs.py:580} INFO - Prioritizing 0 queued jobs
[2016-04-11 13:36:27,401] {models.py:2405} INFO - Checking state for <DagRun update_hdfs @ 2016-04-09 00:00:00: scheduled__2016-04-09T00:00:00, externally triggered: False>
[2016-04-11 13:36:27,401] {jobs.py:488} INFO - Getting list of tasks to skip for active runs.
[2016-04-11 13:36:27,402] {jobs.py:503} INFO - Checking dependencies on 7 tasks instances, minus 4 skippable ones

This seems to be related to:
https://github.com/airbnb/airflow/commit/d498df2e22b75f36da1f365ef0c6a55c40602b1b
If I put the direct query back in all works well, but I'm not sure how the implemented change should work -  process_queue() is being called, but there is nothing in the executor_queue so the loop does nothing (in particular it's not adding anything to self.queued_tis).
AFAICT this all depends upon something calling change_state after a task is queued.

I should mention that these two tasks that get stuck have their own pool (in airflow)

@jlowin

@abridgett there was a conflict with queueing in certain situations that I think has been fixed in Master, could you please try pulling the latest version from github and seeing if the issue is still there?

Thanks - I will do, might be unable to test for a day or so (I had used master and seem something similar last Thursday, decided to use 1.70 instead and that had seemed okay).  I don't support you know which commit(s) may have fixed it in case it's easier to apply the fix rather than go straight to master?

I think it will be this one: https://github.com/airbnb/airflow/pull/1290/commits/b2844af020cb5a470bd83ead09ddb121923084ca

Super - that looks good on a test box, I'll confirm properly tomorrow

FYI just that one commit doesn't seem sufficient.  Not sure if I'll get around to updating to master today

Deploying on a test instance that had about seven days to catchup, things seemed okay but on the last day it's got stuck again.  Even had this email which seems a little odd (jobs are blocked by themselves?)
Here's a list of tasks thas missed their SLAs:
speedmap on 2016-04-12T00:00:00
cellsmap on 2016-04-12T00:00:00
Blocking tasks:
speedmap on 2016-04-12T00:00:00
cellsmap on 2016-04-12T00:00:00

Scheduler log (there's a time delay task that ran first):
2016-04-13 11:29:26,328 INFO - Checking dependencies on 7 tasks instances, minus 1 skippable ones
2016-04-13 11:29:26,339 INFO - Adding to queue: airflow run update_hdfs cellsmap 2016-04-12T00:00:00 --local --pool cellsmap -sd DAGS_FOLDER/update_hdfs.py
2016-04-13 11:29:26,349 INFO - Adding to queue: airflow run update_hdfs speedmap 2016-04-12T00:00:00 --local --pool speedmap -sd DAGS_FOLDER/update_hdfs.py
2016-04-13 11:29:26,386 INFO - [celery] queuing ('update_hdfs', 'speedmap', datetime.datetime(2016, 4, 12, 0, 0)) through celery, queue=default
2016-04-13 11:29:26,387 INFO - [celery] queuing ('update_hdfs', 'cellsmap', datetime.datetime(2016, 4, 12, 0, 0)) through celery, queue=default
...
2016-04-13 12:30:52,260 INFO - Finding 'running' jobs without a recent heartbeat
2016-04-13 12:30:52,260 INFO - Failing jobs without heartbeat after 2016-04-13 12:28:37.260495
2016-04-13 12:30:57,162 INFO - Prioritizing 0 queued jobs
2016-04-13 12:30:57,233 INFO - Checking state for <DagRun update_hdfs @ 2016-04-12 00:00:00: scheduled__2016-04-12T00:00:00, externally triggered: False>
2016-04-13 12:30:57,237 INFO - Getting list of tasks to skip for active runs.
2016-04-13 12:30:57,238 INFO - Checking dependencies on 7 tasks instances, minus 1 skippable ones
2016-04-13 12:30:57,275 INFO - Done queuing tasks, calling the executor's heartbeat
2016-04-13 12:30:57,275 INFO - Loop took: 0.113775 seconds

This was with master@876d765 (with https://github.com/airbnb/airflow/pull/1274 applied on top)

I think the SLA email makes sense -- those two tasks didn't complete on time and they were also the reason (as opposed to an upstream task, for example).
But I'm still confused on the queued issue. Especially since it worked for 7 runs and then failed on the last one -- if the queuing logic were broken I think none of them would have worked.
I don't suppose there's a way you can share a minimal version of your DAG that demonstrates the issue?

Sure, no problem.  It's 200 lines long (but only a few steps so the relevant bit is probably quite small).  Would rather not make it too public - email? (I'm<EMAIL_ADDRESS>if you want to drop me a line)

@jlowin Wondering if this issue is only present in 1.7x? We saw a very similar task-hanging issue on version 1.6.2, but cannot confirm if it's the same.  e.g.
2016-04-28T01:09:59.70973 [2016-04-28 01:09:59,709] {jobs.py:470} INFO - Checking dependencies on 0 tasks instances, minus 0 skippable ones
2016-04-28T01:09:59.75093 [2016-04-28 01:09:59,750] {jobs.py:633} INFO - Done queuing tasks, calling the executor's heartbeat
2016-04-28T01:09:59.75105 [2016-04-28 01:09:59,750] {jobs.py:636} INFO - Loop took: 1.309239 seconds
2016-04-28T01:09:59.75341 [2016-04-28 01:09:59,753] {models.py:222} INFO - Finding 'running' jobs without a recent heartbeat
2016-04-28T01:09:59.75364 [2016-04-28 01:09:59,753] {models.py:228} INFO - Failing jobs without heartbeat after 2016-04-28 01:07:44.753553
2016-04-28T01:10:58.48647 [2016-04-28 01:10:58,486] {jobs.py:507} INFO - Prioritizing 0 queued jobs
2016-04-28T01:10:58.51371 [2016-04-28 01:10:58,513] {jobs.py:455} INFO - Getting list of tasks to skip for active runs.
2016-04-28T01:10:58.51379 [2016-04-28 01:10:58,513] {jobs.py:470} INFO - Checking dependencies on 0 tasks instances, minus 0 skippable ones

Thanks!

Please track https://issues.apache.org/jira/browse/AIRFLOW-56 for this issue.


UploadField TreeDropdown search error

Silverstripe 3.1.9 - UploadField
When a folder in the Treedropdown is selected all following search requests will fail with the error:
ERROR [User Error]: Couldn't run query:  SELECT "ParentID", "ID" FROM "Folder" WHERE "ID" in .... Table 'database_name.Folder' doesn't exist...
Related
http://www.silverstripe.org/community/forums/all-other-modules/show/97285

Can this be closed - fixed by https://github.com/silverstripe/silverstripe-framework/pull/3871?

Sorry for my late response. Yes, this can be closed. Thank You!


Expose a Python API

cf. #355
@frafra, what do you think of this ?

That seems a great improvement to me! :)


Add support for configuring public access prevention on Buckets

Description of your changes
This PR updates the storage api dependency (and what it cascades to also update) and adds support for configuring the PublicAccessPrevention field on Buckets (see https://cloud.google.com/storage/docs/public-access-prevention)
I removed some potential support for enabling/disabling autopilot on clusters, but I cannot find that this setting is possible to update in newer versions of the libraries, nor any mention of that in the docs (via https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview).
If there is a better way to handle this please let me know.
I was a bit unsure how to best handle the permissible string values whilst avoiding larger refactoring. I figured exposing the raw int enum from the google storage API was undesirable, but I can't see e.g a webhook where the allowed values can be verified early.
I have:

[x] Read and followed Crossplane's [contribution process].
[x] Run make reviewable test to ensure this PR is ready for review.

How has this code been tested
Running locally, creating and updating Bucket resource with setting publicAccessPrevention to inherited/enforced and seeing it update similarly in the GCP console. Also removing the field reflects the current value from the GCP console.

Looking at https://cloud.google.com/config-connector/docs/reference/resource-docs/container/containercluster#autopilot_cluster, it seems to point to enableAutopilot being immutable:
enableAutopilot Optional | boolean Immutable. Enable Autopilot for this cluster.

This fell off my radar, but AFAIK it's still relevant? @Feggah, if you have time: do you think my changes and the answers to your comments: https://github.com/crossplane-contrib/provider-gcp/pull/467#discussion_r1027692123 and https://github.com/crossplane-contrib/provider-gcp/pull/467#discussion_r1027871161 make sense to you?


Offer the ability to use grunt-download-electron instead of grunt-build-atom-shell

Quite a lot here are linked to the infamous build-atom-shell grunt task.
See :
https://github.com/atom/electron-starter/issues/69
https://github.com/atom/electron-starter/issues/77
https://github.com/atom/electron-starter/issues/54
Besides the issues some people are facing, this task is slow.
The last time I needed to add a package to the manifest, it took more than 8 hours to complete
I have a bad connection and all, but still.
The build atom-shell task seems only needed to generate a "binary" of the electron app.
This could be replaced by "gunt-download-electron" which would fetch (and cache!) the binary.
As @kakaol mentioned here : https://github.com/atom/electron-starter/issues/71 the build step seems needed for the icon and the executable. For deployment/release this is important. In "dev mode" I would happily trade an executable name and an icon if this allowed me to iterate more quickly.
So, would it be feasible to have an optional --dev switch on the build (and run ?) task to use the downlad grunt task instead of the build one.

The problem is until recently, grunt-download-electron was production usable on all platforms literally only for Atom itself. This is recently no longer the case, so I'll probably switch to using grunt-download-electron instead

Would love to see this functionality... the build-atom-shell task is painful

:+1: same here. painful waiting for build-atom-shell task to complete.

This could probably help with development under Windows as well given that the electron build fails.

As stated in the README, this package is no longer maintained and is deprecated. We recommend that people use electron-prebuilt and electron-packager instead. Because of this, we are archiving this repository and closing all issues and pull requests. Thanks very much for your support and contributions!


Allow changing buffer size

I have a pretty memory-tight application and the parser buffer size of 512 was way overkill for my use-case (the largest individual keys/values are around 20 chars). I'm apparently not the first to have this issue. #19
This PR allows you to override the buffer size, either through build flags or by defining the prop before the library is imported:
#define JSON_PARSER_BUFFER_MAX_LENGTH 128
#include <JsonStreamingParser.h>

I renamed the definition to reduce the chance of conflicts with definitions in other code.

Note: you'll also need to modify increaseBufferPointer() in JsonStreamingParser.cpp, which also refers to BUFFER_MAX_LENGTH:
void JsonStreamingParser::increaseBufferPointer() {
  bufferPos = min(bufferPos + 1, BUFFER_MAX_LENGTH - 1);
}

@ircmaxell thanks for the heads up, that function got changed: https://github.com/squix78/json-streaming-parser/pull/27/files#diff-5a1f311800a154b95827c7c76e82f0c1R205

Can you raise a pull request for
https://github.com/mrfaptastic/json-streaming-parser2
Because this library isn't maintained anymore it seems .


Update to 4.2.4

4.2.4
Enhancements

[x] Add username/name querying for followers and following (closes #111)

/release 4.2.4


Plug in the actual crypto implementation

Motivation
Adds some missing serialization functionality to the crypto crate & integrates it with the app.
Solution

PR Checklist

[x] Added Tests
[ ] Added Documentation
[ ] Updated the changelog

Codecov Report

Merging #19 (fee2082) into master (4554e48) will decrease coverage by 1.07%.
The diff coverage is 2.42%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #19      +/-   ##
==========================================
- Coverage   23.85%   22.77%   -1.08%
==========================================
  Files          17       20       +3
  Lines        5157     5422     +265
==========================================
+ Hits         1230     1235       +5
- Misses       3927     4187     +260

Flag
Coverage Δ

test
22.77% <2.42%> (-1.08%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

crypto/src/contribution.rs
0.00% <0.00%> (ø)

crypto/src/lib.rs
19.70% <ø> (+0.05%)
:arrow_up:

crypto/src/zcash_format.rs
0.00% <0.00%> (ø)

src/api/v1/contribute.rs
92.55% <ø> (ø)

src/io.rs
0.00% <ø> (ø)

src/main.rs
31.30% <0.00%> (-0.15%)
:arrow_down:

src/test_transcript.rs
100.00% <ø> (ø)

src/api/v1/lobby.rs
79.21% <100.00%> (+0.11%)
:arrow_up:

src/io/transcript.rs
100.00% <100.00%> (ø)

... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4554e48...fee2082. Read the comment docs.

Can you refer to issue 4 re the json format for the contribution: https://github.com/ethereum/kzg-ceremony-sequencer/issues/4 ?


Release new version without gulp-util dep

Hi,
I've been using this repo which uses this package and shows warnings about depending on gulp-util dependency of version 4.0.0.
I've seen that the code of this repo was updated 9 months ago (removing deprecated deps.), but the released version was 1 year ago.
Is it ready for a release to npm feed? Can you publish it?
Cheers

Aaand I am reverting the gulp-util change. I didn't notice tests were failing and published the v0.4.1 🙈

I don't know how serious was the issue but I prefer to revert while I am figuring out the issue. Apologies for any inconvenience caused 🙏


AWS Cloud provider setup has additional awsCloudProvider default parameters

What kind of request is this (question/bug/enhancement/feature request): bug
Steps to reproduce (least amount of steps as possible):

Deploy an rke cluster - AWS - with cloud provider enabled. - 2.3-head - latest and master-head
when the cluster is up and running, go to Edit cluster --> Edit as YAML, we notice

  cloud_provider:
    awsCloudProvider:
      global:
        disable-security-group-ingress: false
        disable-strict-zone-check: false
        elb-security-group: ''
        kubernetes-cluster-id: ''
        kubernetes-cluster-tag: ''
        role-arn: ''
        routetable-id: ''
        subnet-id: ''
        vpc: ''
        zone: ''
      service_override: null
    name: aws

This is a change of behavior from earlier release, where the edit as YAML showed only

  cloud_provider:
    awsCloudProvider:
      global:
        disable-security-group-ingress: false
        disable-strict-zone-check: false
    name: aws

Question: Why is there this change in behavior?
Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): master and 2.3-head
Installation option (single install/HA): single

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported): custom/rke
Kubernetes version (use kubectl version):

1.16

@codyrancher Is this a UI bug or backend?

@deniseschannon it's most likely going to be frontend.

@deniseschannon sorry, skimming through I thought this issue was https://github.com/rancher/rancher/issues/24706.
This one is actually a backend issue. The backend sets these values.

@sowmyav27 What is the "earlier" release? Is this v2.2 vs v2.3 or different?

@superseb It was the 2.3.3 release

@codyrancher Is this not because of https://github.com/rancher/ui/pull/3627 where it was hidden before (if I read the PR correctly)?

@superseb all of the defaults were hidden when awsCloudProvider wasn't present. I don't consider rancher/rancher#3627 to be the cause of this issue if you're thinking that we should revert or re-fix it. I think it revealed a new unrelated issue.
When I was testing rancher/rancher#3627 I'm pretty sure I only saw
cloud_provider:
    awsCloudProvider:
      global:
        disable-security-group-ingress: false
        disable-strict-zone-check: false

But even if the above wasn't the case I believe the backend is responsible for populating those default values because I know the front didn't send them when creating the cluster. Do you know otherwise?

Thanks, will look into it

@codyrancher Based on what I found:
v2.3.3 has the same API output for a cluster with aws cloud provider, and the empty values get filtered by the function removeEmpty.  In https://github.com/rancher/ui/pull/3627 the value awsCloudProvider was added to excluded keys, making the empty values not being removed. Not really sure at this point what else it could be, but let me know if I misinterpreted the change in https://github.com/rancher/ui/pull/3627.

@superseb you're right that the removeEmpty was removing those empty values until my change went in. To me the removeEmpty was covering up the issue that the backend is sending the front useless data. The following is what the backend is sending the front:
"rancherKubernetesEngineConfig": {
    "addonJobTimeout": 30,
    "authentication": {
        "strategy": "x509|webhook",
        "type": "/v3/schemas/authnConfig"
    },
    "authorization": {
        "type": "/v3/schemas/authzConfig"
    },
    "bastionHost": {
        "sshAgentAuth": false,
        "type": "/v3/schemas/bastionHost"
    },
    "cloudProvider": {
        "awsCloudProvider": {
            "global": {
                "disable-security-group-ingress": false,
                "disable-strict-zone-check": false,
                "elb-security-group": "",
                "kubernetes-cluster-id": "",
                "kubernetes-cluster-tag": "",
                "role-arn": "",
                "routetable-id": "",
                "subnet-id": "",
                "type": "/v3/schemas/globalAwsOpts",
                "vpc": "",
                "zone": ""
            },
            "type": "/v3/schemas/awsCloudProvider"
        },
        "name": "aws",
        "type": "/v3/schemas/cloudProvider"
    },
    "ignoreDockerVersion": true,
    "ingress": {
        "provider": "nginx",
        "type": "/v3/schemas/ingressConfig"
    },
    "kubernetesVersion": "v1.16.4-rancher1-1",
    "monitoring": {
        "provider": "metrics-server",
        "type": "/v3/schemas/monitoringConfig"
    },
    "network": {
        "options": {
            "flannel_backend_type": "vxlan"
        },
        "plugin": "canal",
        "type": "/v3/schemas/networkConfig"
    },
    "restore": {
        "restore": false,
        "type": "/v3/schemas/restoreConfig"
    },
    "services": {
        "etcd": {
            "backupConfig": {
                "enabled": true,
                "intervalHours": 12,
                "retention": 6,
                "s3BackupConfig": null,
                "safeTimestamp": false,
                "type": "/v3/schemas/backupConfig"
            },
            "creation": "12h",
            "extraArgs": {
                "election-timeout": "5000",
                "heartbeat-interval": "500"
            },
            "gid": 0,
            "retention": "72h",
            "snapshot": false,
            "type": "/v3/schemas/etcdService",
            "uid": 0
        },
        "kubeApi": {
            "alwaysPullImages": false,
            "podSecurityPolicy": false,
            "serviceNodePortRange": "30000-32767",
            "type": "/v3/schemas/kubeAPIService"
        },
        "kubeController": {
            "type": "/v3/schemas/kubeControllerService"
        },
        "kubelet": {
            "failSwapOn": false,
            "generateServingCertificate": false,
            "type": "/v3/schemas/kubeletService"
        },
        "kubeproxy": {
            "type": "/v3/schemas/kubeproxyService"
        },
        "scheduler": {
            "type": "/v3/schemas/schedulerService"
        },
        "type": "/v3/schemas/rkeConfigServices"
    },
    "sshAgentAuth": false,
    "type": "/v3/schemas/rancherKubernetesEngineConfig"
},

It seems strange to expect the front-end to not display the config the back is sending us.  I suspect there may be some aspect of the backend that I'm not understanding. Can you explain why the front-end is expected to remove those fields?
To me it doesn't seem reasonable for the front-end to remove them. I would actually argue for removing removeEmpty all together but I don't know all of the ramifications. I originally thought removeEmpty was being used to remove extraneous stuff from the store data-structure.

The only reason I can think of is that we don't want to show "useless" parameters that dont have a value to make the YAML view as clean as possible. If people want to use a certain parameter, they can add it and then it will be shown (as being not empty). Otherwise the YAML view will show every possible parameter of the cluster which is not very clean.
Looking into the history, it seems to be done in https://github.com/rancher/ui/pull/2096/files to keep configs consistent.

Based on where we are in the release and the regressions that have occurred around the AWS cloud provider section, I'm moving this out of v2.3.4 as this has no negative impact on the yaml except for no-empty values.
I'd like to update the backend to fix sending non default values for this use case (AWS cloud provider) as well as other areas as part of the RKE config that could be populated into the yaml.

On latest master head - commit it: 8bd9f3be2 - ui tag - latest2.

Create an AWS cluster with cloud provider enabled.
When the cluster comes up Active, Navigate to Cluster --> Edit cluster --> Edit as YAML.
cluster.yml has the following.

  cloud_provider:
    awsCloudProvider:
      global:
        disable-security-group-ingress: false
        disable-strict-zone-check: false
    name: aws

On edit cluster, Modify the following

etcd:
      backup_config:
        enabled: true
        interval_hours: 1

Save the changes. The cluster comes up Active.


recovered process doesn't get added to group

I have a test environment setup where I have two nodes talking to each other using swarm. I start 2 processes, one on each node with swarm in group :foo. When I kill node 2, node1 detects it and creates the pid locally but it does not add the process to swarm group :foo.
I am not sure if this is a bug or it is expected. If this is by design, how can I store this metadata so when it re-initialized, I can add it to the right group.

It should be added to the group automatically, this happens as part of the synchronization of metadata across nodes. If the pid is not part of the group, it's likely this synchronization did not occur for some reason. I'll take a look and see if I can reproduce. If you already have a repo I can clone which reproduces this issue, that would be helpful.

@xinz Do you have a repo I can use to reproduce?

@bitwalker , yes, the repo to reproduce this issue is ready now, please clone the master branch of https://github.com/xinz/swarm_test , the steps are:
1, $ mix deps.get
This issue happened in Swarm 3.0.5 and the latest bitwalker/swarm master branch;
2, $ iex --sname app1 -S mix
3, $ iex --sname app2 -S mix
4, Start a worker in app1 node with SwarmTest.Example.start_worker("a1"), the a1 worker may start on app2 node, and we can find the :foo group membership is working via SwarmTest.Example.get_foos
5, Directly exit the iex shell of app2 node, we will find the "a1" worker re-started on app1 node, but the group membership is lost now.
The above steps can reproduce this issue, and processing handoff cases we can catch this issue as well.
I have prepared a fix for this issue, in the mix.exs file of swarm_test, use {:swarm, github: "xinz/swarm"} to replace  {:swarm, "~> 3.0"}, after update the local deps, re-run the above steps can verify this issue, please review and advise.
This commit covers:
1, nodes down case
2, :restart handoff case
3, :resume handoff case
Please let me know if this commit can submit as a PR, and then I will do that.
https://github.com/xinz/swarm/commit/247be8ccd5926aee549ab4aaa5369a4a95b500bc

@bitwalker  Do we have any updates? Could you please advice about https://github.com/xinz/swarm/commit/247be8ccd5926aee549ab4aaa5369a4a95b500bc when you're free?

Sorry for the delay, I'll try to look at this today or tomorrow.

@bitwalker it doesn't matter :)

This still does not appear to be fixed in the current swarm code.  workers get lost from the group when they change nodes.  With Dina/swarm patch, workers never get misplaced.  Can you tell us if xinz's fix can be pulled into the main swarm repo?
Thanks

@msw10100 I took some time and looked at the patch and I think I can get that merged soon - I need some time to run some scenarios, but at first blush I think it's good to go.

Thanks for your suggestion, the PR is submitted, please review.

I encountered a similar problem and I'm unsure if it can be fixed:
When a tracked and supervised process dies, it will be restarted by the supervisor, but then it won't be re-registered by the tracker, i.e. Swarm.registered() is [].

Oh nevermind, I found #6 later

I think this one can be closed then? @slashdotdash @bitwalker ?

Fixed by #37.


Fix spelling

I think the correct spelling is forgo or forego, correct me if I'm wrong.

This should also be updated in the readme as the text is duplicated there, I can do that later


Not loading bookmarks.data - Console error

When trying to load the page am getting an error as below.

XMLHttpRequest cannot load file:///T:/Web-Dev/Github/bookmarks.public/bookmarks.data. Cross origin requests are only supported for protocol schemes: http, data, chrome, chrome-extension, https, chrome-extension-resource.
send @ jquery-3.0.0.min.js:4

Thought this may be related to storing everything from Github on an external drive but get the same error when copying everything to the C:\ drive and trying to run it there.
From the index file looks like this is what the script is trying to prevent? Could this be a windows 10 only issue? Am also using Google Chrome.
Will update issue as I find anything else useful.

Hum, Works fine in Firefox so may be security settings related.

Yeah this is a security thing; most browsers don't allow you to open a HTML file and use AJAX to load other files locally - because it's a risk if you allow javascript to read a local file and then POST it somewhere else.
I've tried to be careful such that this does work, but really the ideal is to store your bookmarks on a remote webserver - github pages will do - such that you're fetching the CSS, JS, and data via normal HTTP.  That'll work everywhere and allow your bookmarks to be portable too.

Thanks for you help, am now accessing this through a local web server and is working fine.
Will have a look at getting this to connect to a database when I have more time to make it easier to save new bookmarks.


Update README.md

I made a few text edits in the README.
Should any of the other fields be in the schema? I only see virtual:hrefs in the schema
@emmanuelmathot

Closing due to conflict with main branch


[BUG] Card does not refresh over time

Prerequisites

[X] Put an X between the brackets on this line if you have done all of the following:

Checked the Quick Start documentation page and confirmed configuration is correct: https://marksie1988.github.io/atomic-calendar-revive/quickstart.html
Checked the FAQs on documentation page for common solutions: https://marksie1988.github.io/atomic-calendar-revive/faq.html
Checked that your issue isn't already filed: https://github.com/marksie1988/atomic-calendar-revive/issues?q=is%3Aissue

Describe the bug
The card state does not refresh over time. Events are not marked as completed, days do not go forward, progressbars do not progress, new events do not show up, no loading is performed etc. The only way to update is to refresh the entire page or navigate away from the dashboard.
This worked fine on the original atomic-calendar implementation so something is amiss, although I cannot tell if it is my environment that is to blame. This has been observed on Chrome browser on multiple computers, all running Windows 10.
Steps to Reproduce

Create a card with a calendar
Show the lovelace dashboard with the card
Create an event in the calendar. The event will not show up by itself
Refresh the page and wait until the event starts. A progressbar will not appear
Wait until the event completes. It will not grey out.

Expected behavior
New events should be loaded promptly, events should properly be shown as completed, days should reflect actual day by itself etc.
Actual behavior:
The lovelace card is effectively frozen in the state it loaded as until a refresh is forced.
Screenshots

Card Configuration

  - disableEventLink: true
    disableLocationLink: true
    entities:
      - entity: calendar.events
      - blacklist: 'Daily, Lunch, synk'
        entity: calendar.work
      - entity: calendar.personlig
      - entity: calendar.releasecalendar
      - entity: calendar.warhammer
      - entity: calendar.helgdagar_i_sverige
      - entity: calendar.vuxna_man_gor_saker_tillsammans
    language: sv
    maxDaysToShow: 1
    noEventsForNextDaysText: Ingenting kommande dagar
    noEventsForTodayText: Ingenting idag
    showColors: true
    showDate: true
    showFullDayProgress: true
    showLoader: true
    showLocation: false
    showNoEventsForToday: true
    showProgressBar: true
    showWeekDay: true
    sortByStartTime: true
    title: Kalender
    type: 'custom:atomic-calendar-revive'

Versions

HomeAssistant Version: 0.110.3
Plugin Version: 1.8.3

Additional Information

The card will refresh every 600 seconds if there has been an update to the calendar.

That has not been my experience. I have added test-events in my calendars to narrow down the behavior and these events do not show up several hours afterwards.
I have also verified that the card does not update the active day either. This Sunday for instance, it was still showing Saturday as the current day, with several events showing as halfway complete even though they were long since finished. The only way to get the card state to update is doing a refresh of the dashboard page.
This all works fine still in the previous atomic calendar incarnation as I've run it in parallel for comparison.

Ok thanks for the input, I've just released a new version that makes some changes to the way updates are triggered, may be worth trying that to see if it resolves your issue.

This did indeed solve the issue, thanks for the quick response! I have noticed that this release seem to have affected the progressbars though - they are always completely full. both on all-day events and standard events.
That is out of scope of this issue so I regard this one as solved.


Scala Kinesis Enrich: allow records of over 1Mb when running in local mode

If events are being printed to stdout rather than sent to Kinesis, there is no need to fail records of over 50kB

Updated to records over 1Mb


Roll Flutter Engine from 3c23ddae1d2a to 8b595a49f33d (1 revision)

https://github.com/flutter/engine/compare/3c23ddae1d2a...8b595a49f33d
2023-05-18<EMAIL_ADDRESS>Roll Dart SDK from 7bc022c51215 to 95b96a9f498a (2 revisions) (flutter/engine#42138)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Increasing test coverage

Made edits for increasing the test coverage.

It looks the Email(ABC) abstract class is not doing anything useful at the moment -- let's just go ahead and remove it since that __init__ method is awkward.
So, this PR can end up being just the change to test_pdf.py. Note that no CHANGELOG line is necessary if just updating tests.
Also branch also does not look mergeable, it may be easier to start with a fresh branch from main, apply the test_pdf.py change, and force push to this same branch name in your working rep.


Revert removal of pixel snapping from flutter engine

See https://github.com/flutter/flutter/issues/110959, https://github.com/flutter/flutter/issues/110957, likely others.
The assumption of the raster cache is that we can replace any layer or display list with a rasterized version of itself. At very low DPR, this is only true if that layer or display list is aligned to physical pixels. Even if we ensure that the texture of the layer or display list is rasterized pixel aligned, when we go to the draw that entry we may find the offset is not physical pixel aligned. Thus we can either snap it to a pixel, leaving it in a different location than the un-rasterized version, or we can draw it in the correct location with obvious aliasing issues.
Unfortunately, that leaves us in a bit of a bind. It means we can't get rid of pixel snapping as long as we have the raster cache. Furthermore, since we didn't notice this issue until the stable release, and there have been many refactors on top of the raster cache since then, it means that a revert is not very straight forward.
Seemingly the only idea long term solution is to revert back to pixel snapping any layers that may be raster cached. The short term change is more uncertain. I can begin working on a patch against the release branch that reverts the removal of pixel snapping. Separately I can also work on re-landing pixel snapping on ToT.
@CaseyHillers do you have a stable release engine branch I can use to investigate the feasibility of a targeted revert?
FYI @zanderso

we can draw it in the correct location with obvious aliasing issues.

Could you describe what these issues are?

Watch as the tooltip fades in. This is the logic for "draw it in the same place it would be drawn". Notice how the text looks aliased. Once the transition is complete, the raster cache entry is dropped and we draw it normally. Skia must internally have better anti-aliasing logic for fonts if it knows it is being drawn at a fractional pixel vs if its is drawn pixel aligned into a texture which is then composited into an app.
https://user-images.githubusercontent.com/8975114/188994983-b25c4fe9-69e8-47ca-81d7-50a2b0bfbac6.mp4
For high dpr screens, we can get away with just snapping the pixel. For low dpr screens it leads to noticeable movement (which is the whole issue that led me down the path)

Current stable is in https://github.com/flutter/engine/tree/flutter-3.2-candidate.5
For future reference, you can go to https://github.com/flutter/flutter/blob/stable/bin/internal/release-candidate-branch.version to find the current candidate branch used by stable.
The existing stable branch is two months old, and IIRC this was very difficult to get rolled into Google as well.

Wondering if @flar might have any thoughts. I understand @jonahwilliams has spent a fair amount of time thinking about this, but at least on master, we are not under time pressure to come up with a quick solution.

I'm not aware of this change having any impact in google3. @CaseyHillers what made it difficult to land?

Okay, I've concluded my investigation of the release branch. Because the removal of pixel snapping occurred before a raster cache refactor and a display_list layer refactor, there is no clean revert without reverting quite a long chain of other PRs. Furthermore, it isn't sufficient to simply draw the cache entries at pixel snapped positions, since this will conflict with the reported positions for layer diffing.
So for stable channel, the potential options are reverting 10+ commits in order to keep things mostly inline, or to create a new PR against stable that re-introduces pixel snapping to the raster cached layers.

@jonahwilliams I followed up. I misremembered this with https://github.com/flutter/flutter/commit/fba9214
\cc @itsjustkevin

Ahh yeah, I promise we don't do that again. And with this revert, we will still not have pixel snapping in the tester, which means the google3 goldens will remain somewhat stabilized

@jonahwilliams my question here is, "is the juice worth the squeeze?"
With the branch being two months old and the roll state not in an ideal place, can we consider doing this work at another time in the near future?
A chain of 10+ reverts seems inherently risky.

Yes, the juice is worth the squeeze.
https://github.com/flutter/engine/pull/35981 adds back the pixel snapping, and is not as disurptive as I was worried it to be, though a straight revert is infeasible.

@jonahwilliams, depending on how close the snapping behavior is to before removal this one might need to be reverted too. https://github.com/flutter/engine/pull/35601 :-/


math/octree.h: Condition ' Math::is_nan(p_aabb.size.x) ' is true

Seen when closing the editor with a project open:
ERROR: create: Condition ' Math::is_nan(p_aabb.size.x) ' is true. returned: 0
   At: core/math/octree.h:827.

(printed 6 times in a row, if that can give a clue as to what triggers it)

+1
Also 6 times on close project.
#3521

Another clue is that it only happens for 2d projects, but if you switch to 3d tab before closing it won't happen. I guess it's nan because it have never been set.

+1
this print in console when opening project.

Can't see the error anymore.
Tested on ubuntu

@AlexHolly Ahem. Which version of Godot did you test?

2.1 ~2 weeks old

Closing as fixed.

I'm experiencing this error message being printed 15+ times upon opening my project. Using 3.0.3-rc3. Any  idea what causes it?

it's because of 0 width or height of node.
especially scaled (0,0) node in my case.

Is there a way to find out which nodes are causing the error?

that's an interesting question.
I'm not sure which is better between avoiding this error or showing what node causes this.

@volzhs: showing what node is the cause would be better, since 0 width or height is likely to break other things, too

I'm a little in the dark, in this case but let me know if anyone needs anything from me to figure out a solution. I'm happy to help if I can.

then we should open another issue not to forget about this or reopen this one.
@akien-mga what do you think?

Facing the same issue after I imported some transparent png images.
Steps to reproduce:

Create new scene
Add a sprite
Load texture from a transparent png file

The issue doesn't happen for other image files. Also the problem happens only in one of my projects. If I create a new project and use the same png file there, the issue doesn't arise. I think it may be related to corrupted import files.
OS: Mac OS 10.13.5 (High Sierra)
Godot Editor: 3.0.4.stable

This is error is showing up in my project, but as yet does not appear to be disrupting execution. I am not certain what changes I've made to cause it, and I do not know of steps to reproduce it from scratch.
I believe that reopening this issue or opening another one (as suggested by @volzhs ) is in order given the number of people that appear to be experiencing this in stable versions of the editor. Thoughts, @akien-mga ?
OS: Mac OS 10.13.6 (High Sierra)
Godot Editor: 3.0.5.stable

The original bug report was about a very specific reproducing scenario where this message was printed 6 times when opening, and 6 times when closing, any project. This has long since been fixed, which is why this issue is closed.
The same error message in current builds doesn't mean that the same bug came back, but that another bug is triggering a similar symptom, and as such it would be better to open a new bug report for it.


Update titles for length

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

server-2013/lync-server-2013-configure-lync-server-2013-to-work-with-unified-messaging-on-microsoft-exchange-server.md
:white_check_mark:Succeeded
View

server-2013/lync-server-2013-configuring-microsoft-exchange-server-2013-unified-messaging-for-lync-server-2013-voice-mail.md
:white_check_mark:Succeeded
View

server-2013/lync-server-2013-configuring-microsoft-sharepoint-server-2013-to-search-for-archived-lync-server-2013-data.md
:white_check_mark:Succeeded
View

server-2013/lync-server-2013-configuring-partner-applications-in-lync-server-2013-and-exchange-server-2013.md
:white_check_mark:Succeeded
View

server-2013/lync-server-2013-configuring-unified-messaging-on-microsoft-exchange-server-to-work-with-lync-server.md
:white_check_mark:Succeeded
View

server-2013/lync-server-2013-deploying-lync-server-2013-standard-edition-into-an-existing-lync-server-2013-enterprise.md
:white_check_mark:Succeeded
View

server-2013/lync-server-2013-planning-for-extensible-messaging-and-presence-protocol-xmpp-federation.md
:white_check_mark:Succeeded
View

server-2013/lync-server-2013-protecting-data-in-transit-–-archiving-monitoring-group-chat-compliance-server-databases.md
:white_check_mark:Succeeded
View

server-2013/lync-server-2013-testing-civic-addresses-against-the-master-street-address-guide.md
:white_check_mark:Succeeded
View

server-2013/lync-server-2013-testing-configuration-of-the-kerberos-account-assigned-to-a-site.md
:white_check_mark:Succeeded
View

server-2013/migration-from-lync-server-2010-group-chat-or-office-communications-server-2007-r2-group-chat-to-lync-server-2013-persistent-chat-server.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add support for selecting alternate cloud_environments

Add support for selecting alternate cloud_environments
Description
Add support for selecting alternate cloud_environments via "cloud_environments" keyword option when creating compute driver object. Supports default, AzureChinaCloud, AzureUSGovernment, and AzureGermanCloud.  Affects endpoints used by azure_arm compute driver.
Status

In testing

Checklist (tick everything that applies)

[ ] Code linting (required, can be done after the PR checks)
[x] Documentation
[ ] Tests
[x] ICLA (required for bigger changes)

urlparse is a Python 2.7 specific standard library. We have a compatibility option in libcloud.utils.py3

Thanks, I'll see what I can do.

Coverage increased (+0.003%) to 87.273% when pulling 07321c4f1f378cd2298a62cf7b6e2050d1d8e930 on curoverse:azure_cloud_environments into 1a07215991a8c6be7e877d669eca9c007308806f on apache:trunk.

Coverage increased (+0.0004%) to 87.27% when pulling f76ab1998b1101abd0f95203691fa75a152ca491 on curoverse:azure_cloud_environments into 1a07215991a8c6be7e877d669eca9c007308806f on apache:trunk.

Coverage increased (+0.003%) to 87.273% when pulling 148456a45f517ff3908c58a130a3f512b9a352e2 on curoverse:azure_cloud_environments into 1a07215991a8c6be7e877d669eca9c007308806f on apache:trunk.

Coverage increased (+0.003%) to 87.273% when pulling<PHONE_NUMBER>634e1860ce5a2ac9d8312d2b905d6e on curoverse:azure_cloud_environments into 1a07215991a8c6be7e877d669eca9c007308806f on apache:trunk.

Coverage increased (+0.003%) to 87.273% when pulling 0db277452468200980fd9a393f246bdb34171cf9 on curoverse:azure_cloud_environments into 1a07215991a8c6be7e877d669eca9c007308806f on apache:trunk.

Coverage increased (+0.003%) to 87.273% when pulling e87938c08164c00fc198cd781502980973411a9d on curoverse:azure_cloud_environments into 1a07215991a8c6be7e877d669eca9c007308806f on apache:trunk.

Coverage increased (+0.0004%) to 87.27% when pulling e87938c08164c00fc198cd781502980973411a9d on curoverse:azure_cloud_environments into 1a07215991a8c6be7e877d669eca9c007308806f on apache:trunk.

Coverage increased (+0.003%) to 87.273% when pulling 16a2611a62af61aa1690d559a02d4a8c6af4bb62 on curoverse:azure_cloud_environments into 1a07215991a8c6be7e877d669eca9c007308806f on apache:trunk.

@tonybaloney is this ready to merge?

@tetron please can you change the __init__ keyword to region to keep it consistent with the ec2 driver and others.

Azure has two concepts: "location" and "cloud environment".  A cloud environment is a separately operated installation of Azure, it has a distinct set of endpoints and separate accounts.  Within a cloud environment there can be many locations (data centers).
When setting up a connection to Azure, it is only necessary to know the "cloud environment".  The "location" supplied in API requests that need it.
Currently AzureNodeDriver maps the region keyword to location, and accepts cloud_environment as an additional parameter which is passed through to AzureResourceManagementConnection.  If we renamed cloud_environment to region in AzureResourceManagementConnection that would give region an inconsistent meaning between the two classes.
So we can either:

Leave it as-is
Merge cloud_environment and location into a single value for region (which gets split by the driver): user would specify centralus (use default) or AzureChinaCloud/southchina (use an alternate cloud environment).
Something else?

sorry that's just my lack of understand of Azure, agreed 👍  merging.


Page with sub-pages fails to load, see attached error

First off- thank you for creating this! Largely, it has been extremely useful.. So kudos to you.
That said, I've hit what appears to be an edge case.
I have a page, which has several subpages (IE, so in the main nav I have a link "about" with a drop-down to subpages, "what we do," "projects," "partners," and "contact.")
All of the subpages load just fine, however if I click the top level link ("About") it throws the below error:

TypeError: Cannot read property 'typeName' of undefined
at getTemplateKey (/usr/local/lib/node_modules/node-squarespace-server/squarespace-template.js:286:35)
at Object.renderTemplate (/usr/local/lib/node_modules/node-squarespace-server/squarespace-template.js:322:23)
at renderResponse (/usr/local/lib/node_modules/node-squarespace-server/squarespace-server.js:187:21)
at onExpressRouterGET (/usr/local/lib/node_modules/node-squarespace-server/squarespace-server.js:323:5)
at Layer.handle [as handle_request] (/usr/local/lib/node_modules/node-squarespace-server/node_modules/express/lib/router/layer.js:82:5)
at next (/usr/local/lib/node_modules/node-squarespace-server/node_modules/express/lib/router/route.js:100:13)
at Route.dispatch (/usr/local/lib/node_modules/node-squarespace-server/node_modules/express/lib/router/route.js:81:3)
at Layer.handle [as handle_request] (/usr/local/lib/node_modules/node-squarespace-server/node_modules/express/lib/router/layer.js:82:5)
at /usr/local/lib/node_modules/node-squarespace-server/node_modules/express/lib/router/index.js:234:24
at Function.proto.process_params (/usr/local/lib/node_modules/node-squarespace-server/node_modules/express/lib/router/index.js:312:12)

It should be noted that the link does load just fine when viewing the site in its xxx.squarespace.com url. So it only fails in this case.
I have tried modifying the config file - removing and re-adding both sitewide password and sandbox mode.. (which I have been able to track down other issues I have had to one or both of those attributes).
I'm not sure if this is a problem with my local node setup, an issue with this local server, or a sqs issue  - though I've used grunt and node on other (non sqs sites) without any problem at all on my dev machine.
Any insight here? Is this a known issue or am I that obnoxious edge case that simply hasn't been addressed?
Thanks for the great tool, and thanks in advance for any insight you may have!
-Josh

I should note that no other top level pages/links have errors. However, no other pages have sub-pages.

I think I know what you are talking about. I have a question about that. When you load that top level page on the xxx.squarespace.com domain, does it load the first sub-page instead?
I ask because I was playing around with indexes and folders a couple weeks ago and noticed that the top level page for a folder with sub-pages would just load the first sub-page in the tree.
Say I have a folder called test-folder and the first sub-page is called test-page. When I click test-folder it hits /test-folder/ but immediately redirects to /test-page/. As far as I have tested, indexes work just fine however.

In my mind, the solve would be to replicate the redirection that occurs in this scenario on the squarespace.com domains. Seems to make sense.

Even with a query string param of format=json on my test site, the /test-folder/ path redirects to /test-page/ with no query string. In testing the node server, the error is throwing because the json is not coming through but rather the html...

I fixed the same issue by following suit with how Squarespace seems to be handling root level folder requests. The branch is here, https://github.com/kitajchuk/node-squarespace-server/tree/folder-root-redirect.
If you could go ahead and install that branch locally and test with it that would be great. Let me know if you think its working correctly.
Thanks man!

Closing this one out. Merged in the above fix. Published new version<EMAIL_ADDRESS>


Mirrored repositories shows  instead of the ID

What happened?
Shows  for each mirrored repositories whilst it should say the Id

Reposilite version
3.x
Relevant log output
No response

Resolved in #1463


[BUG] Loading center state of  doesn't work

Describe the problem
Whenever the loadingCenter property is enabled when the block property is enabled as well breaks the <Button />.
Actual behavior
The problem is that the loader icon is taken out of the button container, and it's size is greatly increased.
Expected behavior
I would expect that the loading text disappears and the loader is positioned centered to the button width, inside the button container.
To Reproduce
Steps to reproduce the behavior:

Enable block button property
Enable both loading and loadingCenter properties

Screenshots
If applicable, add screenshots or gifs to help explain your problem.
Your Environment:

Firefox

Resolved in https://github.com/Kerosz/biolnk/commit/93b791bc7aa3e27b1466b5267b7c60a378272226


build: allow building in Debug mode on Windows

With this tweak, we should be able to build the package in debug mode
even on Windows.  We always use the release mode DLL form of the C/C++
runtime on Windows and can build the Swift code in Debug or Release mode
optimizations.

@swift-ci please test

@swift-ci please test

@swift-ci please test Windows platform


Determine correct paths and collider models in SteamVR when not using Vive Wand

If the rift and touch are used under SteamVR then you get a different model meaning the SteamVR SDK defaults are incorrect.
Need to detect the headset being used with SteamVR and load the appropriate model stuff.

https://docs.unity3d.com/ScriptReference/VR.VRDevice.html

Need to detect the headset being used with SteamVR and load the appropriate model stuff.
Maybe this functionality should go into DeviceFinder. It would be handy for runtime-SDK-switching.

yeah perhaps, but it's just a UnityEngine.VR.VRDevice that returns a string
I'm going to do it basic on this, then we can refactor in the future somewhere more generic if we need it.

PR to fix https://github.com/thestonefox/VRTK/pull/852

Resolved with https://github.com/thestonefox/VRTK/commit/b94b98126c36411b7371a242a9b654eb4e83718d


add specs for nilling deploy parameters

This finishes #2956 with the provider half of the changes.  Adds deploy provider func tests for the case where the array/hash things are cleared and set to nil.  This will make it substantially easier for users to clear out the capistrano bias from the deploy resource and will make our docs work.

@chef/client-core @chef/client-engineers needs review

@chef/client-core @chef/client-engineers pinging again for review

:+1:

:+1:


[cucumber-expressions] make require-able and add example

@aslakhellesoy
Same as #76 but for cucumber expressions

These examples might belong in the cucumber-expressions doc. There was only code samples for the transforms.

@aslakhellesoy bump on this. I really want this in and to have it published (#77) so I can integrate it with cucumber-js and get 2.0 out the door with it.

Will do my best to get it out this weekend!


Change typings for atr parameter in API.SCARD_READERSTATE_OUT

I didn't check what is actually returned for other functions but looking at #222, I guess in other places !Array.<number> is also used incorrectly.


Add IntensityHybridGrid.

Adds a new structure IntensityHybridGrid, similar to HybridGrid
but which stores intensities instead of probabilities.
InterpolatedGrid is adapted to handle both types of HybridGrids.
Signed-off-by: Wolfgang Hess<EMAIL_ADDRESS>

@bochen87 Yes, performance evaluations have been done. All PRs we will send do not have significant negative effect on performance when intensity scan matching is disabled.


Rationale for optional exception

The proposal includes the following text:

If assertions has an entry entry such that entry.[[Key]] is "type", let type be entry.[[Value]]. The following requirements apply:

If type is "json", then this algorithm must either invoke ParseJSONModule and return the resulting Completion Record, or throw an exception.

I don't see why the proposal allows implementers to optionally throw an exception. I've summarized my understanding below to help others find the hole in my reasoning (and potentially identify an opportunity to improve the proposal's documentation).
I'm aware of the security concerns that were initially raised about an early version of this proposal, but it's not clear if/how the optional exception is related. In that issue (and the subsequent conversations at TC39 plenary throughout 2020), it seemed sufficient to give authors a means to communicate the desire, "this resource should never be evaluated as JavaScript." That allows hosts to fail the import based on additional context (e.g. web browsers receiving HTTP responses with inappropriate Content-Type headers).
The part where I get lost is why the proposal text explicitly allows an exception. Even without import assertions, web browsers are already rejecting modules at their own discretion (e.g. due to Content-Security Policy violations). I haven't been able to find any justification for that kind of rejection in ECMA262--the closest I could find was the exception for Module Records that "[do] not exist or cannot be created." My ignorance doesn't prove anything, of course (as demonstrated by the existence of square dancing), but it makes me wonder whether it's absent because it's not necessary. If it is present, I'd be curious to learn why it doesn't apply to this case.
Even if an error is specifically necessary for modules imported using this new syntax, it seems as though the expected exception (and the security guarantee) would be naturally enforced by the semantics of the ParseJSONModule abstract operation. In order to present any danger in this context, the source text would necessarily fail to parse as JSON, resulting in a SyntaxError.
Thanks for the help!

Browsers require the ability to:

error when someone imports JSON without an explicit syntactic marker
not error when someone imports JSON with that same marker
error when someone imports non-JSON with that same marker

node requires the ability to:

not error when someone imports JSON without an explicit syntactic marker
not error when someone imports JSON with that same marker
error when someone imports non-JSON with that same marker

The first item in each list, taken together, seems to require leaving the decision about whether to throw or not up to the host. Please let me know if I'm misunderstanding something about your OP.

Thanks, @ljharb! That's a good summary of stakeholders' requirements, but it doesn't seem to address the behavior I'm wondering about. The option to "throw an exception" is only available in the presence of the explicit syntactic marker. Since the first item in each list describes its absense, neither appear to be related.

@jugglinmike the second two items in both lists describe its presence.

@ljharb That's true! I was focused on the first item in each list because I thought your explanation concerns them specifically.
The second two items apply to the use of "type": "json", so maybe the optional error is meant to allow hosts to control their own expectation for "JSON" versus "non-JSON".
You and I talked about this in IRC; for others following along:

ljharb: if you import without type json today and get .json, browsers insist that must always error, to avoid the footgun referenced just prior
jugglinmike: Yeah, I've read some of the motivating discussion.
jugglinmike: But isn't the rejection (and maybe most importantly, the refusal to execute code) covered by the semantics of ParseJSONModule?
jugglinmike: In other words, if the server starts sending JS, won't the import naturally fail when it calls JSON.parse?
ljharb: not necessarily - valid json can also be js
ljharb: and browsers want to reject based on mime, not content
ljharb: in the case of something with a filesystem it’d likely be based on file extension, also not content

I'm still pressing on this error because I believe that behavior is already possible for hosts without the additional language currently in the proposal.
My working example is the web platform feature Content-Security Policy. That's a case where browsers are already selectively fail to load modules according to their own internal priorities.
As I understand it, this is allowed by ECMA262 via HostResolveImportedModule. By my reading, web browsers are essentially saying that CSP-violating module records "cannot be created." If that's true, then can't that same language also be used to justify the rejection of modules which hosts have classified to be "non-JSON"?

In this case, the module itself can't be rejected, only the importing callsite.

Would you mind giving an example of a case where the distinction is observable?

Promise.allSettled([import('path/to.json'), import('path/to.json', { assert: { type: 'json' } })]) should in a browser produce a Promise of an array of two settlement objects, one rejected and one fulfilled.
In node, for example (assuming the assertion ends up being optional, which has always been the plan), that should provide a Promise of an array of two settlement objects, both fulfilled.

This change from the Import Assertions proposal reads like it would allow hosts to create distinct module records for each of those two ImportCalls:

Each time this operation is called with a specific referencingScriptOrModule, moduleRequest.[[Specifier]], moduleRequest.[[Assertions]] triple as arguments it must return the same Module Record instance if it completes normally.

It is recommended but not required that implementations additionally conform to the following stronger constraint: each time this operation is called with a specific referencingScriptOrModule, moduleRequest.[[Specifier]] pair as arguments it must return the same Module Record instance if it completes normally.

If so, couldn't browsers reject the module record itself (i.e. without the optional error here in JSON modules) and still implement the expected behavior?

Yes, browsers need the capability to do that because of race conditions - two simultaneous import()s could receive different modules/MIME types, and both need to respond appropriately (rather than the first to arrive dictating the second to arrive's behavior). However, any host that doesn't fetch modules over the network wouldn't need that flexibility (browsers won't either, when the cache is already populated), and they will be able to conform to the stronger constraint.

Is it accurate to say that the exception we're discussing isn't observable and that it's present to allow hosts to conform to an optional constraint?

It would certainly be observable on browsers if you controlled the server and the client. Whenever it's not observable, then engines are encouraged to conform to the stronger constraint.
(It's certainly not something test262 should or can test, I think)

It would certainly be observable on browsers if you controlled the server and the client. Whenever it's not observable, then engines are encouraged to conform to the stronger constraint.

It seems that it isn't possible for authors to differentiate between the following two compliant behaviors:

the host tracks Module Records by the "import triple" and never uses the optional rejection in the JSON Modules proposal
the host tracks Module Records by the "import pair" and uses the optional rejection in the JSON Modules proposal at its discretion

Is that right?

(It's certainly not something test262 should or can test, I think)

That's been my suspicion, as well!


Use after free triggered on application exit

Please fill in the following fields:
Pre-built SDK from the website or open-source from this repo: pre-built
Firebase C++ SDK version: 6.15.0
Firebase plugins in use (Auth, Database, etc.): Auth
Additional SDKs you are using (Facebook, AdMob, etc.): none
Platform you are using the C++ SDK on (Mac, Windows, or Linux): mac
Platform you are targeting (iOS, Android, and/or desktop): all
Please describe the issue here:
When the c++ runtime is cleaning on exit, the cleanup call at https://github.com/firebase/firebase-cpp-sdk/blob/d464bcfd26eec7735d7a8374b0cd325e77fc0280/auth/src/auth.cc#L160 causes a use after free memory access:
=================================================================
==6558==ERROR: AddressSanitizer: heap-use-after-free on address 0x604000004450 at pc 0x000106e57cc1 bp 0x7ffee8f89d30 sp 0x7ffee8f89d28
READ of size 8 at 0x604000004450 thread T0
    #0 0x106e57cc0 in void std::__1::__tree_remove<std::__1::__tree_node_base<void*>*>(std::__1::__tree_node_base<void*>*, std::__1::__tree_node_base<void*>*) __tree:354
    #1 0x1076f2655 in firebase::auth::Auth::DeleteInternal() (flib-ui:x86_64+0x100a7d655)
    #2 0x1076f27a8 in firebase::auth::Auth::~Auth() (flib-ui:x86_64+0x100a7d7a8)
    #3 0x1071e574a in std::__1::default_delete<firebase::auth::Auth>::operator()(firebase::auth::Auth*) const memory:2338
    #4 0x1071e56ce in std::__1::unique_ptr<firebase::auth::Auth, std::__1::default_delete<firebase::auth::Auth> >::reset(firebase::auth::Auth*) memory:2593
    #5 0x1071e5608 in std::__1::unique_ptr<firebase::auth::Auth, std::__1::default_delete<firebase::auth::Auth> >::~unique_ptr() memory:2547
    #6 0x1071e5284 in std::__1::unique_ptr<firebase::auth::Auth, std::__1::default_delete<firebase::auth::Auth> >::~unique_ptr() memory:2547
    #7 0x1071e5190 in my_ns::backend_firebase::~backend_firebase() backend_firebase.hpp:33

This is more or less the way I create and use the auth object:
class backend {
    std::unique_ptr<App> app;
    std::unique_ptr<Auth> auth;
    static std::shared_ptr<backend> instance;

public:
    std::shared_ptr<backend> get_instance() {
        if (instance == nullptr) {
            instance = std::make_shared<backend>();
            instance.app = std::unique_ptr<App>(App::Create(...));
            firebase::InitResult result{};
            auto *auth = auth::Auth::GetAuth(instance.app.get(), &result);
            assert(result == firebase::kInitResultSuccess);
            instance.auth = std::unique_ptr<auth::Auth>(auth);
        }
        return instance;
    }
}

So I lazy load the auth to avoid loading firebase for part of my application that doesn't use it. On exit the c++ runtime will cleanup globals in the reverse order that they were created. It happens that in my case the runtime first instanciates the backend std::shared_ptr with nullptr as value, then the firebase g_auths global. Then I call get_instance and the shared ptr is now holding a backend instance. Then I exit the application. The c++ runtime then first destructs the g_auths and then destructs my backend instance which in turn destructs the auth instance which tries to remove it self from the already destructed g_auths map.
I'm now working around this by setting the backend instance static global to nullptr on exit  but it kind of ruin the point of using smart pointers. Maybe the g_auth map could be part of some global context that I could also lazy load in a smart pointer. Anyway, not a big problem at all but I wanted to let you know of my use case and the problem with the current api. Also maybe you have a better solution for me: if so, I'm all ears ;) Thank you!
Please answer the following, if applicable:
Have you been able to reproduce this issue with just the Firebase C++ quickstarts ? n/a
What's the issue repro rate? (eg 100%, 1/5 etc) 100%

@canatella
Thank you for reporting this.
This seems to be a valid issue when Auth or any Firebase instance is held by a static smart pointer.
As you described, if g_auths is destroyed before Auth instance, accessing g_auths and g_auths_mutex  would be pretty bad in Auth::DeleteInternal().
Perhaps making g_auth a pointer like what we do in database.
https://github.com/firebase/firebase-cpp-sdk/blob/d464bcfd26eec7735d7a8374b0cd325e77fc0280/database/src/common/database.cc#L62
Still feel pretty nervous to access the mutex though.
I'll report to the team about this issue.  May require some refactoring.
At the meantime, please try to clear your smart pointers before the app shutdown.

After doing a quick scan, I think Auth probably is the only one that is NOT using a pointer for the map (All app, database, firestore, storage are using pointer).
Perhaps this is where we can start.


Fix_ml_ga_example

Co-authored-by: Kathy Pippert<EMAIL_ADDRESS>

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 63.76%. Comparing base (5442a4e) to head (6894263).
Report is 3 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #3050       +/-   ##
===========================================
- Coverage   86.63%   63.76%   -22.88%
===========================================
  Files          52       52
  Lines        9498     9498
===========================================
- Hits         8229     6056     -2173
- Misses       1269     3442     +2173


Cognito Error when deploying a public cloud tamer environment: InvalidEmailRoleAccessPolicyException

Receiving the following error when deploying tag tamer to a new vpc (public) -  cloudforamtion template
"errorCode": "InvalidEmailRoleAccessPolicyException",
"errorMessage": "Cognito is not allowed to use your email identity",
Changing line 567, EmailSendingAccount: COGNITO_DEFAULT to EmailSendingAccount: DEVELOPER fixes the issue

Hi gesparza8138,
Thanks for reporting the issue you encountered and the fix you identified.
Before you ran the AWS CloudFormation template did you apply an Identity Policy, similar to the one below, to your Amazon SES validated email address so Amazon Cognito could access the validated email address?
{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "stmnt1234567891234",
            "Effect": "Allow",
            "Principal": {
                "Service": "cognito-idp.amazonaws.com"
            },
            "Action": [
                "ses:SendEmail",
                "ses:SendRawEmail"
            ],
            "Resource": "<your SES identity ARN>"
        }
    ]
}

Hi billdry,
Happy to help. Yes, i did have a ses identity policy configured (copied below). Not exactly sure why it was failing but one theory i have from reading the cognito documentation is that when using COGNITO_DEFAULT some parameters aren't allowed of which two (EmailVerificationMessage & EmailVerificationSubject) are referenced in the cloudformation template
my ses identity policy below
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "stmnt1626199382155",
"Effect": "Allow",
"Principal": {
"Service": "cognito-idp.amazonaws.com"
},
"Action": [
"ses:SendEmail",
"ses:SendRawEmail"
],
"Resource"<EMAIL_ADDRESS>}
]
}

Thanks for sharing your identity policy.
I will test the AWS CloudFormation Template deployment using "DEVELOPER" and "COGNITO_DEFAULT" to see if I can replicate your issue.

I replicated your AWS CloudFormation stack launch failure when "EmailSendingAccount" is set to "COGNITO_DEFAULT" in the CloudFormation template but the SES authorized email's Identity Policy is empty/blank.
Setting "EmailSendingAccount" in the CloudFormation template to "DEVELOPER" creates an IAM Role called "AWSServiceRoleForAmazonCognitoIdpEmailService" in the account.  This role allows the Amazon Cognito User Pool to access and send emails using the SES authorized email address instead of Cognito using the SES authorized email's Identity Policy.
I made this change from "COGNITO_DEFAULT" to "DEVELOPER".  All solution installation tests have passed.  I will commit new CloudFormation templates with this change into the repo.
Further explanation of IAM role: AWSServiceRoleForAmazonCognitoIdpEmailService


[cmake_test] remove libxc patch

NOT TO BE MERGED.
A CI run to determine what happens without the Libxc patch and to check when the upstream solves this problem.

Logged at https://gitlab.com/octopus-code/octopus/-/issues/996