custom cloudformation parameters for environments

How can I pass custom cloudformation parameters whose values change for different environments? I'd like to leverage Amplify Environments but I don't know what I need to do to get environment specific CF parameters passed to each CF stack. Does the Amplify CI/CD Console help?
I discovered if I create the "parameters.json" file in my Function stack folder, the values are passed to CF deploy. What happens when I define another environment? Does the "parameters.json" file get replaced?
Also, I only discovered "parameters.json" on a wild guess. Is there a bug in the code generator for Functions that this file is not created automatically?
** Which Category is your question related to? **
I want to give my Function stack custom Cloudformation parameters that are specific to the environment. For example, my API may have a dependency URI different in DEV and PROD environments.
** What AWS Services are you utilizing? **
AWS AppSync, Lambda, Cognito, Amplify CI/CD, etc
** Provide additional details e.g. code snippets **
E.g. Sample code, versions of Amplify you are using
npm list -g --depth=0 | grep -i amplify
<EMAIL_ADDRESS>

Closing based on the last response. Please comment on this thread if you're still stuck and we'll re-open the issue.

@kaustavghosh06 this works great when i run amplify push from my local machine.
When I commit to codecommit and git push the amplify CICD build breaks saying my custom parameters were not provided.

This is linking the CICD question, which is related to the local CLI problems in EDIT 2
https://forums.aws.amazon.com/thread.jspa?threadID=318114

@cyrfer If you made changes to your team-provider.info file and want to do a push - you can install the latest version of the CLI - 4.16.1 and run amplify push --force - this would pickup your changes from the team-provider-info.json file and populate it into the Cloudformation tempalte.
Are you still stuck with the Amplify console build issue? Could you share the error stack from the Amplify console build step?

Closing this issue. If you're still stuck please comment on this thread.

@kaustavghosh06 yes, still stuck in amplify console. The build console output before the error was not that useful, but the downloadable logs were more useful.
Here is the Cloudformation event showing the real errors, obviously from the custom parameters that are still not being passed from the team file (attached):
Parameters: [InternalBucketName, MongoCollectionJobs, MongoUrl, MongoCollectionUsers, MongoDb, MongoUrlSecret, stateMachineArn] must have values

I want this tool to work for me but it really drives me crazy at times.
I wish I could get some insights at when the backend-config or team provider files are used in deployment. Today I found a critical feature that is barely documented, but I cant get it to work either.
@kaustavghosh06 you ask me to reply but you don't open the ticket. What should a developer  do to get support with the CLI workflow since it is not supported by AWS Support?

How about using SSM Parameter Store to store and get CFn parameters.
e.g. SSM Parameter Store Key: cognito-federation-access-id-${env name}
You can access amplify env name in CFn template.
e.g. Parameters in auth category template
Parameters:
  env:
    Type: String

We found a semi work around for passing new environment variables to the functions:
Modifying the team-provider-info.json we were able to add a new section under function like this:
"dev": {
        "awscloudformation": {
           . . .
        },
        "categories": {
            "function": {
                "demowbd58adad4": {
                    "url": "dev.example.com
                }
            }
        }
    }

We then took went to the CloudFormation template for the function and added it to the parameters at the top and was able to !Ref it later in the file for environment variables. This seemed to work just fine when using amplify push.
Now the bug we found is around this:
Next we need to create a new env for prod. So we run amplify env add and fill in the defaults for  the new env.
Now we will update the team-provider-info.json with our environment variables that we added like above:
"prod": {
        "awscloudformation": {
           . . .
        },
        "categories": {
            "function": {
                "demowbd58adad4": {
                    "url": "prod.example.com
                }
            }
        }
    }

We decide not to run push and check out our dev environment again, the team-provider-info.json file deletes any variables put in the categories that was not synced. When checkout any env it will remove all the variables from all env even if they are pushed.
Other ways of it causing issues include:

Running amplify init --env  will also remove this variables (This could be the cause of aws-amplify/amplify-console#206 )

@jaga810 please, continue.

Store specific configurations for each environment in Parameter Store using env name in key.

Refer configuration in Parameter Store in your CFn template using !Sub and resolve:ssm

Parameters:
  env:
    Default: production
    Type: String

Resources:
  S3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub '{{resolve:ssm:/your-project/${env}/s3-bucket-name:2}}'

If you store credentials, you should use resolve:ssm-secure instead of resolve:ssm

@jaga810 that's really cool. I did not know about resolve. I'll look for opportunities to apply it.
Unfortunately, I should have mentioned, some of the parameters I want to pass are resource ARNs that are created in other stacks of Amplify. So for an extra environment, there would be new ARNs that and I would not want to go around updating SSM Parameters.
Luckily for this part of the problem, I discovered and noted a solution over here. The file parameters.json supports a syntax gem that helps a ton. Anyone know where this is documented?
{
  "stateMachineArn": {
    "Fn::GetAtt": [
        "stateMachineaudio",
        "Outputs.StateMachineArn"
    ]
  }
}

I was able to use the CLI (version 4.19.0, with bug fix #4161) to deploy a 2nd environment including specifications for environment-specific resources and environment variables.
I did run into a small problem that I wrote a separate ticket for.
https://github.com/aws-amplify/amplify-cli/issues/4335
Now I have only a couple concerns remaining about using Amplify for multiple environments,

I have not yet (but will soon) tested deploying into a different account or region.

I responded to your issue #4335
Regarding multiple environments, you will be able to deploy your distinct environments in multiple regions or different accounts. The key out here is to have the right AWS profiles set and have that associated with the corresponding environments that you create (you make that association whenever you create a new env). Please feel free to comment on this thread if you  have more questions. Closing this issue.

@kaustavghosh06 using the profiles makes sense for multiple accounts and regions.
For point 2, I think I need a subset of the app deployed, not the full app. For example, I need a State Machine in multiple regions, but my API in only one region.

@kaustavghosh06 this issue is not resolved. Custom parameters per environment are still a challenge. After seeing success passing parameters using team...json file, I see them overwritten when I:

git clone MYREPO
amplify init

The clone has my parameters, but running amplify init overwrites them. I must discard the change amplify init causes for my parameters to work.
Only the parameters for the function resource categories were affected. The hosting and api categories were not overwritten.
$ amplify -v
4.19.0

@cyrfer If you're sharing your project with your team or cloning it on a different machine it is recommended to remove the team-provider-info.json file from your .gitignore file - that way you'll not loose your team-provider-info.json file.
We've documented this out here as well - https://docs.amplify.aws/cli/teams/shared#sharing-projects-outside-the-team

@kaustavghosh06 I did. Like I said, some categories were not affected.

@cyrfer The overwriting of these parameters was fixed in #4161 (and is a part of the latest version of the CLI) and you'd previously acknowledged of that the fix worked for you. Is that still an issue?
This is the reason why anyone would add the team-provier-info.json file to the .gitignore file - https://docs.amplify.aws/cli/teams/shared#sharing-projects-outside-the-team

and you'd previously acknowledged of that the fix worked for you. Is that still an issue?

I acknowledged that CLI 4.19.0 allowed me to successfully deploy one environment with env-specific parameters, specified in the team.json file, per working with @wizage offline.
Deploying to one environment is no longer an issue.

@kaustavghosh06 please keep this ticket open until this is resolved.
https://github.com/aws-amplify/amplify-cli/issues/3240#issuecomment-634109040

Okay - so the request out here is to store the team-provider-info.json file in the cloud as well
to share across team-members since there is hesitation that git isn't the right way to share this file within the team? Let me know if that's the request out here? I'll change the GitHub issue title since is a different request than the original bug filed initially which we fixed.

Not even close. :(

@cyrfer What do you mean?

I don't know how to say it any clearer than I did here.
https://github.com/aws-amplify/amplify-cli/issues/3240#issuecomment-634109040

Okay - so the request out here is to store the team-provider-info.json file in the cloud as well

This is very close to what Terraform does with its .tfstate files.  Big fan of it tbh along with state locking when deploying using DynamoDB.
That aside.  Having read through this thread I do think that the docs could benefit from an end to end guide on how to add custom variables to Lambda functions from beginning to end.  If this already exists I'd really appreciate a link :pray:

@Mentioum
Here is the current guide on using Custom CF, which provides enough info that one can use it for custom environment variables in Lambda Functions.
https://docs.amplify.aws/cli/usage/customcf
To be clear, I am not asking for help with custom environment variables for a Lambda. I'm comfortable customizing the CF templates generated by the Amplify CLI, and passing CF template parameters as Function environment variables.

@Mentioum
Here is the current guide on using Custom CF, which provides enough info that one can use it for custom environment variables in Lambda Functions.
https://docs.amplify.aws/cli/usage/customcf
To be clear, I am not asking for help with custom environment variables for a Lambda. I'm comfortable customizing the CF templates generated by the Amplify CLI, and passing CF template parameters as Function environment variables.

Thanks for the link. Yep I understand that and I'll want that too eventually.  At the moment I'm just adding All envs to the CF template and the using them conditionally based on the ENV environment variable.
Unideal and obviously not as secure as I'd like in the longer run. But ok... for now.
Looking forward to multienv env support 💪, until then it looks looks like KMS is the best option... bit of a pain.

Storing environment specific properties in the team-provider-info.json file breaks down when using AWS Console PR previews. Deployments fail in CF with “variable must contain a value” errors because the synthesized environment names for each preview are not in the team-provider-info.json file.
Is there any workable solution to that issue?

cc @edwardfoyle @swaminator for visibility on state management rework

i figured out a couple of helpful tips and tricks with regards to custom cf services in amplify.
for starts, i can reformat the auto generated cf json script to yaml, and it still works.
i then add all my general aws cf requirements into amplify custom services. and everything is once again in one space (instead of having CF scripts in git repos pipelines etc).
i also figured out how to pass parameters into the custom CF scripts, by using the parameters.json file (this has to remain a json file though).
but - my question (and i think others had the same question, yet no answers)....
how can I set up ENV variables in amplify, and pass those env variables down into the amplify / cf-cli runtime so that i can then feed my amplify custom cf scripts with env variables which were created in the amplify console?
P.S.

alternative is to use ssmparms - this is not an option, since we even use CF to create ssm parameters
alternative is to use CF mappings / conditions based on the amplify "env" variable - this is clumsy and tuff to maintain in a large setup
alternative is to execute aws cloudformation cli directly from the amplify pipeline - once again this feels counterproductive since amplify gave us this nice custom service feature

and one last question:
how does amplify assign a value to the "env" variable which is a standard variable in all the cf scripts?
i tried to trace this back through all the json files to see how this gets an assigned value, but could not find how it is done


Strange Glitch and enormous use of CPU

I'm new to this i have just started the demo using the precompiled version and i get this strange glitch, and a very high use of CPU (i have an i7 4790k and i start from 40c going to 82c using torque! why?)

Need more info, like a console log shoved over onto a gist, http://pastebin.com/ , or https://hastebin.com/ say.

Hi @RealAquilus have you still have the same problem? can you share more info about the issue, like what build you used "Debug" or "Release" and share the console.log.
@Azaezel we can close this issue if we don't have any feedback.


Quaternation issues?

Following a thread posted on forums.gta5-mods.com, it seems ScriptHookVDotNet have some problems calculating quaternions in some particular scenarios.
More exactly —and to repost an example in wrote there—, I firstly converted a Menyoo-based map to .ymap (Los Santos Electric Railway [Replace]洛圣都电气化铁路 to be exact) but two specific replaced items aren't rotated in-game (rail catenaries replacing prop_cs_20m_rope.ydr and prop_cs_30m_rope.ydr in \Grand Theft Auto V\mods\x64c.rpf\levels\gta5\props\lev_des\lev_des.rpf), standing always in a horizontal position when loaded as addon map.
However they are correctly rotated/shifted when loaded in both Menyoo or Map Editor in-game or when displayed in the Codewalker app.
Another user also reported the same problem while exporting their map as addon.
Does this quaternation issue is related to a bug within ScriptHookVDotNet?

@CalicoApricot
I have fixed issues on quaternions in PR #773 #774 #775 .
Can you verify one again, if the issues are still persisting ?


Candy Crush Game

Create a Candy Crush Game using HTML, CSS and Javascript
User will be able to drag the elements to match 3 of them and get points

@Dharmesh177 pls check

Hey @dhairyaostwal @nandurijv , I would like Create a Candy Crush Game using HTML, CSS and Javascript.
I'm a contributor to GSSOC'23 please assign this issue to me as a part of GSSOC'23

@cleveranu Can this pls be assigned ?

@Dharmesh177

Sure !!
Assigned to u. Complete it in 10 Days and follow contributing.md  Guide


Inference stuck！

System Info
onnxoptimizer           0.2.7
optimum                 1.12.0
system: centos7

Problem: Convert llama2 to onnx, and then inferencing stuck！

inference code:

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("./hugg_face_with_past/")
# if with past: use_cache=True, use_io_binding=True
model = ORTModelForCausalLM.from_pretrained("./hugg_face_with_past/", use_cache=True, use_io_binding=True)
inputs = tokenizer("My name is Arthur and I live in", return_tensors="pt")
inputs.pop('token_type_ids')

gen_tokens = model.generate(**inputs)
print(tokenizer.batch_decode(gen_tokens))

log:
2023-09-07 19:37:07.305776: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
[2023-09-07 19:37:08,463] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
/home/centos/.conda/envs/llms/lib/python3.9/site-packages/transformers/generation/utils.py:1338: UserWarning: Using `max_length`'s default (4096) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.
  warnings.warn(

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction (minimal, reproducible, runnable)

..

### Expected behavior

Inference success!

Hi @empty2enrich, which model are you using? Do you face the same issue when loading with transformers AutoModelForCausalLM?
Something you could try is to add for example max_new_tokens=3 to the generate call to check whether it is not simply the model generating a very long sequence.


Update index.css

beautified the website

Plz accept my pull request 🙏🥲 I am just short of 1 request to clear
hacktoctoberfest
On Tue, Nov 1, 2022, 12:07 AM Avinash Singh @.***>
wrote:

Closed #396
https://github.com/avinash201199/Free-courses-with-Certificates/pull/396
.
—
Reply to this email directly, view it on GitHub
https://github.com/avinash201199/Free-courses-with-Certificates/pull/396#event-7707025133,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQTZ26ZSBRDOIUE7YKB7CLDWGAGYJANCNFSM6AAAAAARTK5T4E
.
You are receiving this because you authored the thread.Message ID:
<avinash201199/Free-courses-with-Certificates/pull/396/issue_event/7707025133
@github.com>

they review every pr you should have contributed more not only one word

Actually, I got to know about hacktoctoberfest fest quite late 🥲 and the
deadline was quite narrow by then. 😅
On Tue, Nov 1, 2022, 12:51 AM Avinash Singh @.***>
wrote:

Plz accept my pull request 🙏🥲 I am just short of 1 request to clear
hacktoctoberfest
… <#m_5648581186962637010_>
On Tue, Nov 1, 2022, 12:07 AM Avinash Singh @.***> wrote: Closed #396
https://github.com/avinash201199/Free-courses-with-Certificates/pull/396
<#396
https://github.com/avinash201199/Free-courses-with-Certificates/pull/396>
. — Reply to this email directly, view it on GitHub <#396 (comment)
https://github.com/avinash201199/Free-courses-with-Certificates/pull/396#event-7707025133>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQTZ26ZSBRDOIUE7YKB7CLDWGAGYJANCNFSM6AAAAAARTK5T4E
. You are receiving this because you authored the thread.Message ID: <
/pull/396
https://github.com/avinash201199/Free-courses-with-Certificates/pull/396/issue_event/7707025133
@github.com>
done
—
Reply to this email directly, view it on GitHub
https://github.com/avinash201199/Free-courses-with-Certificates/pull/396#issuecomment-1297559021,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQTZ264MISZVNMO7655FAG3WGALZ3ANCNFSM6AAAAAARTK5T4E
.
You are receiving this because you authored the thread.Message ID:
@.***
com>


remove inactive people from OWNERS

remove people that are no longer active in this project from OWNERS file.
In the OWNERS file that would be left with just one person, I randomly added someone 😃

/retest

/lgtm

/approve


Update qt5 to 5.12.1 and add some missing modules

still missing are androidextras, macextras, x11extras, webengine, webglplugin, webview, serialbus

thinking about using features for the qt5 port instead of putting everything in as default. What do you think?

thinking about using features for the qt5 port instead of putting everything in as default. What do you think?

Could you, please, explain?

@Zingam sure :) I would split the qt5 port into features like quick, web, GUI, plugins so that when you install qt5 only the base is installed and not the whole framework. The other modules are installed via features or by specifying them like it is now. So the qt[quick] for example would include qt5-base, qt5-declarative, qt5-quickcontrols and qt5-quickcontrols2.

Sounds great! What about Web related stuff?

@Zingam some modules are still not working and I currently don't know how to get them to work if that's what you mean. So the web feature would be a thing for the future with webengine and other related modules.

fixes #3859 #5038 #4804

Excellent job @TheScarfix!
Looks like there are no regressions, so I'll merge this PR.
Here's the current status of Qt 5 on each platform.

package
arm-uwp
arm64-windows
x64-linux
x64-osx
x64-uwp
x64-windows
x64-windows-static
x86-windows

qt5
skip
skip
skip
skip
skip
pass
skip
pass

qt5-3d
skip
skip
skip
skip
skip
pass
skip
pass

qt5-activeqt
skip
skip
fail
skip
skip
pass
fail
pass

qt5-base
skip
skip
pass
skip
skip
pass
pass
pass

qt5-charts
skip
skip
fail
skip
skip
pass
pass
pass

qt5-connectivity
skip
skip
fail
skip
skip
pass
pass
pass

qt5-datavis3d
skip
skip
fail
skip
skip
pass
pass
pass

qt5-declarative
skip
skip
fail
skip
skip
pass
fail
pass

qt5-gamepad
skip
skip
fail
skip
skip
pass
pass
pass

qt5-graphicaleffects
skip
skip
skip
skip
skip
pass
skip
pass

qt5-imageformats
skip
skip
fail
skip
skip
pass
pass
pass

qt5-location
skip
skip
skip
skip
skip
pass
skip
pass

qt5-modularscripts
pass
pass
pass
pass
pass
pass
pass
pass

qt5-multimedia
skip
skip
skip
skip
skip
pass
skip
pass

qt5-networkauth
skip
skip
fail
skip
skip
pass
pass
pass

qt5-purchasing
skip
skip
fail
skip
skip
pass
fail
pass

qt5-quickcontrols
skip
skip
skip
skip
skip
pass
skip
pass

qt5-quickcontrols2
skip
skip
skip
skip
skip
pass
skip
pass

qt5-remoteobjects
skip
skip
fail
skip
skip
pass
fail
pass

qt5-script
skip
skip
fail
skip
skip
pass
pass
pass

qt5-scxml
skip
skip
skip
skip
skip
pass
skip
pass

qt5-sensors
skip
skip
fail
skip
skip
pass
pass
pass

qt5-serialport
skip
skip
fail
skip
skip
pass
pass
pass

qt5-speech
skip
skip
fail
skip
skip
pass
pass
pass

qt5-svg
skip
skip
fail
skip
skip
pass
pass
pass

qt5-tools
skip
skip
skip
skip
skip
pass
skip
pass

qt5-virtualkeyboard
skip
skip
fail
skip
skip
pass
pass
pass

qt5-webchannel
skip
skip
fail
skip
skip
pass
pass
pass

qt5-websockets
skip
skip
fail
skip
skip
pass
pass
pass

qt5-winextras
skip
skip
skip
skip
skip
pass
pass
pass

qt5-xmlpatterns
skip
skip
fail
skip
skip
pass
fail
pass

Missing QtWebEngine. If you can successfully get Qt to 5.12.4 then I might consider sharing my port if there is any interest.


[files/deployment.yaml] switch without_* to with_*

Depends-On: packit-service/deployment#97

recheck


ci: turn on sanitizers

Description: turn on tsan and asan runs for core tests.
Risk Level: low
Testing: CI
Signed-off-by: Jose Nino<EMAIL_ADDRESS>

nice work

Going to merge in spite of the size increase. As before, I have noted in #447, and we'll run another size analysis in early January.


FDM not working on Windows

I try to run a basic FDM example, but I get an error:
Traceback (most recent call last):
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\FDM_TEST.py", line 25, in <module>
    fdm_conn.start()  # Start the FDM RX/TX loop
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\venv\lib\site-packages\flightgear_python\fg_if.py", line 118, in start
    self.rx_proc.start()
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'construct.core.Pass'>: it's not the same object as construct.core.Pass

Process finished with exit code 1

What can I do to fix it?
python version: 3.8 (I also tested on 3.11.2 but nothing changed)
my start options: --fdm=null --max-fps=30 --native-fdm=socket,out,30,,5501,udp --native-fdm=socket,in,30,,5502,udp
When I work via telnet everything is fine

Thanks for the bug report, I appreciate it! :smile:
Ok, so I'm able to reproduce this and it seems that it's only a problem on Windows for some reason. Did you also get this error when running it?
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\...\lib\multiprocessing\spawn.py", line 102, in spawn_main
    source_process = _winapi.OpenProcess(
OSError: [WinError 87] The parameter is incorrect

I don't think so. This is all the error log I have.
I also realised that the error is the same whether the flight gear is running or not.
My friend has a different error, but from what I understand it is a normal error when the flight gear is off:
Traceback (most recent call last):
  File "C:\PYTHON\lib\site-packages\flightgear_python\fg_if.py", line 61, in connect_rx
    self.fg_rx_sock.bind(fg_rx_addr)
OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\PYTHON\lib\site-packages\IPython\core\interactiveshell.py", line 3433, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-87ad27a78f0e>", line 1, in <module>
    runfile('C:/Users/BRINE/AppData/Roaming/JetBrains/PyCharm2022.2/scratches/scratch_4.py', wdir='C:/Users/BRINE/AppData/Roaming/JetBrains/PyCharm2022.2/scratches')
  File "C:\Program Files\JetBrains\PyCharm 2022.2\plugins\python\helpers\pydev_pydev_bundle\pydev_umd.py", line 198, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2022.2\plugins\python\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/BRINE/AppData/Roaming/JetBrains/PyCharm2022.2/scratches/scratch_4.py", line 21, in <module>
    fdm_event_pipe = fdm_conn.connect_rx('localhost', 5501, fdm_callback)
  File "C:\PYTHON\lib\site-packages\flightgear_python\fg_if.py", line 63, in connect_rx
    raise FGConnectionError(f'Could not bind to {fg_rx_addr}: {e}')
flightgear_python.fg_util.FGConnectionError: Could not bind to ('localhost', 5501): [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted

My full error log:

Thanks for the info! I've released a new version (1.2.1) that I'm pretty sure will fix both of your issues (Can't pickle... and Could not bind to). Could you please update to that version and let me know if it's still crashing?

I upgraded to the new version and used the code from the example simple_fdm.py and got this error:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\FDM_TEST.py", line 24, in <module>
    fdm_conn.start()  # Start the FDM RX/TX loop
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\venv\lib\site-packages\flightgear_python\fg_if.py", line 119, in start
    self.rx_proc.start()
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

Then I put all the code except the function under if name == 'main' and got this error:
Process Process-1:
Traceback (most recent call last):
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\venv\lib\site-packages\flightgear_python\fg_if.py", line 84, in _fg_packet_roundtrip
    rx_msg, _ = self.fg_rx_sock.recvfrom(1024)
socket.timeout: timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\venv\lib\site-packages\flightgear_python\fg_if.py", line 112, in _rx_process
    self._fg_packet_roundtrip()
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\venv\lib\site-packages\flightgear_python\fg_if.py", line 86, in _fg_packet_roundtrip
    raise FGConnectionError(f'Timeout waiting for data, waited {self.rx_timeout_s} seconds') from e
flightgear_python.fg_util.FGConnectionError: Timeout waiting for data, waited 2.0 seconds

Nothing is happening in flightgear at this point. And after I manually switched off the program, the error is complete with this:
Traceback (most recent call last):
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\FDM_TEST.py", line 31, in <module>
    time.sleep(0.5)
KeyboardInterrupt

It all seems very strange. Could be a problem with some libraries or with pycharm itself.
Maybe something is corrupted or a problem with my system?
The errors in all cases also do not change depending on whether flight gear is on or off.
In addition I tried to run the simple_ctrls.py example and got an error like the first time:
Traceback (most recent call last):
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\FDM_TEST.py", line 35, in <module>
    ctrls_conn.start()  # Start the Ctrls RX/TX loop
  File "C:\Users\Grishin\PycharmProjects\pythonProject1\venv\lib\site-packages\flightgear_python\fg_if.py", line 119, in start
    self.rx_proc.start()
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Grishin\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'construct.core.Flag'>: it's not the same object as construct.core.Flag

I just have a feeling that it's my system that's the problem or something, and I'm making you look for bugs in the code that might not be there. I will ask a friend to check on his system later today.

Thanks again for the logs :smile:
No this is definitely a bug in flightgear-python (also a bug in construct: construct/construct#1018, but I'll fix that at a later time haha)
I've gone through and fully tested both the FDM and Ctrls interface on Windows and released a new version (1.2.2) that should work.
:warning: Note that you need to change your user code! :warning:

You need to add if __name__ == '__main__': into your code, before you start executing stuff. This is due to how processes are created on Windows. (your code is actually imported by the child process)

I've updated the examples to reflect this

You need to explicitly specify localhost in your FlightGear socket options

I've also updated the docs for this

Let me know how it goes!

Thank you very much, now everything works. But now I have a question. When I use code like this where essentially nothing happens over time or sometimes the plane starts jerking right away. Visually it looks like stutters but actually FlightGear itself works fine. It feels like the fdm_callback updates the entire model with a delay, and it becomes noticeable over time.
My code:
from flightgear_python.fg_if import FDMConnection

def fdm_callback(fdm_data, event_pipe):
    return fdm_data

if __name__ == '__main__':
    fdm_conn = FDMConnection(fdm_version=24)
    fdm_event_pipe = fdm_conn.connect_rx('localhost', 5501, fdm_callback)
    fdm_conn.connect_tx('localhost', 5502)
    fdm_conn.start()

Video:
https://user-images.githubusercontent.com/43480503/224843125-698d3528-9ca4-4c34-acc2-bc6fe7569bb6.mp4

Fixed by #5 and #6

Yeah that's expected when you didn't set --fdm=null in FG. The Flight Dynamics Model (FDM) interface is intended to replace the internal flight model. If you don't set --fdm=null you'll have two FDMs trying to set the aircraft state: the internal FDM and your custom Python FDM. More info here: https://wiki.flightgear.org/Flight_Dynamics_Model
Now even if you set --fdm=null there might still be some jerkyness in FG, I think this might be a bug in FG? I'm still investigating it and seeing if I can come up with a workaround on the Python end. It's got something to do with the order data is set and sent within FG.

How is it possible to simply take data from FlighGear such as elevation, pitch, heading, roll, vertical speed? And use them in PID to get elevator signal without replacing the whole model. I want essentially the same behavior as with TELNET get_prop() and set_prop(), but only much faster to reduce "dt" in pid and response time.

I've added an example: simple_wing_level.py that shows how that could work

I thought about just not doing "return" in fdm_callback but didn't have time to check it. Thanks a lot for all this.


When using the S3 loader, it has the wrong parameters for the Unstructured API

I was trying to use the S3 loader and it calls the Unstructured API. It seems it crashes the service when using it because of the strategy set to hi-res. When calling the API according to the Unstructured example, it seems that by default, the value for strategy is empty. When I made a copy of your S3 loader and changes the UnstructuredLoaderOptions, it worked. These are the values of the options that I changed in order to match what it has in the Unstructured API example and it worked:
      const options: UnstructuredLoaderOptions = {         apiUrl: this.unstructuredAPIURL,         apiKey: this.unstructuredAPIKey,         coordinates: true,         strategy: "auto"       };

Hello, you can close this issue. I honestly didn't like using the Unstructured API because it broke down my PDFs into one document per line. So I made a modification to it so that it uses your own PDF loader. I also made different versions for text loaders and audio loaders. I suggest that maybe you should allow the developer using your S3 Loader to allow a person to pass in their own document loaders to parse the type of file they want to parse. I basically made a copy of your code in my own project with modifications, which I guess the MIT license allows. Do I need to do anything else in order to use it as my own code? Like put a comment on the top saying "This is a copy of the Langchain.js code"?


[FEATURE] Add dependabot support for SDK

Is your feature request related to a problem?
Add github dependabot support which raises PRs when security issues are found within code.
What solution would you like?
Write up dependabot.yml and configure it for the repo[1].
[1] https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuring-dependabot-version-updates

I recently replaced Dependabot on my repo with Renovate.  I consider it much better of a product.  Some advantages:

If you want it to act just like Dependabot you can, but you can even customize time of day of the PRs, or just let them come whenever detected.
You can group/batch version bumps together. I have done this to batch my build plugins (to get a single PR instead of 3 for ones that release together, for example) to reduce the PR noise.
It creates a "Dashboard" issue that shows you any pending bumps or ones you've intentionally suppressed.  See example here.
It has the ability to auto-merge its own PRs, which I've enabled for minor and patch versions of build plugins.  We probably don't want that feature here, but it's available :)

Unless there's a reason we are tied to the same system as OpenSearch, I'd consider it.

Some bloggers like Renovate:

Software Dependency Management: From Dependabot to Renovate
Automate Dependency Updates With Renovate, Not With Dependabot

Other reasons Renovate is awesome (and no I'm not a paid shill):

The company behind it (Mend) is a rebrand from WhiteSource which we already use/trust in our CI
Dependabot works great for Github.  Because it's developed by GitHub.  As an independent product, Renovate supports many more platforms than GitHub, which may or may not be relevant in our future if we have more Docker integration, etc.

Thanks @dbwiddis. I didnt know Mend Renovate is really better. I really like that.
I am happy as long as dependencies are being upgraded : )

This needs to be initially setup by an owner of opensearch-project. How do we request that?


Update jsoniter-scala-core, ... to 2.9.0

Updates

com.github.plokhotnyuk.jsoniter-scala:jsoniter-scala-core
com.github.plokhotnyuk.jsoniter-scala:jsoniter-scala-macros

from 2.7.3 to 2.9.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (2.7.3).
You might want to review and update them manually.
docs/website/package-lock.json

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.github.plokhotnyuk.jsoniter-scala" } ]

labels: library-update, semver-minor, old-version-remains

Superseded by #844.


Level 1 optimization (default) and fonts with color keywords

I'm trying to use the Google font "Crimson Text", however with the default optimization level the "Crimson" text is being replaced with its hex color equivalent "#dc143c" via the shorten-hex.js. Any thoughts on how to fix?

@nsciacca Please open up an issue over at clean-css - this project is just a small wrapper around the core lib. Anything regarding rules and output discrepancies surely need to be discussed there.


Main v1beta1 v2

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: AmitSahastra
To complete the pull request process, please assign jzhoucliqr
You can assign the PR to them by writing /assign @jzhoucliqr in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@AmitSahastra: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

cluster-api-provider-maas_build
f2e8ae25965e306a087aff8a62962ec9c02e8d30
link
/test cluster-api-provider-maas_build

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@AmitSahastra: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

cluster-api-provider-maas_build
f2e8ae25965e306a087aff8a62962ec9c02e8d30
link
/test cluster-api-provider-maas_build

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: AmitSahastra, sadysnaat
To complete the pull request process, please assign jzhoucliqr
You can assign the PR to them by writing /assign @jzhoucliqr in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


suggested minimum instance count value should be 3

we know that even configured minimum instance as 0, there are always 2 instance running internally. Thus there is no such difference configured minimum instance count from 0 to 2. So the downtime of scaling out “from 0 to 5” and “from 2 to 5” will be the same. I suppose that we should edit this part minimum instance count of at least 3 instead.

@maotsuk : Thanks for your contribution! The author(s) have been notified to review your proposed change.

This change has to be consistent if ever made – for example, on portal, our validations, etc.
Also, this change as minimum 3 will push customers to incur 30 CUs as mandatory in the bill, which we don’t want to do now. What we can do is mention in our documentation that we’ll provision two instances at any given time for HA and customers can choose a min of 1 or 2 to get the guaranteed 10 or 20 CUs that they’d need or they can choose 0, if they don’t need any guarantee in terms of CU. We have a task internally to get this done, this can be closed.

@surajmb I merged instead of closing, out of habit. I'll create a PR to overwrite this one right now. I'll post the link here.

Thanks @ktoliver

I did a straight copy-and-paste and merged the PR: https://github.com/MicrosoftDocs/azure-docs/pull/64480

I believe that we talk and agreed with deleting the below sentence since  there is no difference for the down time between 0, 1 or 2 when creating a new instance. COuld you please take a look again? @surajmb @ktoliver
--
However, creating a new instance can take some time (around six or seven minutes). Hence, if you do not want to cope with this downtime you can configure a minimum instance count of 2, ideally with Availability Zone support.


Release 0.1.9

Fixes for previous botched releases

Had tagged 0.1.8, but missed these changes. Fixing by bumping past it
Also missed that pynvjitlink/VERSION has a hard-coded version in 0.1.7. So updating it now

Would recommend we simplify/automate this release process

Thanks Bradley and Brandon! 🙏


Diff threshold parameter added.

Diff threshold defines minimal diff value to report. Rendered diff
type can now handle ratio_diffs flag properly. Only PNG files
that differ remain in render directory after render diff operation.

@dougfelt : could you review it?

LGTM, except that I think we should combine gpos_bound and diff_threshold into a single argument called diff_threshold.

Looks like this is also waiting on the CLA?

@iverg, could you incorporate the changes I suggested?

I'll just do it. Anyways, I don't think we have to worry about the CLA since Igor's a googlei18n member.


Add mc-sgx-core-build crate

Add the mc-sgx-core-build crate to hold common build utilities used
with other SGX crates.

Graphite Merge Job
Current status: 🔄 Waiting for checks to pass
This pull request is currently waiting for CI and mergeability checks to pass before being merged.
This comment was auto-generated by Graphite.

Job Reference: QB8STOU63qtQFHJyhfrd


A way to handle complex grid contexts

It can be difficult and confusing to work with multiple nested context when the site gets big, especially if we are working with asymmetric grids.
We need to use the nested() mixin to create contexts, resulting in code like
$susy: (
  columns: 1 2 3 4 5
);

.parent {
  @include span(3 at 2);
}

// Complicated context retrieving takes place here
.child {
  @include nested(3 of map-get($large, columns) at 2) {
    @include span(2 last);
  }
}

I thought of a way to store the context and retrieve it whenever needed
The code above will become:
.parent {
  // Stores nested(3 at 2) as "parent" in a $contexts map
  @include span-ac(3 at 2, parent);
}

.child {
  // retrieves "parent" context from $contexts map
  @include with-context(parent) {
    @include span(2 last);
  }
}

Optionally, you can also set flags for different breakpoints:
.parent {
  // Stores nested(3 at 2) as "parent: (large: <context>)" in a $contexts map
  @include span-ac(3 at 2, parent, large);
}

.child {
  // retrieves "parent: (large)" context from $contexts map
  @include with-context(parent, large) {
    @include span(2 last);
  }
}

Here's a Demo on Sassmeister:
Play with this gist on SassMeister.
Its working pretty well for me right now. Will anyone be interested in this?

That's an interesting idea. I'm not opposed to something like that, but is it really needed?
 You can already pass the entire $large map as context, without getting just the columns:
  @include nested(3 of $large at 2) {
    @include span(2 last);
  }

The advantage of your solution is that it allows you to create any new named context on the fly. That is a pretty slick feature. span currently takes a single argument, so not hard to add a second one to the existing mixin if we want to go that route. We could also add a keyword like "as"?
I wonder if this feature could serve double-purpose?
.initial {
  @include span(last 2 of (1 2 4 6 3 1) as sidebar-right);
}

// Nested Context
.nested {
 @include span(1 of sidebar-right);
}

// Similar Span
.similar {
  @include span(sidebar-right);
}

The beauty of this is that you can make it work with deep nested contexts as well, which means you don't have to remember and calculate the context at a later stage. It's going to be helpful if we're working with asymmetric grids since that's more complex.
What you suggested is cool, "as" sounds like a great keyword to use for this feature, and I hadn't thought of the .similar before. It might be very useful indeed.
How about handling contexts at different media queries then? Do we leave it to the users?
.initial {
  @include span(last 2 of (1 2 4 6 3 1) as sidebar-right);

  @include breakpoint(600px) {
    @include span(last 2 of (1 3 3 4 4 4) as sidebar-right-large);
  }
}

.nested {
  @includen nested(sidebar-right) {
    @include span(1 of sidebar-right);
  }

  // This currently does not work with asymmetric grids because it uses with-layout()
  @include susy-breakpoint(600px, sidebar-right-large) {
    @include span(1);
  }

  // This might work with the current code
  @include breakpoint(600px) {
    @include nested(sidebar-right-large) {
       @include span(1)
     }
  }
}


Chapter attention mechanisms/attention cues

Job d2l-zh/PR-818/1 is complete.
Check the results at http://preview.d2l.ai/d2l-zh/PR-818/


Intermittently missing Datapoints

Hi,
We've created a downsample Task and it fails intermittently with error
'Error exhausting result iterator; Err: runtime error @18:6-19:146: reduce: failed to evaluate reduce function: cannot convert argument of type invalid to float: runtime error @18:6-19:146: reduce: failed to evaluate reduce function: cannot convert argument of type invalid to float'.

Below is our Task code snippet:
calculateAttPerHour = (column, tables=<-, outputField="att") =>
	(tables
	  |> reduce(identity: {tc: 0.0, tt: 0.0, attph: 0.0}, fn: (r, accumulator) =>
               ({
                  tc: accumulator.tc + r.c,
                  tt: accumulator.tt + r.att * r.c,
                  attph: math.floor(x: (accumulator.tt + r.att * r.c) / (accumulator.tc + r.c))
                })
           )
       )

attph = from(bucket: "<bucket>")
	|> range(start: -1h)
	|> pivot(rowKey: ["_start", "_stop", "_time", "u", "f"], columnKey: ["_field"], valueColumn: "_value")
	|> aggregateWindow(every: 1h, fn: calculateAttPerHour, createEmpty: false)

When we checked the data for the failure, it's happening when datapoint for either att or c field is missing.
We use line protocol to write data to InfluxDB as below sample format 'measurement,u=<_userId>,f=<_flowId> c=230,att=478 timeInMilliseconds'. So datapoints for both fields 'c' and 'att' should present or both should miss. But with the task error and with our findings, datapoint for only one of the field(c or att) is missing.
Could you please let us know what could be the reason that some datapoints are missing intermittently.
Expected behavior:
Task should complete successfully.
Actual behavior:
Task is failing intermittently.

InfluxDB version: Cloud 2.0

Hi @Bharathebt.
I'll be taking a look at the problem you are running into. If I may ask, are you still experiencing this problem and at what interval are points written in? Are the points written at random times (like events) or at a regular interval like every 10 seconds? For the row where a value is missing, is it the most recent or at a random time in the middle of the interval? Which cloud provider and region are you using?
To clarify something with this statement:

We use line protocol to write data to InfluxDB as below sample format 'measurement,u=<_userId>,f=<_flowId> c=230,att=478 timeInMilliseconds'. So datapoints for both fields 'c' and 'att' should present or both should miss. But with the task error and with our findings, datapoint for only one of the field(c or att) is missing.

While line protocol will have these on the same line, these are still considered separate points by influxdb. After consulting with a member of the storage team, they told me that when /write returns it guarantees that the point will be persisted, but there can still be a small section of time when the point isn't yet readable or one point is readable and the other not yet. With some more information about the write pattern, we might be able to modify the flux script or find a way to make the task more resilient to this write/read lag.

Hi @Bharathebt.
I'll be taking a look at the problem you are running into. If I may ask, are you still experiencing this problem and at what interval are points written in? Are the points written at random times (like events) or at a regular interval like every 10 seconds? For the row where a value is missing, is it the most recent or at a random time in the middle of the interval? Which cloud provider and region are you using?
To clarify something with this statement:

We use line protocol to write data to InfluxDB as below sample format 'measurement,u=<_userId>,f=<_flowId> c=230,att=478 timeInMilliseconds'. So datapoints for both fields 'c' and 'att' should present or both should miss. But with the task error and with our findings, datapoint for only one of the field(c or att) is missing.

While line protocol will have these on the same line, these are still considered separate points by influxdb. After consulting with a member of the storage team, they told me that when /write returns it guarantees that the point will be persisted, but there can still be a small section of time when the point isn't yet readable or one point is readable and the other not yet. With some more information about the write pattern, we might be able to modify the flux script or find a way to make the task more resilient to this write/read lag.

Hi @jsternberg, Thanks for responding. Below are my answers.
Are you still experiencing this problem and at what interval are points written in?
-- Yes. We have multiple processes which consume events and post them to InfluxDB every 30s.
But the processes don’t follow the same time that is process 1's 30s window may differ from process 2's 30s window. So we can consider it as random.
For the row where a value is missing, is it the most recent or at a random time in the middle of the interval?
-- Its somewhere in the middle of the interval. The downsample task which we have works on a 2 hour old data.
Which cloud provider and region are you using?
-- For the InfluxDB cloud environment which we use the cloud provider is AWS and region is use-west-2.

Hi @jsternberg, Thanks for responding. Below are my answers.
Are you still experiencing this problem and at what interval are points written in?
-- Yes. We have multiple processes which consume events and post them to InfluxDB every 30s.
But the processes don’t follow the same time that is process 1's 30s window may differ from process 2's 30s window. So we can consider it as random.
For the row where a value is missing, is it the most recent or at a random time in the middle of the interval?
-- Its somewhere in the middle of the interval. The downsample task which we have works on a 2 hour old data.
Which cloud provider and region are you using?
-- For the InfluxDB cloud environment which we use the cloud provider is AWS and region is use-west-2.

Hi @Bharathebt in parallel to @jsternberg's questions could you also provide me with your org_id and bucket_id for the bucket in question.  Rather than post those here you can email them to me<EMAIL_ADDRESS>We will run a tool to help us identify if there is a difference between the replicas of the data in your bucket and why that might be.

Hi @Bharathebt in parallel to @jsternberg's questions could you also provide me with your org_id and bucket_id for the bucket in question.  Rather than post those here you can email them to me<EMAIL_ADDRESS>We will run a tool to help us identify if there is a difference between the replicas of the data in your bucket and why that might be.

@Bharathebt - a few more question please:
You said that the downsample task runs on 2 hour old data. The snippet you shared of the task shows range(start: -1h), which is operating on the recent hour of data. Are you using the most recent hour or are you doing range(start: -2h, stop: -1h)?
How often is your task scheduled to run? Hourly?
If you are using range(start: -1h), could you reconfigure your task to use the offset option of about 5 minutes and adjust the range in the task to use task.every? These settings will run your task with an offset delay but keep the input data on the hour boundaries. This will avoid the readability delay that @jsternberg explained. The documentation for these settings is here: https://docs.influxdata.com/influxdb/v2.0/process-data/get-started/#define-a-data-source
If you are still seeing the problem, please share the complete task (remove or change private information) instead of a snippet.

@Bharathebt - a few more question please:
You said that the downsample task runs on 2 hour old data. The snippet you shared of the task shows range(start: -1h), which is operating on the recent hour of data. Are you using the most recent hour or are you doing range(start: -2h, stop: -1h)?
How often is your task scheduled to run? Hourly?
If you are using range(start: -1h), could you reconfigure your task to use the offset option of about 5 minutes and adjust the range in the task to use task.every? These settings will run your task with an offset delay but keep the input data on the hour boundaries. This will avoid the readability delay that @jsternberg explained. The documentation for these settings is here: https://docs.influxdata.com/influxdb/v2.0/process-data/get-started/#define-a-data-source
If you are still seeing the problem, please share the complete task (remove or change private information) instead of a snippet.

Hi @philjb We already have offset of 2h in our task code and it scheduled to run every 1 hr and querying the data for last 1hr, which is task will pick the data between the last 3hr data and last 2hr data..
option task = {
name: "flowEvents_downSample_DO_NOT_RUN_MANUALLY",
cron: "0 * * * *",
retry: 3,
offset: 2h,
}
And we did some more debugging found some issues, first we wrote a query to identify the missing datapoints (i.e either field 'c' or 'att' is missing)
Query1:
from(bucket: "flowEvents")
|> range(start: -4d, stop: -1d)
|> filter(fn: (r) => (r._measurement == "s"))
|> pivot(rowKey: ["_start", "_stop", "_time", "u", "f", "ei"], columnKey: ["_field"], valueColumn: "_value")
|> filter(fn: (r) => ((exists r.c and not exists r.att) or (exists r.att and not exists r.c )))
Here you can see, for few rows values for 'c' field is present but 'att' field is missing and sometimes 'c' is present and 'att' is missing (this should never happen as we write both c and att together to influx in one call as "measurement,u=,f=,ei= c=230,att=478 timeInMilliseconds").

We went ahead and queried the data for that particular time range(2021-01-15T01:48:00.000Z) to check whether data is really missing.
Query2:
from(bucket: "flowEvents")
|> range(start: -4d, stop: -1d)
|> filter(fn: (r) => (r._measurement == "s"))
|> filter(fn: (r) => (r.ei == "5e2a09ac9b9a4925f2f27ef3"))
|> filter(fn: (r) => (r._time == 2021-01-15T01:48:00.000Z))
The query results were inconsistent i.e. sometimes it's returning only 'att' and sometimes it's returning both 'att' and 'c'.

Hence, it seems that the task is failing due to inconsistent query results and not due to missing data. Please verify if this is the reason and check further but from the debugging that we could do, it seems the query results are not consistent.
@8none1 With the above results, do you still require bucket_id and org_id, if yes I can mail you.

Hi @philjb We already have offset of 2h in our task code and it scheduled to run every 1 hr and querying the data for last 1hr, which is task will pick the data between the last 3hr data and last 2hr data..
option task = {
name: "flowEvents_downSample_DO_NOT_RUN_MANUALLY",
cron: "0 * * * *",
retry: 3,
offset: 2h,
}
And we did some more debugging found some issues, first we wrote a query to identify the missing datapoints (i.e either field 'c' or 'att' is missing)
Query1:
from(bucket: "flowEvents")
|> range(start: -4d, stop: -1d)
|> filter(fn: (r) => (r._measurement == "s"))
|> pivot(rowKey: ["_start", "_stop", "_time", "u", "f", "ei"], columnKey: ["_field"], valueColumn: "_value")
|> filter(fn: (r) => ((exists r.c and not exists r.att) or (exists r.att and not exists r.c )))
Here you can see, for few rows values for 'c' field is present but 'att' field is missing and sometimes 'c' is present and 'att' is missing (this should never happen as we write both c and att together to influx in one call as "measurement,u=,f=,ei= c=230,att=478 timeInMilliseconds").

We went ahead and queried the data for that particular time range(2021-01-15T01:48:00.000Z) to check whether data is really missing.
Query2:
from(bucket: "flowEvents")
|> range(start: -4d, stop: -1d)
|> filter(fn: (r) => (r._measurement == "s"))
|> filter(fn: (r) => (r.ei == "5e2a09ac9b9a4925f2f27ef3"))
|> filter(fn: (r) => (r._time == 2021-01-15T01:48:00.000Z))
The query results were inconsistent i.e. sometimes it's returning only 'att' and sometimes it's returning both 'att' and 'c'.

Hence, it seems that the task is failing due to inconsistent query results and not due to missing data. Please verify if this is the reason and check further but from the debugging that we could do, it seems the query results are not consistent.
@8none1 With the above results, do you still require bucket_id and org_id, if yes I can mail you.

@Bharathebt - yes please, email me the bucket_id and your org id.

@Bharathebt - yes please, email me the bucket_id and your org id.

@Bharathebt - thank you for providing the information.  We've set up an account and are running tasks like yours to try and recreate the issue so that we can get this fixed.  Please bear with us.

@Bharathebt - thank you for providing the information.  We've set up an account and are running tasks like yours to try and recreate the issue so that we can get this fixed.  Please bear with us.

Hi @Bharathebt - we think that this should be resolved now.  Please could you rerun your queries and confirm that you are seeing consistent results now?

Hi @Bharathebt - we think that this should be resolved now.  Please could you rerun your queries and confirm that you are seeing consistent results now?

Hi @8none1 We've run the query and it looks like the issue is still happening.
We still get the datapoints ((c exists and att not exists) or (c not exists and att exists)) when we run Query1. The timestamp of those datapoints are of 24th and 25th date.

Below are the Query2 results:

Hi @8none1 We've run the query and it looks like the issue is still happening.
We still get the datapoints ((c exists and att not exists) or (c not exists and att exists)) when we run Query1. The timestamp of those datapoints are of 24th and 25th date.

Below are the Query2 results:

This is now fixed in InfluxDB and is rolling out to production now.
Fixed via PR: https://github.com/influxdata/idpe/pull/9743#event-4420553541


fixing docker run display commands

Fixing issue https://github.com/nasa/astrobee/issues/513 for bionic
Seems like the display settings were not being parsed properly
This bug was introduced in https://github.com/nasa/astrobee/pull/458

Looks great! (This kind of thing is why I hate bash scripting...)

My dream is for Github to be able to run gazebo inside a docker image such that we can run the integration tests on PRs :)


Update web interface

Update from bootstrap 3 to 4 by switching from unmaintained flask-bootstrap to bootstrap-flask (confusing, I know)
Add confidence bars (with color!) to interface
Right align line numbers

Very nice!


REQUIRE Heisenbug

I'm getting a particularly nasty Heisenbug under Catch 1.3.1, and it seems to relate to REQUIRE. Updating to 1.10 fixes it, but because it's a Heisenbug, I can't be sure if it is really fixed or not. I've read the changelog and I can't find anything that looks like a fix for this... so I'd be grateful if you could let me know if there are any known issues here.

It causes SIGABRT, SIGSEGV, SIGILL or bad_alloc, all of which suggest memory corruption.  It is reproducible in the sense that running exactly the same code produces exactly the same results, but any of the following can stop the bug from happening (or change which error occurs):
-- Removing source files containing tests that are not being run
-- moving the (inline) function in question from one file to another
-- copying the function, giving it a fresh name and calling the copy
This makes it hard to give a self-contained minimal example, but the problem definitely occurs in this code:
REQUIRE(true == true);
REQUIRE(879.9 == Approx(876.5).epsilon(0.0000001));

Both REQUIREs are needed to trigger the bug.
(The actual numbers are arbitrary.) It's only reproducible using clang on release mode; debuggers, even using release build with deb. symbols, don't find it.  I'm testing on Windows 10/Cygwin 2.8.2.
Edit: Could this issue relate to the lifetime of the Approx object? I don't know how REQUIRE is expanded, but is is possible that the Approx is not being kept alive for long enough?

FWIW, this is the actual output:
/home/Mohan/cpp/tests/engine/../Match.h:41: FAILED:
  REQUIRE( 879.9 == Approx(876.5).epsilon(0.0000001) )
with expansion:
  879.9 == Approx( 876.5 )

terminate called without an active exception
/home/Mohan/cpp/tests/engine/EngineTest.cpp:19: FAILED:
  {Unknown expression after the reported line}
due to a fatal error condition:
  SIGABRT - Abort (abnormal termination) signal

I am using an inline header file which is included by multiple tests, so there is also a possibility that the line number reporting code is playing up?

If the lifetime of the Approx object was incorrect, there would be a lot more reports.
Have you tried another platform? Maybe Cygwin's illusion of POSIX is not good enough, or its clang ran into a corner case.

Can't reproduce unter Ubuntu LTS, but it's a Heisenbug so it may still be there.
Have reproduced under Catch 1.10.0; error is
11852 Illegal instruction     (core dumped) .

So if I understand this correctly, either Clang is miscompiling, or we have some edge-case UB that we run into (Clang likes turning undefined behaviour into SIGILLs).
What version of Clang and what settings are you reproducing this with? Is it with multiple versions of Clang or only specific one?

$ clang --version
clang version 4.0.1 (tags/RELEASE_401/final)
Target: x86_64-unknown-windows-cygnus
Thread model: posix
InstalledDir: /usr/bin
Release build with
-std=c++11 -Wall -Wextra -Wno-missing-braces -pedantic -Wnarrowing -Wshorten-64-to-32
I haven't tried another version of clang yet -- switching is very fiddly under Cygwin, unfortunately.

I'n now getting a similar crash on Ubuntu LTS. Trying to narrow it down.

Stack trace:
*** Error in `./tests': free(): invalid size: 0x00000000013dabc0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fe9796fe7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fe97970737a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fe97970b53c]
./tests(_ZN5Catch15ConsoleReporterD2Ev+0x42)[0x516872]
./tests(_ZN5Catch15ConsoleReporterD0Ev+0x9)[0x5169b9]
./tests(_ZN5Catch8runTestsERKNS_3PtrINS_6ConfigEEE+0x886)[0x50b2b6]
./tests(_ZN5Catch7Session11runInternalEv+0x202)[0x535162]
./tests(_ZN5Catch7Session3runEv+0x8a)[0x52fc0a]
./tests(Z9run_catchiPPKcNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_S7+0x70)[0x5171c0]
./tests(main+0x2fe)[0x45f98e]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fe9796a7830]
./tests(_start+0x29)[0x45f5b9]

Run under valgrind?


More devops-style WASM transform deployment

Currently the wasm artefacts are built locally and copied into a redpanda container for deployment.
AFAIK, they can be OCI compliantly pushed/tagged/published as artefacts
Can the transforms be deployed in a more efficient and elegant manner?

So, as far as Ive learnt, we need to host the wasm binaries in some public location (like S3/CDN etc) where they can be pulled from during the "rpk transform deploy --file=https://my-website.com/my-file.wasm" .
Since, I ve pushed them (for now) into git directly. I m going to wait for people's feedback.


Change order of arguments to set-attr

The current definition of the set-attr command is:
set-attr [options] <ext-archive> <file-in-archive> [attrs...]
The optional attrs argument is a space-separated list of attributes, e.g. type=BIN aux=0x2000.  If the attribute list is omitted, the command behaves like get-attr instead (there is no get-attr command).  There are a couple of problems with this:

The pattern diverges from commands like UNIX chmod, which take a list of filenames.
The pattern diverges from other cp2 commands, like export [options] <ext-archive> <export-spec> [file-in-archive...], which put a "specification" between the ext-archive argument and the filename list.
Getting attributes with set-attr feels a little weird.

Furthermore, the current argument order prevents the command from allowing the user to mark a big pile of files read-only with a statement like cp2 set-attr mydisk.po locked file1 file2 file3.
Four changes should be made:

The order of arguments should be changed, so that the attributes come before the filename.
The attributes should be a comma-separated list, rather than space-separated.  This is for convenience when setting multiple attributes.
The command should take multiple filenames, with the usual wildcard support.
The no-attribute form of the command no longer makes sense, so we need a get-attr command.

Switching from space-separated to comma-separated shouldn't cause problems.  The only opportunity for commas to appear naturally is in the Mac creator/file type strings, but that should be rare, and those allow hex digits to be provided.
This should improve the function of the command, and allow the behavior requested in issue #27.
The command should only descend into subdirectories if --recurse is given on the command line. It's reasonable to change the attributes of a directory.
The get-attr command is fundamentally a single-file command.  The argument order should be the same as the old set-attr, so that we can take an attribute name as an optional argument:
get-attr [options] <ext-archive> <file-in-archive> [attr]
The idea is that get-attr file dumps the full list of attributes as human-readable name/value pairs, while e.g. get-attr file type just prints the value in a machine-readable sort of way.  This could be useful for shell scripts.
Sample of current output:
Attributes for 'Pics.3200/ROSE.3200':
  File Type   : PNT $c0
  Aux Type    : $8005
  HFS Type    : '␀␀␀␀' $00000000
  HFS Creator : '␀␀␀␀' $00000000
  Access      : $e3 [dnbwr]
  Create Date : 24-Sep-1991 01:12:00
  Mod Date    : 24-Sep-1991 01:12:00
  Comment     : (none)

Available in https://github.com/fadden/CiderPress2/releases/tag/v1.0.4-dev2


Supporting AWS proxy protocol

It would appear that in order to support proxy protocol integration with AWS using Ambassador one would need to create a Module configuration which will enable it in the Ambassador control plane:
apiVersion: getambassador.io/v2
kind:  Module
metadata:
  name: ambassador
  namespace: ambassador
spec:
  config:
    use_proxy_proto: true

And decorate the service with relevant annotations, this for classic ELB:
apiVersion: v1
kind: Service
metadata:
  name: ambassador
  namespace: kubeaddons
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"

This all according to my review of your documentation:

https://www.getambassador.io/docs/latest/topics/running/ambassador/#allow-proxy-protocol-use_proxy_proto
https://www.getambassador.io/docs/pre-release/topics/running/ambassador-with-aws/#ambassador-edge-stack-on-aws

There may be more to it than that (I'll need to continue to review the documentation) but the purpose of this issue is specific to the helm chart:
I haven't seen any reference to the use of proxy_proto throughout the chart except for a single reference to a configuration override, so it seems you can do this by providing annotation overrides and configuration overrides in the values, but I don't see a higher level configuration available to enable proxy protocol on specific cloud providers.
The purpose of this issue is to request a high level configuration option for the chart to deploy a proxy_protocol ready and enabled service configuration, preferably something like this:
$ helm install --create-namespace --namespace ambassador --set cloudproviders.aws.proxyprotocol.enabled=true ambassador ambassador

Thank you!

@shaneutt , we've made updates since this issue and confirm that proxy_proto works with AWS.  Please see this doc for reference. But please reopen the issue if you find it persists on 2.x.


SceneTransitioner instantiation

fixes #4550

Thanks @fredj.  It was probably leftover code from a previous refactoring.


Create a central workflow for "main" branch

Summary

A main.yml needs to be created under .github/workflows/ to include future linter checks and testing checks for the main branch (or any other dev branches)

Summary

Created other PR workflows (GitHub Actions)

Create Branch from Issue
Issue Autolink
release-please

Estimated Work Time (in hrs)

12 hrs

Related commits:

feat: test linter automation
fix: test 2 for github actions
fix: remove working_directory field
fix: fix issues with yarn
fix: test lint automation
fix: test 2 lint automation
fix: test 3 automation
fix: test 4 lint
fix: stash previous changes
fix: test 5 lint automation
fix: test 5 lint auto
fix: test 6 lint auto
lint test 7
lint test 8
lint test 9
lint test 9
remove husky from package

Notes
After troubleshooting for 3-4 days and with the help of peers and colleagues, I was finally able to get these workflows working successfully.


mindustry: 126.2 → 139

Description of changes

Bump mindustry version to current.
Get back to using the latest Java and gradle versions (as enabled and required by https://github.com/Anuken/Mindustry/issues/5114 )
Eliminate more (maybe the last of?) uses of binaries-not-built-here.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

FYI
Mindustry maintainer @fgaz
Author of #108274 that fixed Arc to build from source (thanks!) @ghost
Other folks that have done mindustry version bumps @maxhbr @Mindavi

Also it looks like 139 is still a beta version, while 126.2 is the latest stable. Do you think we should upgrade anyway?

Oh, you're right.  It looks like the next stable release is scheduled for 2022-11-11, 13 days from now.
It looks like the upstream project's guidance on using the beta (at least on Steam) is to require users to opt-into it, so maybe we shouldn't update the main "mindustry" package to a beta version.  Also, it seems like unnecessary churn to introduce a top-level "mindustry-beta" or "mindustry7" package just to remove it or alias it in two weeks when 7 is stabilized.
I'll hold off on this.  It was good to have figured out how to update the build process to account for the changes since 126.2.  There probably won't be any more significant changes to the build process between now and then.
I'll plan to send another PR in two weeks with the actual 7.0 stable release.  I'll leave the chkno:mindustry-bump branch here for anyone that wants to try the beta.
Thanks!

It's out! https://github.com/Anuken/Mindustry/releases/tag/v140.3


Clickable=none should be added

Description
Any existing property "Clickable" I like to have an additional "none" option. This is essential to use e.g. a ui-switch as indicator.

Yes, there is a enabled  option already. However, that behavior is differed as it not just disables control but also switches style to disabled/not available (grayed out).
You maybe consider this while fixing issue 662

@Subnum12
So that way you get a switch that will look like a normal switch, but you can't click it.  Personally I would find that very confusing.  But on the other hand I would never use that "None" option myself, so it would not bother me either that this new option is available there for users like you that like to have it that way.
I will try to find some time this evening to create a PR.

Would not something like the LED indicator be better for this?

Clickable = None
You may use everything that the dashboard node "switch" already can do
AND you can use another icon, so it looks not like a switch but like a "sign"
vs.
LED
You CANNOT use another icon, so you have only the LED itself.
(Or someone has to pimp up the node "LED", so that custom icons are possible like in node "switch")

I would appreciate the option "Clickable = None" in Dashboard 2.0 nodes, where applicable

The option is not making the config screen more complex, because it is hidden in a dropdown.  And even if you open the dropdown, it is an extra option that makes sense between the other available options.  So I don't think it will be of any harm to other users, which won't be using this new option.

An alternative, which would be more general, might be a new node type that displays one of a configurable set of icons, where the icon to display is determined by the incoming message.

1. Clickable = None
   You may use everything that the dashboard node "switch" already can do
   AND you can use another icon, so it looks not like a switch but like a "sign"

Actual you describe my exact use case. I use switches with material design icons  (e.g. door open/close, pump on/off).
advantages of "Clickable = None" would be

no impact on existing behavior/flows
less implementation work as will you have for a new node (less code required to be maintained)
a way more flexible as LED's.  Icons often do not require description (open/close door icons speaks for it self - LED likely not)
personalty I prefer just a few high flexible nodes instead having a lot for just a single use-case .

@colinl I guess with 2 options (0/1) you can have it with ui-switch requested feature implementation for more options you maybe use ui-button-group .Of course that would require request Clickable option to be added here as well. (I'm using ui-button-group with material design icons for air-condition mode heat/cool/dry/off )

Unfortunately I wasn't able to finalize my pull request, due to a CSS issue.  I have asked on Discourse for assistance.  Hopefully somebody jumps in to assist me...

use ui-button-group .Of course that would require request Clickable option to be added here as well

Personally I don't like that idea.  If you click on the label of a switch then you can toggle it.  But if you click on the label of a button-group (with mire than 2 buttons) then the behaviour will become confusing.

use ui-button-group .Of course that would require request Clickable option to be added here as well

Personally I don't like that idea. If you click on the label of a switch then you can toggle it. But if you click on the label of a button-group (with mire than 2 buttons) then the behaviour will become confusing.

Actually this FR is about indicator functionality "Clickable = None" so you not supposed to click anywhere.
Let me show you an example for previously mentioned air condition mode indicator:

However, using ui-button-group for control would be possible today already...


Browsersupport?

Which browsers does this lib support? I can't find anything in the readme?

Hi @ctaepper, thank you for the comment.
Modern browsers and IE 11 are supported.
If you find something wrong, please report that.

No reply came, and I close this issue.

thanks, will let you know, if something browser related comes up.

😄


Scrolling with ComboBox or AutoComplete disconnects popup

Scrolling with any popup open will fix the position of the popup on the screen and disconnect it from its anchor.
https://user-images.githubusercontent.com/40908164/159106851-a5c45b9c-760a-4fec-b41e-a51f4e5942ca.mov
Is there any way to fix this?

Thanks! The fix works great!


require path node module in README.md example code

in order to prevent this error when running the handler example code

:+1:


cannot import name 'modules' from 'nussl.deep'

Trying to import a version of nussl that uses deep learning (specifically the clustering branch) results in errors due to unspecified dependencies.
Reproduction:
> pip install git+https://github.com/interactiveaudiolab/nussl.git@clustering
...
> python -c 'import nussl'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/nathanshelly/projects/personal/sandbox/python_dependencies/virtualenv/env/lib/python3.7/site-packages/nussl/__init__.py", line 27, in <module>
    from .separation import *
  File "/Users/nathanshelly/projects/personal/sandbox/python_dependencies/virtualenv/env/lib/python3.7/site-packages/nussl/separation/__init__.py", line 55, in <module>
    from .clustering import SpatialClustering, PrimitiveClustering
  File "/Users/nathanshelly/projects/personal/sandbox/python_dependencies/virtualenv/env/lib/python3.7/site-packages/nussl/separation/clustering/__init__.py", line 1, in <module>
    from .clustering_algorithms import SpatialClustering, DeepClustering, PrimitiveClustering
  File "/Users/nathanshelly/projects/personal/sandbox/python_dependencies/virtualenv/env/lib/python3.7/site-packages/nussl/separation/clustering/clustering_algorithms.py", line 1, in <module>
    from .clustering_separation_base import ClusteringSeparationBase
  File "/Users/nathanshelly/projects/personal/sandbox/python_dependencies/virtualenv/env/lib/python3.7/site-packages/nussl/separation/clustering/clustering_separation_base.py", line 11, in <module>
    from ...deep import modules
ImportError: cannot import name 'modules' from 'nussl.deep' (/Users/nathanshelly/projects/personal/sandbox/python_dependencies/virtualenv/env/lib/python3.7/site-packages/nussl/deep/__init__.py)

This can be solved by also installing torch & zarr - pip install torch zarr.

Touches on a similar issue to #130. Potentially an opportunity to revisit that question.

Fixed in the latest refactor.


Lab 2 - Sorting

Fixes #181
Review of colleague's PR #176
Changes proposed in this PR:
3 sort algorithms - Lab02:

Implement BubbleSort
Implement Insertion
Implement QuickSort
Add Tests

Codecov Report

Merging #182 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #182   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           2      3    +1
  Lines          18     56   +38
=====================================
+ Hits           18     56   +38

Impacted Files
Coverage Δ

02_bubble/n0npax/main.go
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9605cdc...599aeba. Read the comment docs.

@juliaogris could you please review changes?

@juliaogris @rokane Could you review this PR please?


Fix Tween follow not working

Fix regression from 01ef7a73de46236c4b0cd0e7853f9d3c91768111
original code looks a little bit weird, but anyway
Variant result = _run_equation(data);
emit_signal("tween_step", object, data.key, data.elapsed, result);
_apply_tween_value(data, result);

this codes run on all kind of data.type but except INTER_CALLBACK
enum InterpolateType {
		INTER_PROPERTY,
		INTER_METHOD,
		FOLLOW_PROPERTY,
		FOLLOW_METHOD,
		TARGETING_PROPERTY,
		TARGETING_METHOD,
		INTER_CALLBACK,
	};

but after 01ef7a73de46236c4b0cd0e7853f9d3c91768111,
only _apply_tween_value is called only INTER_PROPERTY, INTER_METHOD cases
This PR make following works again, and call callback successfully.
here is test code.
func _ready():
	test_tween()

func test_tween():
	var tween = Tween.new()
	tween.interpolate_callback(self, 1, "dummy")
	add_child(tween)
	tween.start()
	tween.connect("tween_complete", self, "_on_tween_complete", [tween])

func dummy():
	print("called dummy")

func _on_tween_complete(obj, key, tween):
	print("_on_tween_complete", obj, key)
	tween.queue_free()

possibly fix what https://github.com/godotengine/godot/issues/8752#issuecomment-348679007 said

Please add [2.1] to PR title? Otherwise we see two identically titled PRs in the list...

you can see 2.1 milestone in the list but... no problem.


Blazor Hybrid vs PWA

Do we have a doc that covers the differences between using a blazor webassembly PWA and a Blazor Hybrid app. Could we have one? Here is some initial content for it.

/cc: @danroth27

We do have https://docs.microsoft.com/aspnet/core/blazor/hosting-models which addresses the pros/cons of the different Blazor hosting models including Blazor Hybrid vs Blazor WebAssembly. Instead of creating a new topic I prefer to update the existing one if needed.

The topic covers some/most of these points, but the table that we have doesn't split out PWAs for contrast with Hybrid. I think we should try to resolve this using the existing table first, so I'll see if I can add a column to the existing table and add rows for missing characteristics from the content here. If that doesn't work, then we might need a new table. Let's see on the PR how things compose.
https://docs.microsoft.com/aspnet/core/blazor/hosting-models#which-blazor-hosting-model-should-i-choose
Note to self: Table content in text ...

Hybrid
PWA

No need to host the content anywhere public, it is rendered directly from the app into a webview.
PWAs are essentially websites (you need to host the content on some server.

The static content is packages with the application in the appropriate installer (apk, msix, etc.).
They're installed from the browser (or from some stores) but there is no installer.

Same access to the platform as any other native app would have, directly from within C#.
Access to native capabilities is limited (whatever the browser offers) and via JS interop with C#.

The app is deployed and updated as any other native app for a given platform. Following the appropriate procedures for a given platform.
They're updated automatically when the site is updated (no updates via stores, no validation/verification process for updates).

Will run using the .NET runtime natively on the target platform, which offers the best possible speed.
Will run using the .NET runtime for WebAssembly, which is slower than running directly on the platform, even for apps ahead-of-time (AOT) compiled for WebAssembly in the broswer.


Coverage

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@5e5e083). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             master       #48   +/-   ##
==========================================
  Coverage          ?   100.00%
==========================================
  Files             ?         2
  Lines             ?         2
  Branches          ?         0
==========================================
  Hits              ?         2
  Misses            ?         0
  Partials          ?         0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e5e083...b64d2b2. Read the comment docs.


Applying custom style to paragraph using a role

I'm searching for a way to apply my custom style to a paragraph, I couldn't see anything in the https://github.com/asciidoctor/asciidoctor-pdf/blob/master/docs/theming-guide.adoc
I would like to define my style, eg.
mystyle:
font_size: 24
and apply it to my paragraph:, eg.
[.mystyle]
paragraph content
Is there a way to achive that in asciidoctor-pdf?

This is not really possible as of yet. My vision is to allow a role to be specified and then allow that role to be assigned properties in the theme. The theme part you cited is correct. However, you'd use a role not a block style in the AsciiDoc:
[.myStyle]
paragraph content

That sounds good.
So my understanding is that currently it is not possible to create my own style and apply it to some element. I could apply something only using extensions, right?

my understanding is that currently it is not possible to create my own style and apply it to some element

That is correct.
You'd need to extend the converter as described in the following blog post:
http://raibledesigns.com/rd/entry/re_customizing_an_asciidoctor_pdf
the updated source of which is available in the following repository:
https://github.com/mraible/infoq-mini-book/blob/master/src/main/ruby/asciidoctor-pdf-extensions.rb
Do keep in mind, though, that it may be necessary to refactor the custom code once we address #53.

We've now defined a place for roles in the theme (see https://github.com/asciidoctor/asciidoctor-pdf/blob/master/docs/theming-guide.adoc#keys-role). The next step would be to make use of them for blocks, like paragraphs.

I have figured out a way to support roles on paragraphs. Let's start there to make this capability available, then expand that support to other blocks in separate issues.


z.enum for as const numbers?

This works nicely:
const VALUES = ["Salmon", "Tuna", "Trout"] as const;
const FishEnum = z.enum(VALUES);
// 'Salmon' | 'Tuna' | 'Trout'

But this does not work:
const VALUES = [1, 2, 3] as const;
const OrderEnum = z.enum(VALUES);
// 1 | 2 | 3

This is due to the requirement of having an array of strings, although a union of numbers work.
Thoughts?

This would be very helpful to the problem I am currently facing at work, my data is represented as an array of numbers but because Zod is forcing to use string, creates ambiguity.
Is there a reason why it has to be an array of strings and that it can't numbers?
@colinhacks Would love your input!

@scotttrinh agreed that union could work but the use case is for an already existing array.

Enums should never be numbers. By default enum values in TypeScript are numbers, but this is a footgun and an antipattern.
enum Color { Red, Green };
Color.Red; // 0

If you add another color the actual under-the-hood value changes:
enum Color { Blue, Red, Green };
Color.Red; // 1

Which is the source of a lot of bugs and confusion. So Zod forces you to use strings.
This also lets you access the elements of the enum like so:
const Color = z.enum(["Red", "Green"]);
Color.enum.Red; // "Red"

Which is importany, and wouldn't be possible with numbers.
I recommend writing a generic function that accepts a list of numbers and exports a z.custom schema:
const numberEnum = <Num extends number, T extends Readonly<Num[]>>(
    args: T
  ): z.ZodSchema<T[number]> => {
    return z.custom<T[number]>((val: any) => args.includes(val));
  };

  const nums = [1, 2, 3] as const;
  const schema = numberEnum(nums);

Heads up, this is now broken with v3.9.3 @colinhacks

@mmahalwy
CodeSandbox
Am I setting this up incorrectly? It throws in my testing on 3.9.3 and 3.9.4 🤔

@scotttrinh reproduced it here: https://codesandbox.io/s/dreamy-dawn-2b79i?file=/src/index.ts
I think when you have a union it breaks:
import { z } from "zod";

const schema = z.custom((val: any) => {
  return false;
}, `Error please`);

const objSchema = z.union([
  z.object({
    schemaValidation: schema,
    shouldBeNull: z.null()
  }),
  z.object({
    schemaValidation: schema,
    shouldBeNumber: z.number()
  })
]);

const result = objSchema.safeParse({
  schemaValidation: 1,
  shouldBeNumber: 1
});
console.log(result);
console.log(!result.success && JSON.stringify(result.error, null, 2));

Ahh, I see. Others have reported issues with union lately, so I suspect that is the issue. Once @colinhacks has a chance to address, I'll keep an eye on this repro case.

@colinhacks this feature would increase the parity with the native TypeScript enum. Also would allow validation of a specific set of numbers.
I know it's possible to achieve the same validation using union, but the error is not as descriptive.
Numbers seem to be already supported, so it's a matter of adjusting the types:
https://github.com/colinhacks/zod/blob/ab75f4dad2a6738c019fe4edea6c8603c25ecb18/src/types.ts#L2940-L3012


Reopen Add first frame on Android #1788

Description
Would like to reopen - Add first frame on Android #1788.
This was fixed in v0.1.204 , but the issue started reoccurring from v 0.1.209.
Note : This Happens only in Android.
Version
0.1.209
Steps to reproduce
return (

);
Snack, code example, screenshot, or link to a repository
return (

);

I am using

I am using SkiaView

Got. Did you try with 0.1.212? I believe that this should have been fixed now.
Beware that SkiaView is likely to be deprecated in the future, we strongly recommend to use Canvas instead.

yes, tried with 212 , same issue .
Works well in 204-208.
Issue replicable in 209 -212

Please provide a small example of the regression and we will take a look at it.
We also strongly recommend to migrate away from SkiaView as we are not supporting this API officially and it may get deprecated in the future.

Closing it for now but please reopen it if you have more information about the issue and/or a reproducible example.
Please bare in mind that the SkiaView API is not officially supported and is likely to be deprecated in the near future.


pulseaudio: no sound on USB-speaker until reconnecting USB-speaker

Issue description
I'm using an external speaker with volume controls: The speaker is connected via 3.5mm jack, the volume controls of the speaker are connected via USB (Class "Audio", SubClass "Control Device").
With ALSA / without hardware.pulseaudio.enable = true; in configuration.nix, sound works well. Except in Firefox, which needs pulseaudio to play sound. So, to play sounds in Firefox, pulseaudio has to be enabled.
With hardware.pulseaudio.enable = true;, and disconnected USB-speaker, sound works, too.
But with connected USB-speaker, there's no sound at all. Only if I re-connect the speaker-volume-buttons to USB, sound gets enabled for the currently running audio stream. If I stop the audio stream (e.g. stop mpv) and start a new one or jump to a different position in running a Youtube-video in Firefox, the sound is gone again until I reconnect the volume-buttons to USB (again).
So, every time I start some audio/video or jump to a different location in a Youtube-video, I have to reconnect my USB-speaker to get some sound.
Steps to reproduce

use external speaker with USB-volume-buttons
play some audio -> everything works well
set hardware.pulseaudio.enable = true; in configuration.nix
nixos-rebuild switch
play any audio (e.g. via mpv or a Youtube video on firefox) -> no sound
reconnect the USB-speaker/USB-volumen-buttons -> sound
stop audio, start audio -> no sound
reconnect the USB-speaker/USB-volumen-buttons -> sound

Technical details

USB-volume-buttons: Class "Audio", SubClass "Control Device"
system: "x86_64-linux"
host os: Linux 4.14.102, NixOS, 18.09.2253.753f58d9a42 (Jellyfish)
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.0.4
channels(root): "nixos-18.09.2253.753f58d9a42"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Ok, I've found the solution:

Since version 11.0/11.1, pulseaudio prefers USB/bluetooth devices over internal sound cards.
(https://www.freedesktop.org/wiki/Software/PulseAudio/Notes/11.0/)

pulseaudio thinks that my external USB-volume-buttons are an audio output device, and erroneously  uses it as default output.

To test this, I would recommend to (temporarily) disable the card via pacmd or pavucontrol:

pacmd list-cards to get a list of all cards
pacmd set-card-profile alsa_card.usb-Elite_Silicon_USB_Audio_Device-00 off (or similar) to disable a card
or pavucontrol as GUI: Playback -> Built-in Audio Analog Stereo; or Configuration -> USB Audio Device -> Off

So, disable this "card" permanently, use configuration.nix:
hardware.pulseaudio.extraConfig = ''
    set-card-profile alsa_card.usb-Elite_Silicon_USB_Audio_Device-00 off
'';


Test Python versions 3.11 and 3.12

Description

The purpose of this PR is twofold:

Test Py3.11 and Py3.12 against master
Retrigger the test suite on master to make sure that the Py3.8/Windows/onnxruntime errors are present on master (and to debug which version of onnxruntime the issue was introduced in).

Linked issues

Fixes #1319.

Closing this PR because I don't think 3.11/3.12 are high priority right now, and I've documented all of the quirks I discovered in the process of testing this PR.

Reopening this PR to see if I can reproduce the behavior seen in https://github.com/ivadomed/ivadomed/pull/1323/.
For some reason:

This PR: Python 3.8-3.12 all fail (see: https://github.com/ivadomed/ivadomed/actions/runs/11180519695/job/31082434928.)
That PR: Python 3.8 fails but Py3.9/3.10 pass.

If I reopen this PR and remove the onnx limit, will all 5 fail or just Py3.8?

Seems like it reallt is just 3.8. Closing again.