[DO NOT MERGE] Change for links in header and footer

change 'Forum' title in footer section to 'Community'
change 'Rravel buddies and for sale' link to 'Travel Companions' in footer
reorder links in 'Community' section in footer
remove '/airport-transfers' link from global header Bookings section

Details: https://trello.com/c/fNRcOqdf/1550-as-john-i-want-changes-in-global-footer

We need to merge the same changes in rizzo next first https://github.com/lonelyplanet/rizzo-next/pull/837

What about this one? https://github.com/lonelyplanet/rizzo/blob/278b0e601e6869fb442e89d9326c4a2ac228b59f/app/data/layouts/primary_nav.yml#L118

@ is it a part of the footer?

@WunderBart is it a part of the footer? The task was to change the text only in footer.

No, it's primary nav. I know we've disabled it probably everywhere, but it still doesn't make sense to have Thorn Tree there and Community in the footer, IMO.

Sorry, that was the requirement: Community in the footer, and Thorn Tree in  avatar drop down menu.


Build and tests jobs should query the ansible logs at the beginning of a job run

At the beginning of a jenkins job, jenkins should query the ansible log on the machine it is running on and display the output as part of the console output.
Something like tail -n 1 $path_to_ansible_log would display the last ansible change to the machine
On aix, linux and mac the logs are in /var/log/ansible.log. Windows logs are in C:\Users\Administrator (or which ever user is the admin of the machine). These files will need to be readable by jenkins
fyi @smlambert
Not sure if this is the appropriate repo, but I don't think this feature should be exclusive to test jobs. All jobs running on a non dynamic host should have this

Thanks @Haroon-Khel !
From a test perspective, I will likely add it to the EnvDetector code in TKG, in that way, we will also see this information when running from commandline or in other non-Jenkins automation (if it exists).

Thanks @Haroon-Khel . Could you include git repo and sha of the last ansible playbook that applied?

remove mac id


Optimize Images [Easy]

A nice and easy fix for all those looking for a PR.

What is this issue about, can you elaborate please?

@Vaibhavdixit02  Issue is with loading time, due to size of images , gif and other media. It would be great if you optimized these(lossless compression will work) images.

@Vaibhavdixit02  Ideally, any page should never serve images that are larger than the version that's rendered on the user's screen. Anything larger than that just results in wasted bytes and slows down page load time.
So, please run an audit and resize those images (image compression is not needed).

Minifiquei as imagens e consegui minificar o gif de 5mb para 800kb :D

is there any data of images for which we need to work?

You can get them by running audit in ChromeDev Tools.

@ajiteshr7 How is this site hosted? I can setup a gulp script to minify and concatenate scripts and generate a build of the website.

@rafaelfragosom hosted on github pages.

@rafaelfragosom appreciated your concern. But we won't be needing gulp script (not a part of our workflow here).
If you want, what you can do is try fixing the index page background image animation to something better (adding a slide animation or improving upon current animation).

@rafaelfragosom appreciated your concerns.But gulp script is not a part of our workflow here.

@ajiteshr7 That's a concern and unproductive. What happens when you change an image, replace a font or even wants to concatenate and minify your scripts? You do this by hand?
When I work with Github Pages, I have a gulp scripts that automates everything and even deploy my page to Github. Let me know if you want something alike.

@rafaelfragosom the webpage is on a "create once, consume once" model and hence gulp is not being used here.


How to run the lint process?

Question
I'm trying to run the lint process but I'm getting this error:
➜  py-pdf-parser git:(mater) docker-compose run --rm lint
(root) Additional property docs-autobuild is not allowed

There is something I'm missing?
Update: I'm having the same issue with the other docker-file jobs.
Before submitting an issue, please ensure you have read our CONTRIBUTING.md, and follow the Code of Conduct.
Thanks for taking the time to submit an issue. To help us understand your question and answer it quickly, please include the following information where possible:

A good description of the question, including what you are trying to achieve and what the problems are.
A (small as possible) example highlighting your question. Please include code that you have tried, and any files required to run it.
Any required context.

Hey @dantehemerson,
Thanks for getting in touch. A few points:
Linting is now done by pre-commit. You can do a run locally by doing pre-commit run --all-files (you may need to pip-install pre-commit first). Apologies that the documentation is out of date there - part of resolving this issue can be to update that, although I'll not have time until the new year.
Re not being able to run the jobs from docker-compose, what docker-compose version are you running? I can see the docker-compose.yml in this repo doesn't explicitly specify a version, and the error you're getting is about one of the docker services. I suspect your version is either too new or too old to handle this. I can update docker-compose.yml, but again won't have time to get around to this for a while.
Finally, I'd note that you don't need to run these unless you're trying to develop this package. If you're just trying to use py-pdf-parser, then you can simply pip install it, and work away. See https://py-pdf-parser.readthedocs.io/en/latest/overview.html#setup for details.
Hope that helps!


More flexible state

State is currently
type Traversal struct {
	err error
	msg json.RawMessage
}

I propose to make it:
type Traversal struct {
	Err error
	Array []json.RawMessage
}

members are public
state is a JSON Array

Any operation must apply to every element of the Array. (There could be just one, but never zero)
We don't need the End element with io.Writer, the last element can just return *Traversal like everything else. Although we could provide an optional End element for backwards compatibility.

I do like this, it'll make things easier to retrieve when dealing with slices. Plus if it's a single object we can create wrappers to just return things like a msg json.RawMessage


[FLINK-36866] Fix unable to narrow casting on numeric values

A follow up of #3357. It would be surprising if narrowing expressions like CAST(3.14 AS INT) are forbidden.

@yuxiqian The change look good to me, could you rebase latest master?


DNN-8091 portal settings don't support empty values

The root cause of this issue is https://github.com/dnnsoftware/Dnn.Platform/blob/development/DNN Platform/Library/Entities/Portals/PortalController.cs#L3019, which returns the default value is the stored string is null _or string.Empty.  Since string.Empty is a valid value, and the most reasonable value to store is null is desired for a nullable value type, this makes it impossible to retrieve that value through this API.  I'd guess it's too dangerous to change that API now to be consistent, but if consistency is more desired, that method should be fixed. /cc @ashishpd

Thanks for the PR @bdukes, I've assigned to @pberna for reviewing and merging. He will process this on Wednesday (he's away for 2 days).


CI Warnings

We have deprecation warnings which are raised for Linux OS.

Hi @oz123 , I can fix this warning


Update terraform-installer.ts

Add support for downloading arm release from Terraform. It seems to be supported in all their releases - https://releases.hashicorp.com/terraform/

@Ajay-MS  - I do not have access to see why my build fails.
https://dev.azure.com/mseng/b924d696-3eae-4116-8443-9a18392d8544/_build/results?buildId=12182744
Any chance the build result can be made public so we can see?

Please fork and PR in V3 in new repo at: https://github.com/microsoft/azure-pipelines-terraform


Cursor color

Is it possible to change console cursor color?
Maybe it should the same as foreground color set by function 2,15.

It is now the same as the foreground colour ; part of the console code rewrite


Thorax Architecture Example Page 404

http://addyosmani.github.io/todomvc/labs/architecture-examples/thorax/
Trying to figure out why the thorax generator on my machine doesn't seem to work properly but can't find a working example to compare it too.

I think this has been moved to http://todomvc.com/examples/thorax/
Where did you find the link above?
What sort of failure are you seeing with the generator?

It's on the Getting Started page.  None of the links in the TodoMVC section work.
When a new app is created with the generator, it runs both bower install and npm install.  For some reason on my machine (I haven't tested others), I was required to choose a particular version of jquery.  The first few times through, I picked the wrong one.  Once I figured that out, everything worked.

Ok. I'll work on getting the links updated.
Do you know what versions of jQuery were failing? What sort of failures
were you seeing?
On Wed Jan 07 2015 at 6:16:36 AM Brett Bim<EMAIL_ADDRESS>wrote:

It's on the Getting Started page. None of the links in the TodoMVC section
work.
When a new app is created with the generator, it runs both bower install
and npm install. For some reason on my machine (I haven't tested others), I
was required to choose a particular version of jquery. The first few times
through, I picked the wrong one. Once I figured that out, everything
worked.
—
Reply to this email directly or view it on GitHub
https://github.com/walmartlabs/thorax/issues/406#issuecomment-69014524.

The links have been updated. If you saw issues running Thorax under specific versions of jQuery, could you open a bug with that information and we'll be glad to take a look?


MSIX package deployment appsetting JSON variable substitution.

Hello,
Please can someone direct me to steps on JSON variable substitution with MSIX package. I need to have different MSIX packages for different environments as the WPF projects appsettings.json file values change based on environments. I've tried the Release Pipeline "File Transform" task replacing zip with msix but no luck. Since I'm new to this MSIX stuff maybe i'm using the wrong approach?

@syedwajeehbukhari the File Transform task does not operate on MSIX package. You need to provide path to json or xml configuration files for the task to be able to work on.


Create PyPI pip package?

And conda package?

From Sept 29 notes we decided to create a PyPI package.
I propose that we change the package name to echobiome so that it is more descriptive of what the package is for.

The name change is now tracked in #144. I'll remove the "?" in the issue title since we've decided to do this.

This was included as part of PR #241 and can now be closed.


HWKBTM-319 : Add modules deps on templates module

The hawtio plugin modules (APM & BTM) must depend on the  templates
module, "hawkularbtm-templates", so that when they are loaded, templates
are already in the cache and thus not necessary to find them in the FS.

Processing ....

Thanks Alexandre!


Test Issue created at Sun, 13 Feb 2022 02:20:50 GMT

Test issue body  Sun, 13 Feb 2022 02:20:50 GMT

Comment on issue at Sun, 13 Feb 2022 02:20:52 GMT


Update to Foundry V10

There is an issue when installing it to Foundry V10.x, it would be possible to fix it?
I'll be really thankful for any one with the time and knowledge to do it.

Updating it for v10 is on my to do list now that the PF2e system is v10 ready. I’ve got no promises regarding the timeline it’ll depend a lot on how much updating is required.

Cool and thank you so much.

closed in v0.4.0


[RFC] Minimizing quary build work when have --cached-views

Background
When running quary build, the operations it pushes are quite disruptive:

dropping and recreating views
dropping and recreating tables
dropping and recreating materialized views

In the case of systems that track view dependencies at the moment we also run DROP <view_name> CASCADE.
This:

breaks downstream systems
can be un-necessary cost

In order to avoid this, we can use the --cached-views systems to do the minimum amount of work to 1 speed up CI jobs by only changing what is necessary and 2 be more reliable. At the moment it is only being used by the extension to speed up queries when developing.
Proposal
When running quary build --cached-views, we pull the cached views and for instances where it makes sense, we try to do the least disruptive work if the models have not changed.

Cache Status
View
Table
Materialized View

Cached and matching
Leave
Rerun
Refresh Data

Not matching
Recreate
Rerun
Recreate

Abandoned Ideas
See some discussion here: https://github.com/quarylabs/quary/issues/88
Sections
Implementation
Steps:

[ ] Pull the views
[ ] In the create/drop view functions allow for optional return
[ ] In the create/drop view functions return appropriately for all the things
[ ] Tie up to the quary build command

UX
From a UX perspective this won't look all that different.

@kraftaa this is my current proposal?

@louisjoecodes any thoughts, I think we should move issues here ...


WASI compile the library

Brief
We would like to provide an easy-to-execute way for others to create a WASI module from CryptID.native.
Task
Create a build script which is able to compile CryptID.native into a WASI-compatible WASM module.

Depends on #67


smctl command not recognized in windows

Issue Template
Step 1: Is this issue for here?
smctl related issue.
Step 2: Describe the problem (very short text description):
I am trying to install service-manager-cli
Step 3: Describe the problem (steps):
Steps to reproduce:

I was installing smctl for windows 64 bit.
I have downloaded smactl_windows64.exe and added exe to the path.

Observed Results:

But the "smctl" command was not recognized.

Expected Results:

"smctl" should have been recognized.

After a long debugging, I understood that the smctl_windows64.exe has to be renamed to smctl.exe .
Please update your readme with this point.

Hello,
The readme file is updated with the installation guide already.
https://github.com/Peripli/service-manager-cli#rename-the-cli-binary


[fix] cms page_filter contains_urls pagenation

Coverage decreased (-0.003%) to 84.348% when pulling d89f88509d0bb81b0330749ab1f0360c5b51593d on fix-contains-urls-pagenation into f90e0cf42d5baa0da653379f82301ab944a16913 on master.


Update vae.py: do not use global encoder and decoder

They are attributes of the VAE class.

maybe
        z_mean, z_log_var, z = self.encoder(data)
        reconstruction = self.decoder(z)


OpenCV 3 not supported

In your code, you use the nonfree module of OpenCV 2, thus when build your code the OpenCV 3 cannot be used. Hope write this down in your installation guide.

@AIBluefisher The non-free module is used for feature extraction (sift) and can be removed. OpenCV 3 can also be built with non-free modules (opencv_contrib). But I am not sure the currently repo can work with opencv3 due to the interface changes. You can go ahead and try, and pull request your findings if necessary :).


WateringCan Water Level Management

Description
Implement the Watering-Can level component that will manage and update the water that can be carried within the watering-can as apart of the farming tools #51. It will contain two key variables, MaxCapacity and CurrentWaterLevel, and the supporting functions to update these values. Along with all unit testing for this component file to ensure reliability.
Completion Deadline: Aug. 28
Milestones

[x] Watering-can level component
[x] Unit tests

Member
Liam Anderson (@LiamAnd)

done


Ignoring InterruptedException during a SiddhiRuntime shutdown

Purpose

When SiddhiRuntime is shut down, the scheduler services will also be shut down. In such a case, if one tries to submit a new task to the scheduler, it will throw an InterruptedException. Hence, such cases can be ignored.

Codecov Report

Merging #1322 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@             Coverage Diff              @@
##             master    #1322      +/-   ##
============================================
- Coverage     62.78%   62.78%   -0.01%
  Complexity       37       37
============================================
  Files           623      623
  Lines         25615    25616       +1
  Branches       4015     4016       +1
============================================
  Hits          16083    16083
- Misses         8247     8248       +1
  Partials       1285     1285

Impacted Files
Coverage Δ
Complexity Δ

...e/src/main/java/io/siddhi/core/util/Scheduler.java
82.16% <0%> (-0.53%)
0 <0> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4140ad8...d052adb. Read the comment docs.


Update solution.py

Changing max range on second for loop to be 100

n = 5
[1,1,99,99,99]
If max range is left as n, the result of the above would be 2 when the result should be 3.
I realize its a small change, but your work has helped me study so I figured I'd return the favor and make this one correct for other studiers looking at your work. Thank you.

Hi @dsims21,
Thanks for the correction :)
I'm happy that this work helps other people to learn so feel free to send other PRs if you notice something that could be improved.


compensationMethod spring transaction didn't  take effect

    @SagaStart
	@Transactional
	@Compensable(compensationMethod = "cancelDisAgreePatient")
	public void disAgreePatient(String doctorNo, String patientNo) {
		 //xxxxxxx
	}

	@Transactional
	public void cancelDisAgreePatient(String doctorNo, String patientNo) {
		  // Spring transaction didn't take effect
		 // xxxxxx
	}

What should I do?

I read the latest source code and found the alpha server grpc only listening the tcc event, do I misunderstand? is the compensate mechanism still effective?

@minlywang  If there is an exception which is thrown from disAgreePatient method, the cancelDisAgreePatient method won't be invoked.
@alexyezhigang Current Alpha Server supports the TCC and Saga events at the same time.

What I mean is the annotation of "@Transactional" does not work

@minlywang Normally we cannot put the SagaStart and Compensable on the same method.  SagaStart should be on the method which could call the Compensable method remotely.

        @Transactional
	@Compensable(compensationMethod = "cancelDisAgreePatient")
	public void disAgreePatient(String doctorNo, String patientNo) {
		 //xxxxxxx
	}

	@Transactional
	public void cancelDisAgreePatient(String doctorNo, String patientNo) {
		  // Spring transaction didn't take effect
		 // xxxxxx
	}

In this way, the spring transcation of cancelDisAgreePatient method still didn't take effect.

@RestController
public class TestController{
	@Autowire
	private BiZService biZService;

	//@Transactional
	@Compensable(compensationMethod = "cancelBiz")
	public void doBiz() {
		 biZService.doBiz();
	}

	//@Transactional
	public void cancelBiz() {
		 biZService.cancelBiz();
	}
}

@Service
public class BiZService{

	@Transactional
	public void doBiz(){
		......
	}

	@Transactional
	public void cancelBiz(){
		......
	}
}

I have to put the transacton methods in a separate class in order to make the spring transaction take effect.

We just need to make sure the @Transactional is the last one to take effect.  I have the same conversation with @cherrylzhao.  @cherrylzhao Could you help us double check it?

The following is what I really want it to be
@RestController
public class TestController{
	@Autowire
	private BiZService biZService;

	@SagaStart
	public void doBiz() {
		 biZService.doBiz();
		 //remote service
		 //remote service
		 //remote service
		 //remote service
	}

}

@Service
public class BiZService{

	@Transactional
	@Compensable(compensationMethod = "cancelBiz")
	public void doBiz(){
		......
	}

	//in this way the spring transaction didn't take effect
	@Transactional
	public void cancelBiz(){
		......
	}
}

Can you make the above work?

Yeah, it should work that way. Let's find out what's wrong in your demo first.
Could you provide a simple test case for us to reproduce the error?

I think this problem is related to the machanism of spring transaction and the machanism of invocation of compensationMethod.
@RestController
public class TestController{
	@Autowire
	private BiZService biZService;

	public void doBiz() {
		//in this way the spring transaction of doSomething don't take effect
		biZService.doBiz();
		//in this way the spring transaction of doSomething takes effect
		biZService.doSomething();
	}

}

@Service
public class BiZService{

	public void doBiz(){
		//in this way the spring transaction of doSomething don't take effect
		doSomething();
	}

	@Transactional
	public void doSomething(){
		...
	}
}

The machanism of spring transaction relates to AOP.

I don't have the sample of POC and test it directly in our project.
You can test it in any spring boot project.
The real problem is spring transaction machanism and the machanism of invocation of compensationMethod.

@minlywang
I have reproduce this problem in my machine.
we should cached the transaction proxy bean in callback context instead of raw bean.
I'll try to fix this bug.

@cherry|zhao，Thanks for your time. Could you fill a JIRA for it?

I have filled it yesterday

@cherrylzhao did you deploy the maven jar of this fix?

this is our JIRA system, I have fixed it just now, please see the PR
https://github.com/apache/incubator-servicecomb-saga/pull/321

@WillemJiang When can I test this fix?

@WillemJiang When can I test this fix?

@minlywang The code was merged into master branch, you can build it yourself or just using the SNAPSHOT version for Apache. https://repository.apache.org/content/repositories/snapshots/

@minlywang  Can you check the code and build it yourself?
I guess there are some jars are out of date.

@WillemJiang It seems that omega-spring-tx is out of date


Avoid logging parameters when logging statements

https://hibernate.atlassian.net/browse/HHH-11082

I've been wondering if we should still at leave give an option to log the parameters, but considering that this log message was just recently introduced: seems users have been happy enough without any log at all for many years. So I'm proposing to keep it simple.

I agree with keeping it simple :+1:

I need to re-merge, apparently I was using the wrong JIRA id.

merged properly now, and backported to 5.4. Thanks!


Training and inference discussion for active learning round 2

Continuation from the previous round of training: https://github.com/sct-pipeline/fmri-segmentation/issues/35
What is the round 2 model
The model which was fine-tuned on the manually corrected segmentations as per the QCs mentioned in #35  is the round 2 model. A total of 26 images were added in the training of this model since we fine-tuned the previously trained round 1 model.
A list of subjects used for the fine-tuning is below:  finetuning.yml
The config (containing preprocessing, hyperparameters) for nnUNetv2 training is:  plans.json
After the training was completed, I ran inference on the rest of the images whose segmentations have to be included in the consequent rounds of training (226 images), below is the QC. 40 subjects from these images will be chosen and included in the round 3 of training:
qc_round2_inference.zip
The steps to reproduce the above QC results (/run inference) are the following:

Clone this repos
cd fmri-segmentation
Download the model weights (the whole folder) from the link: https://drive.google.com/drive/folders/1WSn-15wGWz6i2_aZeQTwKls2sZ6dpfHf?usp=share_link
Install dependencies

pip install -r run_nnunet_inference_requirements.txt

Run the command:

python run_nnunet_inference.py --path-dataset <PATH TO FOLDER CONTAINING IMAGES, SUFFIXED WITH _0000> --path-out <PATH TO O/P FOLDER> --path-model <PATH TO DOWNLOADED WEIGHTS FOLDER>

Next steps:

[x] Add inference QCs in this comment
[x] Choose and correct 40 segmentations and include it in the next training round along with the ones that have good segmentation after inference.
[ ]  Get metric by running evaluation on the held-out test set (#33)

The list of 40 subjects chosen for manual corrections is below:
qc_fail.yml.zip
The QC report for the segmentations manually corrected by me for the above 40 subjects is below:
qc_round_2_corrected.zip
@jcohenadad , I would like your inputs for the above manually corrected images. I will use these images for the next round of training.
CC: @MerveKaptan

Julien: Redo KCL subjects

here you go: qc_flags.json
two issues only:

Closing the issue since the round 2 training was successfully completed (including running inference and manual correction)


ember-cli-update to v3.2.0

We are upgrading our app to v3.2 right now. Are you planning to release a new version with this update soon?

@viniciussbs yes, soon. I was under the impression that this update wouldn't actually matter to consumers of this addon given that it's only updating dev dependencies. Do you have reason to believe otherwise?

@bgentry No reason. I just asked because we are updating our packages that were in beta. We will update to 1.0.2, then. ✌


Feature/5 transfer the old react project into this one

Ok, this mr is growing out of control, next issues will be shorter and mainly focused on smaller chunks of the project. Closing this one rn.


Tool support for creating an uber jar containing tracer and framework integrations of choice

The opentracing-agent.jar produced by this project is a management mechanism for installing OpenTracing based instrumentation code into a JVM based application - however it does not actually bundle a specific OpenTracing Tracer implementation, rules, or OpenTracing based framework integrations.
This is to enable an end user/organisation to select the Tracer implementation of choice, and also the specific rules/framework integrations (and versions) they require for their applications. They then have two choses, place these dependencies onto the classpath of the application being instrumented, where the agent will detect and install them, or create an 'uber' jar containing the agent and all of the relevant dependencies.
This issue has been created to discuss whether tool support can be provided to make it easier to create the uber jar, e.g. based on some manifest listing the desired tracer, custom rules, framework integrations, etc.

This would be really great!
Maybe another option would be just to package the agent and set a property with fully qualified tracer class which would be used by the agent.

The agent uses the global-tracer, so as long as the Tracer impl is in the classpath and can be instantiated by the global-tracer then it would be fine. Same approach would apply whether the Trace impl was packaged with, or independent of the agent.

Hey @objectiser could you decribe more less how to build the uber jar? I followed your java-agent-spring-boot-example but I end up with 2 jars, one is just the original agent and the other is the just the rules and Tracer, no agent.

@jam01 The uber jar can be built using the maven shade plugin, which can include the agent, list of required rule dependencies and a tracer implementation - however it requires the user to define a maven project. This won't be a problem for some users.
So it just depends whether there is benefit in a separate tool that effectively provides a command line wrapper around the maven shade plugin, being supplied some form of manifest (e.g. yaml) with the list of rules and tracer.


[Feature Request] Magenta and Google Colaboratory Tutorial

Was hoping someone could do a tutorial about how to use Magenta on Google Colaboratory. Many of us want to train our own models but don't have access to a powerful enough machine to do so (can't even run tensorflow on my machine...). Colaboratory is a good option but try as I might I could not get an example to work. I think it has something to do about loading the magenta environment. I know there are some Jupyter notebooks that will run on Colaboratory but they are not really teaching users how to train their own models and generate new melodies. I, for one, and many others would very much appreciate any efforts put towards making magenta more accessible. It sounds like a lot of fun and would really like to try it. Thank you in advance.

all of the links posted show how to use magenta trained models to make music but not how to take your own data for something like performanceRNN and train it using the gpu on colab and then create music. this is totally missing from the documentation

Hi @Ddedalous, the training instructions is contained in the README in the directory of each model. Agreed it's not as simple as running directly on Colab. I'll reopen this Issue but I doubt any of us on the Google side will be able to get to this anytime soon. Please feel free to contribute a Colab yourself -- it shouldn't be too much work to call the same functions that are in our training script from Colab.

I also noticed that for the training part is missing a huge part of documentation. For example I don't understand which are all the hyperparams that can be set since is not written somewhere and so on.


[Bug]: Lost connection to miniserver

Describe the bug
Realized that I cannot control some entities, started to look into the logs and found some errors.
Need to further triage this, just recording the logs and stuff so I can have a look into it.
Firmware of your Miniserver
<IP_ADDRESS>
HomeAssistant install method
HAOS on VM
Version of HomeAssistant
Core 2023.12.1
Supervisor 2023.11.6
Operating System 11.2
Frontend 20231208.2
Version of Pyloxone
0.4.8
Update pyloxone
Not yet, updating to 0.4.12 and will monitor
Log

2023-12-10 22:10:00.523 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1302, in close_connection
    await self.transfer_data_task
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 959, in transfer_data
    message = await self.read_message()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1029, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1104, in read_data_frame
    frame = await self.read_frame(max_size)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1161, in read_frame
    frame = await Frame.read(
            ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/framing.py", line 68, in read
    data = await reader(2)
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/streams.py", line 732, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/local/lib/python3.11/asyncio/streams.py", line 525, in _wait_for_data
    await self._waiter
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/loxone/miniserver.py", line 189, in listen_loxone_send
    await self.api.send_websocket_command(device_uuid, value)
  File "/config/custom_components/loxone/api.py", line 386, in send_websocket_command
    await self._ws.send(command)
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 935, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: sent 1011 (unexpected error) keepalive ping timeout; no close frame received

@ztamas83 you need the newest pyloxone for ha 2023.12

Thanks! Updated and monitoring now. I close this issue then and reopen should it still be a problem

It looks like the problem is still there, somehow the connection is lost and never recovered, right now the only clue is the following log. I try to capture the situation when the connection actually brakes.
2023-12-13 06:03:16.920 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1302, in close_connection
    await self.transfer_data_task
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 959, in transfer_data
    message = await self.read_message()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1029, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1104, in read_data_frame
    frame = await self.read_frame(max_size)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1161, in read_frame
    frame = await Frame.read(
            ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/framing.py", line 68, in read
    data = await reader(2)
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/streams.py", line 732, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/local/lib/python3.11/asyncio/streams.py", line 525, in _wait_for_data
    await self._waiter
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/loxone/miniserver.py", line 182, in listen_loxone_send
    await self.api.send_websocket_command(device_uuid, value)
  File "/config/custom_components/loxone/api.py", line 369, in send_websocket_command
    await self._ws.send(command)
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 635, in send
    await self.ensure_open()
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 935, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: sent 1011 (unexpected error) keepalive ping timeout; no close frame received

PR #223 helps to recover from the error at least.
Will continue looking for where the connection is lost and why it doesn't recover.

Keepalive timeout
2023-12-13 22:55:13.673 DEBUG (MainThread) [custom_components.loxone.api.websocket] % sending keepalive ping
2023-12-13 22:55:13.673 DEBUG (MainThread) [custom_components.loxone.api.websocket] > PING 39 ff ce 63 [binary, 4 bytes]
2023-12-13 22:55:13.677 DEBUG (MainThread) [custom_components.loxone.api.websocket] < PONG 39 ff ce 63 [binary, 4 bytes]
2023-12-13 22:55:13.677 DEBUG (MainThread) [custom_components.loxone.api.websocket] % received keepalive pong
2023-12-13 22:55:13.690 DEBUG (MainThread) [custom_components.loxone.api.websocket] < BINARY 03 01 00 00 58 1b 00 00 [8 bytes]
2023-12-13 22:55:13.692 DEBUG (MainThread) [custom_components.loxone.api.websocket] < BINARY 1f 8b 08 00 00 00 00 00 00 13 cd 5d e9 6f e3 46 ... dd 93 65 39 c0 6c 00 00 [7000 bytes]
2023-12-13 22:55:33.680 DEBUG (MainThread) [custom_components.loxone.api.websocket] % sending keepalive ping
2023-12-13 22:55:33.681 DEBUG (MainThread) [custom_components.loxone.api.websocket] > PING ce e8 42 d0 [binary, 4 bytes]
2023-12-13 22:55:35.594 INFO (MainThread) [gql.transport.websockets] >>> {"type": "ping"}
2023-12-13 22:55:35.665 DEBUG (MainThread) [custom_components.loxone.api.websocket] > TEXT 'keepalive' [9 bytes]
2023-12-13 22:55:36.004 DEBUG (MainThread) [custom_components.loxone.api.websocket] > TEXT 'keepalive' [9 bytes]
2023-12-13 22:55:39.673 INFO (MainThread) [gql.transport.websockets] <<< {"type":"pong"}
2023-12-13 22:55:44.847 DEBUG (MainThread) [custom_components.loxone.api.websocket] > TEXT 'keepalive' [9 bytes]
2023-12-13 22:55:53.684 DEBUG (MainThread) [custom_components.loxone.api.websocket] ! timed out waiting for keepalive pong
2023-12-13 22:55:53.685 DEBUG (MainThread) [custom_components.loxone.api.websocket] ! failing connection with code 1011
2023-12-13 22:55:53.685 DEBUG (MainThread) [custom_components.loxone.api.websocket] = connection is CLOSING
2023-12-13 22:55:53.685 DEBUG (MainThread) [custom_components.loxone.api.websocket] > CLOSE 1011 (unexpected error) keepalive ping timeout [24 bytes]
2023-12-13 22:56:09.676 INFO (MainThread) [gql.transport.websockets] >>> {"type": "ping"}
2023-12-13 22:56:09.720 ERROR (MainThread) [tibber.home] Error in rt_subscribe
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 959, in transfer_data
    message = await self.read_message()
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1029, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1104, in read_data_frame
    frame = await self.read_frame(max_size)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 1161, in read_frame
    frame = await Frame.read(
            ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/framing.py", line 68, in read
    data = await reader(2)
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/streams.py", line 732, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/local/lib/python3.11/asyncio/streams.py", line 525, in _wait_for_data
    await self._waiter
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 964, in _read_ready__get_buffer
    nbytes = self._sock.recv_into(buf)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/tibber/home.py", line 439, in _start
    async for data in self._tibber_control.realtime.sub_manager.session.subscribe(
  File "/usr/local/lib/python3.11/site-packages/gql/client.py", line 1064, in subscribe
    async for result in inner_generator:
  File "/usr/local/lib/python3.11/site-packages/gql/client.py", line 960, in _subscribe
    async for result in inner_generator:
  File "/usr/local/lib/python3.11/site-packages/gql/transport/websockets_base.py", line 388, in subscribe
    answer_type, execution_result = await listener.get()
                                    ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/gql/transport/websockets_base.py", line 56, in get
    raise item
  File "/usr/local/lib/python3.11/site-packages/gql/transport/websockets_base.py", line 301, in _receive_data_loop
    answer = await self._receive()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tibber/websocker_transport.py", line 37, in _receive
    msg = await asyncio.wait_for(super()._receive(), timeout=self._timeout)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/tasks.py", line 489, in wait_for
    return fut.result()
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/gql/transport/websockets_base.py", line 231, in _receive
    data: Data = await self.websocket.recv()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 568, in recv
    await self.ensure_open()
  File "/usr/local/lib/python3.11/site-packages/websockets/legacy/protocol.py", line 935, in ensure_open
    raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: no close frame received or sent


gh-oauth.imsun.net 证书失效

登录之后控制台出现如下错误：

POST https://gh-oauth.imsun.net/ net::ERR_CERT_AUTHORITY_INVALID

无法进行后续操作

Same issue here. Looking into a solution now.

Problem is on Line #3415 of gitment.browser.js:
_utils.http.post('https://gh-oauth.imsun.net/', {
to
_utils.http.post('https://cors-anywhere.herokuapp.com/https://github.com/login/oauth/access_token', {
He was using gh-oauth.imsun.net as a CORS proxy to bypass the issue of the no accept headers issue. if you replace it with the above code it will work, but now spits out the error validation problem, which I believe is solved by a pull request. His website has an expired ssl cert which doesnt allow ssl websites from accessing it. So I bypassed his proxy with a well know CORS proxy above. While it is a solution to the problem, still passing keys to an uncontrolled proxy. FYI

same issue here

一样的问题  都没解决吗？

修改gitment.browser.js文件的3416行将https://gh-oauth.imsun.net/
换成：
https://baifachuan.com:8443/
我自己重新搭建了一个这个代理服务。纯代理。

修改gitment.browser.js文件的3416行将https://gh-oauth.imsun.net/
换成：
https://baifachuan.com:8443/
我自己重新搭建了一个这个代理服务。
可以继续实现访问。

修改gitment.browser.js文件的3416行将https://gh-oauth.imsun.net/替换成：https://baifachuan.com:8443/
该服务器代码在：https://github.com/imsun/gh-oauth-server，不会记录任何信息，我也是自己的博客使用：https:www.baifachuan.com，因为该服务器需要https，所以构建在自己的服务下，需要的可以使用。
update the gitment.browser.js file  replace https://gh-oauth.imsun.net/ to https://baifachuan.com:8443/.
This service won't record or store anything. It only attaches a CORS header to that request and provides proxy. So that users can login in the frontend without any server-side implementation.


Add vim temp files to gitignore

This PR does pretty much what it says on the tin. The various .<FILENAME>.sw[op] files that vim creates as temp files are now excluded from git.

Thanks!


Add ToHtmlSerializerPlugins to PegDownPlugins builder

This change adds a way to register ToHtmlSerializerPlugin's with PegDownPlugins.  PegDownProcessor was updated to pass ToHtmlSerializerPlugin's down to the ToHtmlSerializer.
This seems to address #124

@sirthias could it be merged please?

Cheery-picked second commit.
Thanks, Sterling!


fav-info is stored in in-process mem.

Found that fav-info is stored in in-process mem,
and is brutal-forcely do fav-save when the ssh/telnet/ws session is done.
https://github.com/ptt/pttbbs/blob/master/mbbsd/board.c#L1647
https://github.com/ptt/pttbbs/blob/master/mbbsd/mbbsd.c#L206
Design concern:

At least the fav-info in-file should be with one complete version. it shouldn't be parts from one connection and the others from another connection (resulting broken file)
We should request the fav-info from file as least as possible.
fav-info is a many-reads-few-writes operation.

Currently pttbbs allows at most 3 concurrent connections. So it's ok to have possible conflicting fav-setup as long as we stick to Design 1.
(In rare situation, the user has a dead process, and the user allows the dead process continue working,
and the user modify the fav-info and logout before the dead-process is killed.)
(The user will just reset it again)
Requires frontend to take care of fav-editing and middleware to have cache of the fav-info.
When doing get-fav from frontend-UI:

middleware check the cache first.
if the cache is within n seconds (10 for now). return the cache.
middleware provides last-load-time with the go-bbs and try to get the fav from file.
go-bbs: first compare the load-time with the mtime of the file. return mtime as no-need to update. otherwise return the full fav.
middleware update the cache accordingly.

When doing update-fav from frontend-UI:

frontend takes care of the editing, do only fav-save to the middleware. (frontend shouldn't do fav-save within 10 seconds.)
the middleware store in tmp-db, spawn process/thread to do fav-save after 10 second.
the middleware check that the fav-save is still the newest setup to do. do save on go-bbs.
go-bbs do fav-save only when the load-time from the middleware is newer than the mtime. return fav-from-file to middleware to ensure sync from fav-from-file.

long-term TODO: pttbbs use db to save .FAV, to avoid continuous-querying-file from go-bbs.

目前應該從靜態檔案讀取我的最愛的部分也還沒實作？

It is in the process of development.
https://github.com/chhsiao1981/go-bbs/tree/dev/fav


Cannot install with Homebrew

Hi, thanks for creating this project. It is just what I need. Unfortunately, I cannot install it with Homebrew.
This is the error it throws:

My setup is the following:

Homebrew 3.1.3 (brew doctor throws no errors)
Macos Catalina 10.15.7

Any suggestion on how to proceed?

It looks like Process.kill was deprecated for Process.signal in Crystal v1.0.0. I went ahead and released a new version of Thyme (v0.1.5) which fixes that issue.
Try running brew install hughbien/tap/thyme and it should work this time. Let me know if it doesn't.

Thanks for the quick response. Now, it throws the following error:

do I really need to install the whole XCode for this?

Crystal Lang is a requirement, but the Command Line Tools should be enough without Xcode.
That error message is unexpected though, because if you got the previous error (undefined method kill) that means you should already have Crystal 1.0 installed. I'm not sure why it's asking for Xcode now.
When you run crystal -v, does it show v1.0.0?
Some options that might work if you already have Crystal installed:

brew install hughbien/tap/thyme --without-crystal
git clone https://github.com/hughbien/thyme.git && cd thyme && make build install

If Crystal isn't installed yet, you'll need to get it here: https://crystal-lang.org/install/on_mac_os/ until I can figure out why it's trying to install Xcode as a dependency.

When you run crystal -v, does it show v1.0.0?

Yes, it says Crystal 1.0.0 (2021-03-22)

brew install hughbien/tap/thyme --without-crystal

It throws the following:

Thanks for the help!

Sorry for the delay on this! It looks like the build is having trouble finding the shards binary, most likely due to it being shimmed. I added a HOMEBREW_SHARDS env var option to help with it:
HOMEBREW_SHARDS=`which shards` brew install hughbien/tap/thyme --without-crystal

Hopefully that helps!

Sorry for the delay on this!
It turns out Homebrew runs in a new process without shims loaded, so if you have Crystal installed as a shim the formula will fail. I added a few options to solve this.
If you have Crystal v1.0.0 installed via Homebrew, you can run:
HOMEBREW_SHIM=$(brew --prefix crystal)/bin brew install hughbien/tap/thyme --without-crystal

If you have Crystal v1.0.0 installed via Asdf, you can run:
HOMEBREW_SHIM=$(asdf which crystal)/../../embedded/bin HOMEBREW_CRYSTAL_PATH=`crystal env CRYSTAL_PATH` brew install hughbien/tap/thyme --without-crystal

You can also install it manually via:
git clone https://github.com/hughbien/thyme.git
cd thyme
make
make install

Hopefully that helps and thanks for the bug report!

Hey 👋🏼  I just updated to big sur, and tried brew install hughbien/tap/thyme and worked with no issues at all. With this, I will proceed to close this issue as I'm not able to reproduce the error anymore. Thanks for this awesome tool 🍅 !


Suggestion: Top Role Colour Variable

It would be nice to have a variable so your top-most role colour shows up in-game 👍

Will add in next update

Added in 3.2.8


Add support for Silvercrest Sous Vide Stick SSVSS 1200 A1 (100358082)

Device is similar to Inkbird sous vide, but doesn't support id: 102 for current status.

I modified the inkbird config to also cover this, since they are almost the same.


Failed to ingest IBM Db2 metadata using SQLAlchemy source

Describe the Bug
While trying to ingest metadata for IBM Db2 using SQLAlchemy Source and dialect ibm-db-sa, following error occurred
Traceback (most recent call last):   File "/data/vdc/conda/condapub/svc_am_cicd/envs/dh-actions/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 373, in run     for record_envelope in self.transform(record_envelopes):   File "/data/vdc/conda/condapub/svc_am_cicd/envs/dh-actions/lib/python3.10/site-packages/datahub/ingestion/extractor/mce_extractor.py", line 77, in get_records     raise ValueError( ValueError: source produced an invalid metadata work unit: MetadataChangeEventClass(
Source has produced invalid metadata workunit. It results in Datasets without SchemaMetadata.
Cli Version: 0.10.2
Datahub Version: v0.10.2
SqlAlchemy Version: 1.4.41
To Reproduce
Recipe Used
source:  type: sqlalchemy  config:    connect_uri: "ibm_db_sa://user:password@host.name.com:50000/database "    platform: "db2"    include_tables: true    include_views: false sink:  type: "datahub-rest"  config:    server: ${DATAHUB_GMS_HOST}    token: ${DATAHUB_GMS_TOKEN}
Expected behavior
Complete Ingestion of DB2 Metadata
Screenshot
As we can see no schemetadata has been pushed

Attaching Logs:
exec-urn_li_dataHubExecutionRequest_sqlalchemy-2023_09_01-14_44_01.log
Reason for the behaviour
DataSetProperties Class field "description" has invalid data
DatasetPropertiesClass({'customProperties': {}, 'externalUrl': None, 'name': 'vvaf_drf_im_loc', 'qualifiedName': None, 'description': (None,),
I have fixed the issue and will raise PR

Can you please suggest some beginner friendly issues.

@HHQuraishi  Source produced invalid metadata workunit during ingestion


Make error message more descriptive

Hey, this is to help resolve the issue (https://github.com/jenkins-x/jx/issues/8480). It seems to be the case that the error message generated here and upstreamed is not quite descriptive for many newcomers onboarding jenkins-x.
Also, would be great if any feedback regarding the wording of the error message, though the newer line seemed to be descriptive and clear enough to me.

/assign @garethjevans

Hey @garethjevans, thanks for the review.
I do think you have a point that this might not be the correct place for the message to go, though it does'nt seem the piece of code you referenced is being called given that the error "failed to create the jx client" does not show up in the issue.
Will research a little more on it and where we can ideally place the error message


Question on gradient of depth

Thanks for the great work!  This work is awesome.
I note that the gradient is not backpropaged from the pred_depth, do you plan to add it? This would be super helpful for those who want to add geometry supervision.
https://github.com/wanmeihuali/taichi_3d_gaussian_splatting/blob/f7631e327d3e6e995324f1755bc3da25d603a584/taichi_3d_gaussian_splatting/GaussianPointCloudRasterisation.py#L972

Sounds not very difficult. I can try to do that if I have time... Also, contributions are welcomed!

Thank you @wanmeihuali , I quickly skim the code,  seems challenging to me 😂 , need to propagage the gradient from predicted_depth to alpha (and then to point_uv) and point_depth, and then merge  to get the gradient of covariance and point location.
Let me know if my understanding is not correct. I would be happy to give it a try if there is a simple way to implement it, though I haven't wrote taichi before.

I can provide some idea, please correct me if I am wrong.
$$
T_i=\prod_{j=1}^{i-1} (1-\alpha_{j})
$$
$$
d_{pixel}=\frac{\sum_{i=1}^{n}{d_{i} \alpha_{i} \prod_{j=1}^{i-1} (1-\alpha_{j})}}{\sum_{i=1}^{n}{\alpha_{i} \prod_{j=1}^{i-1} (1-\alpha_{j})}}=\frac{\sum_{i=1}^{n}{d_{i} \alpha_{i} T_i}}{\sum_{i=1}^{n}{\alpha_{i} T_i}}
$$
in which $d_{pixel}$ is the pixel level depth, $d_{i}$ is the depth of the $i$-th effective gaussian point on the pixel. $\sum_{i=1}^{n}{\alpha_{i} T_i}$ is in range $(0,1)$ and as we only wants to do gradient descent on $d_{pixel}$, we can ignore it. Then we have:
$$
\frac{\partial{d_{pixel}}}{\partial{d_i}}=\frac{\alpha_{i} T_i}{\sum_{i=1}^{n}{\alpha_{i} T_i}} \propto \alpha_{i} T_i
$$
$$
\frac{\partial{loss}}{\partial{d_i}}=\frac{\partial{loss}}{\partial{d_{pixel}}} \frac{\partial{d_{pixel}}}{\partial{d_i}} \propto \alpha_{i} T_i \frac{\partial{loss}}{\partial{d_{pixel}}}
$$
If we only want geometry supervision for point location, then optimize point depth $d_i$ directly is enough. However, if we want to optimize $\alpha_i$ as well, then we need to consider the effect of $\alpha_i$ on $d_{pixel}$.
$$
\frac{\partial{loss}}{\partial{\alpha_i}}=\frac{\partial{loss}}{\partial{d_{pixel}}} \frac{\partial{d_{pixel}}}{\partial{\alpha_i}} + \frac{\partial{loss}}{\partial{color}} \frac{\partial{color}}{\partial{\alpha_i}}
$$
in which $\frac{\partial{color}}{\partial{\alpha_i}}$ is the color gradient of the $i$-th effective gaussian point on the pixel, which is the current $\frac{\partial{loss}}{\partial{\alpha_i}}$ in the code. We just need to add $\frac{\partial{loss}}{\partial{d_{pixel}}} \frac{\partial{d_{pixel}}}{\partial{\alpha_i}}$ to it.
$$
\frac{\partial{d_{pixel}}}{\partial{\alpha_i}}=\frac{d_i T_i - \sum_{j=i+1}^{n}{\frac{d_j \alpha_j T_j}{1-\alpha_i}}} {\sum_{i=1}^{n}{\alpha_{i} T_i}} \propto {d_i T_i - \sum_{j=i+1}^{n}{\frac{d_j \alpha_j T_j}{1-\alpha_i}}}
$$
in the code, we already have:
$$
w_i = \sum_{j=i+1}^{n}{\frac{color_j \alpha_j T_j}{1-\alpha_i}}
$$
so we can define a new variable to store the value of $\sum_{j=i+1}^{n}{\frac{d_j \alpha_j T_j}{1-\alpha_i}}$  in a similar way. And handle $\frac{\partial{\alpha_i}}{\partial{position_i}}$ and $\frac{\partial{\alpha_i}}{\partial{covariance_i}}$ is already implemented in the code and you don't need to change it.

I created a draft PR(https://github.com/wanmeihuali/taichi_3d_gaussian_splatting/pull/122) that modified the rasterizer(taichi) part to support gradient from depth. The code is not tested(I don't have any test dataset either...), also, the pytorch part shall also need to be modified. You can start from here to modify the code and do experiments. @chenyuntc

I tested it this morning, it works for the pts' coordinates!  Thank you so much!
import taichi as ti
from taichi_3d_gaussian_splatting.GaussianPointCloudRasterisation import GaussianPointCloudRasterisation
import torch
import numpy as np
from taichi_3d_gaussian_splatting.Camera import CameraInfo
from taichi_3d_gaussian_splatting.utils import se3_to_quaternion_and_translation_torch

RasterConifg = GaussianPointCloudRasterisation.GaussianPointCloudRasterisationConfig
def render(pts, pts_feat, c2w, intrin, HW):
    rasterisation = GaussianPointCloudRasterisation(
            config=RasterConifg(near_plane=0.4, far_plane=2000.0, depth_to_sort_key_scale=10.0, rgb_only=False),
        )
    camera_info = CameraInfo(camera_intrinsics=intrin.to(pts.device),camera_height=HW[0],camera_width=HW[1],
                             camera_id=0)
    q_pointcloud_camera, t_pointcloud_camera = se3_to_quaternion_and_translation_torch(c2w[None])
    gaussian_input = GaussianPointCloudRasterisation.GaussianPointCloudRasterisationInput(
        point_cloud=pts.float(),
        point_cloud_features=pts_feat.cuda(),
        point_object_id=torch.zeros(pts.shape[0], dtype=torch.int32,device=pts.device),
        point_invalid_mask=torch.zeros(pts.shape[0], dtype=torch.int8,device=pts.device),
        camera_info=camera_info,
        q_pointcloud_camera=q_pointcloud_camera.cuda().contiguous(),
        t_pointcloud_camera=t_pointcloud_camera.cuda().contiguous(),
        color_max_sh_band=6,
    )
    res = rasterisation(gaussian_input)
    return res
if __name__ == '__main__':
    ti.init(arch=ti.cuda, device_memory_GB=0.1)
    pts = torch.randn(10000,3,device='cuda').requires_grad_(True)
    pts_feat = torch.zeros(pts.shape[0], 56,device=pts.device)
    pts_feat[:, 0:4] = torch.rand_like(pts_feat[:, 0:4])
    pts_feat[:, 0:4] = pts_feat[:, 0:4] /  torch.norm(pts_feat[:, 0:4], dim=1, keepdim=True)
    pts_feat[:, 4:7] = torch.zeros(pts.shape[0],1,device=pts.device).repeat(1,3)
    pts_feat[:, 7] =  0.05
    pts_feat[:, 8] = 1.0
    pts_feat[:, 9:24] = 0.0
    pts_feat[:, 24] = 1.0
    pts_feat[:, 25:40] = 0.0
    pts_feat[:, 40] = 1.0
    pts_feat[:, 41:56] = 0.0
    pts_feat.requires_grad_(True)

    c2w = torch.eye(4,device='cuda')
    HW = [1080//64*16,1920//64*16]
    intrin = torch.Tensor([[100,0,200],[0,100,200],[0,0,1]]).cuda()
    iteration = 0
    import tqdm
    optimizer = torch.optim.Adam([pts,pts_feat],lr=0.01)
    for ii in tqdm.trange(100000):
        optimizer.zero_grad()
        res=render(pts, pts_feat, c2w, intrin, HW)
        mask = res[1]>0
        loss = (res[1]-3).abs()[mask].mean()
        loss.backward()
        optimizer.step()
        if ii%500==0:
            #save_img(res[1])
            #plot3d(pts)
            print(loss)

Thanks! @chenyuntc , If you further prove that the geometry supervision for $\alpha$ works(either by similar test code or your own experiment), please notify me or @yanzhoupan , and we will try to merge this branch. Also, can you put your plot code in a separate directory in the repo? e.g. scratch or experiment(you can just create one).

Hello @wanmeihuali , while I try to test the supervision  for $\alpha$, I found it difficult to converge only optimizing the alpha.
I derive the gradient on my own, it seems different from the one you showed.
Given
$$
T_i=\prod_{j=1}^{i-1} (1-\alpha_{j})\
$$
We have
$$
\frac{d_{T_i}}{d_{\alpha_{k}} } = -T_i / (1-\alpha_k))
$$
Let:
$$
f={\sum_{i=1}^{n}{d_{i} \alpha_{i} T_i}}, g={\sum_{i=1}^{n}{\alpha_{i} T_i}}
$$
we convert $d_{pixel}=f/g$
$$
d_{pixel}=\frac{\sum_{i=1}^{n}{d_{i} \alpha_{i} \prod_{j=1}^{i-1} (1-\alpha_{j})}}{\sum_{i=1}^{n}{\alpha_{i} \prod_{j=1}^{i-1} (1-\alpha_{j})}}=\frac{\sum_{i=1}^{n}{d_{i} \alpha_{i} T_i}}{\sum_{i=1}^{n}{\alpha_{i} T_i}} = \frac f g \
$$
We have
$$
d' \propto (f'g -g'f)
$$
$$
f'=\frac {df}{d{\alpha_k}} = d_kT_k + \sum_{i=K+1}^n d_i\alpha_i \frac{-T_i}{1-\alpha_k}
$$
$$
g'=\frac {dg}{d{\alpha_k}} =  T_k + \sum_{i=K+1}^n \alpha_i \frac{-T_i}{1-\alpha_k} \
$$
that is
$$
d' = (d_kT_k + \sum_{i=K+1}^n d_i\alpha_i \frac{-T_i}{1-\alpha_k})g - (T_k + \sum_{i=K+1}^n \alpha_i \frac{-T_i}{1-\alpha_k})f
$$
$$
\frac {d'}{g} = (d_kT_k + \sum_{i=K+1}^n d_i\alpha_i \frac{-T_i}{1-\alpha_k})  - (T_k + \sum_{i=K+1}^n \alpha_i \frac{-T_i}{1-\alpha_k})\frac f g \
$$
Because $d_{pixel}=f/g$
$$
\frac {d'}{g} =T_k(d_k-d_{pixel}) +  \sum_{i=K+1}^n  \alpha_i \frac{-T_i}{1-\alpha_k} (d_i - d_{pixel})
$$
This equation is more intuitive to me as I can see the gradient is relative to the $(d_i - d_{pixel})$
I haven't done the calculus for a while, not sure correct or not, but this result is more intuitive to me as I can see the gradient is relative to the $(d_i - d_{pixel})$

Hi @chenyuntc , I see what happens. In my calculation, I treat $\sum_{i=1}^{n}{\alpha_{i} T_i}$ as a constant, but it is in fact a variable related to $\alpha$. So my calculation may not converge.
In your calculation, I'm wondering why are you calculating $\frac{d'}{g}$, I think that means $(\frac{1}{g}\frac{\partial{d_{pixel}}}{\partial{\alpha_k}})$? What you need shall be $\frac{\partial{d_{pixel}}}{\partial{\alpha_k}}$. And
$$
\frac{\partial{d_{pixel}}}{\partial{\alpha_k}}=\frac{\partial{\frac{f}{g}}}{\partial{\alpha_k}}=\frac{g\frac{\partial{f}}{\partial{\alpha_k}}-f\frac{\partial{g}}{\partial{\alpha_k}}}{g^2}=\frac{d_kT_k + \sum_{i=K+1}^n d_i\alpha_i \frac{-T_i}{1-\alpha_k}}{g}  - (T_k + \sum_{i=K+1}^n \alpha_i \frac{-T_i}{1-\alpha_k})\frac{f}{g^2}
$$
Which is actually your result divided by g... Anyway, have you tried your equation? Maybe the calculation does not need to be mathematically correct, it's more like designing the loss. I mean, As far as the gradient penalize the alpha for points far from $d_{pixel}$ and award the alpha for points close to $d_{pixel}$, it shall work.

Yes, your are correct,  The $\frac {d'} {g}$ is actually $\frac {\partial d_{pixel}} {\partial \alpha_k} *g$, because I use approximation $d'=\frac {(f'g -g'f)}{g^2} \propto (f'g -g'f)$
I haven't try the implementation in taichi yet as I'm not familiar with taichi. Would be appreciated if you can try it when available

Hello @wanmeihuali ,

while I'm going over the existing implementation,  I have one question:   The w_i in code  ,  it seems to be
$$w_i = \sum_{j=1}^{i}{ {color_j \alpha_j T_j}}$$
rather than  $$w_i = \sum_{j=i+1}^{n}{\frac{color_j \alpha_j T_j}{1-\alpha_i}}$$
as you mentioned (note the supscript is $i$ instead of $n$), Right? Unless it's inverse order?
You may ignore this one, if it's inverse order. What I did here is trying to figure out how to implement it in the forward-order.   I note that

$$
\sum_{i=K+1}^{n} \alpha_i T_i = g_n-g_k
$$
$$
\sum_{i=K+1}^{n} \alpha_i T_i d_i = f_n-f_k
$$
where $f_k={\sum_{i=1}^{k}{d_{i} \alpha_{i} T_i}}, g={\sum_{i=1}^{k}{\alpha_{i} T_i}}$,  This can help simplified the gradient to be
$$
\frac {\partial d_{pixel}} {\partial \alpha_k}  = \frac{T_k(d_k-d_{pixel}) +  \frac 1 {1-\alpha_k}[ (f_n-f_k)  - (g_n-g_k)d_{pixel}])} {g_n}
$$
That means we need to store pixel_accumulated_alpha  ($g_n$) and accumulated_depth ($f_n$ ?)  and rasterized_depth ($d_{pixel}$? } ) for backward, and compute the $f_k$ and $g_k$ in the for-loop.
Not sure whether it can help simplify the implementation :thinking:

I can check that when I go back to home tonight. Anyway, backward is in reverse order(from n to 1)

Thank you so much!

Hello @wanmeihuali , I created a PR https://github.com/wanmeihuali/taichi_3d_gaussian_splatting/pull/126 for this,  any feedbacks would be appreciated

Nice work @chenyuntc ! Just see your experiment, the findings seem very reasonable. If there are many points, the rasterizer will stop very early, and only the first several points met by the ray will be optimized. You can try more steps to see if more points get optimized. Also, during real training, points with small alpha will periodically be removed. So I think this issue won't affect results too much during real training.
The next step is to test the code on some more complex case, e.g. more complex object rather than a wall. Then finally test it on a real dataset, I'm not sure if geometry supervision works(unless you get very accurate depth measurements, e.g. from lidar), but I guess letting points converge to the depth estimated(the surface) can help improve training quality.

You can try more steps to see if more points get optimized.

Do you mean more training iterations? I tried for 100K iterations, but the loss did not change at all after 10K iterations.   I

Also, during real training, points with small alpha will periodically be removed.

Actually I also tried removing pts with low-alpha, but still not worked.
I find it's extremely difficult to make it work without  low-alpha initialization
For example, if I initial pts_feat[:, 7] = -4 instead of -5,  the loss decrease in the beginning but then soon saturated.  (it also depends on the number of points, with more points, the initial value need to be smaller)

I'm not sure if geometry supervision works(unless you get very accurate depth measurements, e.g. from lidar),

Based on my previous experience on NeRF, depth supervision  (from lidar) is super important for the sparse observations in urban-driving scenes.  I will let you know if I can make it work in the urban-driving scenes.

Maybe we can first try some easier solution. e.g.
given a list of points through which a ray passes, use some threshold to split the points into three groups: from camera to area close to the estimated/provided depth, points in the area close to the estimated/provided depth, and points deeper than estimated/provided depth. All we need to do is to penalize the first group, award the second group, and ignore the third group. The penalize/award amount shall also be proportional to the color weight($\alpha_i T_i$). Because we are using Gradient Desend. As long as the direction is correct, it shall converge to the estimated/provided depth, the gradient(optimization direction) does not need to be mathematically correct.

Hi @wanmeihuali
Are you open to exploring new career options? If you're interested, I'd be happy to email you a relevant job opportunity link.
my email is chenyuntc at  gmail


Interface to Refract Properties

Refract has three properties:

element
meta
attributes
content

What should the interface to these values be on an element? With #25 I've introduced meta as a special class where each property is a Minim instance. Additionally, with #27 I've introduced elementName instance method and removed the element.

Should elementName be accessible through a getter for element?

Should content be a Minim Collection?

Note, as of now, element is taken care of. It is a getter/setter.

Considering this done for now. Not going to address content as a Minim array at this point.


inv/values refactor

Beginning of a refactor to transfer values between nodes in bulk messages using the inventory/getdata format instead of individual store messages. Should improve efficiency and better enable ban score calculations in the future.

LGTM


OAuth2Client Specified key was too long; max key length is 767 bytes

After updating to 8.5.0 and importing the database on the live server. The error Specified key was too long; max key length is 767 bytes  is thrown.
The index fields  clientKey and clientSecret in the file public/concrete/src/Entity/OAuth/Client.php need to be set to a collation of utf8_general_ci as done in #7221 .
Error
CREATE TABLE IF NOT EXISTS OAuth2Client (
identifier char(36) COLLATE utf8mb4_unicode_ci NOT NULL COMMENT '(DC2Type:guid)',
name varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
redirectUri varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
clientKey varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
clientSecret varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
consentType int(10) unsigned NOT NULL,
PRIMARY KEY (identifier),
UNIQUE KEY client_idx (clientKey,clientSecret)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
MySQL meldt: Documentatie
#1071 - Specified key was too long; max key length is 767 bytes

The reason is probably because your local MySQL/MariaDB version is newer than the live one.
When concrete5 detects a recent MySQL version, it uses the utf8mb4 character set (which has support to the whole range of Unicode characters).
Previous versions of MySQL doesn't support it, and concrete5 configures the database to use utf8 instead of utf8mb4.
You can configure your local concrete5 to use utf8 instead of utf8mb4 by running this CLI command:
./concrete5/bin/concrete5 c5:database:charset:set utf8_unicode_ci

@rikzuiderlicht #7221 has been reverted by https://github.com/concrete5/concrete5/commit/390718ba9e4bf3c73bc7198fc86e7e041480e01a
concrete5 uses the same character set/collation for all the database tables and fields (which are utf8mb4/utf8mb4_unicode_ci if mysql supports them, or utf8/utf8_unicode_ci otherwise)

The reason is probably because your local MySQL/MariaDB version is newer than the live one. When concrete5 detects a recent MySQL version, it uses the utf8mb4 character set (which has support to the whole range of Unicode characters). Previous versions of MySQL doesn't support it, and concrete5 configures the database to use utf8 instead of utf8mb4.
You can configure your local concrete5 to use utf8 instead of utf8mb4 by running this CLI command: ./concrete5/bin/concrete5 c5:database:charset:set utf8_unicode_ci

File not found at this location for me, need to drop the 5 from the intial folder e.g. ./concrete/bin/concrete5 c5:database:charset:set utf8_unicode_ci


Updated version Osprey

Dear,
The last few months I continued using Osprey for the analysis of my spectro data and made some improvements to the code. I herby share the changes I made.
First of all, I solved some problems with the GE nifti files and the Matlab path. Secondly, I made the program a little more user-friendly by:

Adding the buttons from the start screen (Load Job file, Load MRSCont,…) to the interactive screen
In the interactive screen, you can upload a JobFile. The fields will be updated with the values specified in the JobFile
When start Osprey with the Matlab command ‘Osprey(‘start’)’, the program directly opens in the interactive screen rather than in the start screen with the buttons. To avoid too much interference with what you are doing, you can still start Osprey in the normal start screen with the command ‘Osprey’.
I added a button ’New analysis’ to the main screen. This way you can start a new analysis without the need to first close and restart the program.
As most users don’t save their data in a program folder (not recommendable) but somewhere on a data disk or server, I changed the default folder for data selection to the working directory set in the main Matlab screen rather than the ‘Osprey/exampledata’ folder. As such, data selection is more easy if the study data is not stored in the exampledata folder by setting the Matlab’s working directory to the datafolder.

Thirdly, I added the analysis of spectro data taken from the Braino phantom to the program. However, today, it will only work for PRESS data taken on a GE scanner, since that is the only data I have. For this, I added a parameter ‘MRS Protocol’. When setting this parameter to ‘Braino phantom’, the program will change the other parameters like ‘Reference frequency’ and ‘Selected Metabolites’ to the default values for phantom data.
At last, I added the GE’s newest pfile version 28.002 to GEload.
I tried to update my version with your changes and to update conflicts, but please check yourself.
Kind regards,
Peter

Dear Peter,
Thanks for sharing your work on Osprey with us. Your continuous contributions are most appreciated!
I've redirected your commit into its own branch for further testing before merging it into the main develop branch.
Best,
Helge

Dear Peter,
Thanks for sharing your work on Osprey with us. Your continuous contributions are most appreciated!
I've redirected your commit into its own branch for further testing before merging it into the main develop branch.
Best,
Helge

Dear Peter (@P-VS) ,
I've been testing your contributions. I do have some questions about your contributions:

CreateOspreyJob.mlapp and CreateOspreyJob.m seem to be inconsistent or at least I can not start the app on my machine, because of the osprey image. It seems to be defined in the .m file, but not in the .mlapp file. Maybe you need to push the .mlapp file again. I was able to resolve the problem by commenting the image part out for now.

Could you elaborate a bit more on your braino protocol flag. The reason I'm asking is that we do already have a phantom data scenario flag. This flag changes the referencing step, adds some line broadening , and changes the HSVD filter. What was your reasoning to not opt for the phantom switch? I'd like to streamline the phantom processing and I could integrate your changes into the normal phantom flag. In this case, I'd add braino phantom as a metabolite list option instead of adding the new protocol flag to reduce the number of flags in the MRS container.

The rest of your changes work like a charm!
Thanks for your contributions,
Helge

Dear Peter (@P-VS) ,
I've been testing your contributions. I do have some questions about your contributions:

CreateOspreyJob.mlapp and CreateOspreyJob.m seem to be inconsistent or at least I can not start the app on my machine, because of the osprey image. It seems to be defined in the .m file, but not in the .mlapp file. Maybe you need to push the .mlapp file again. I was able to resolve the problem by commenting the image part out for now.

Could you elaborate a bit more on your braino protocol flag. The reason I'm asking is that we do already have a phantom data scenario flag. This flag changes the referencing step, adds some line broadening , and changes the HSVD filter. What was your reasoning to not opt for the phantom switch? I'd like to streamline the phantom processing and I could integrate your changes into the normal phantom flag. In this case, I'd add braino phantom as a metabolite list option instead of adding the new protocol flag to reduce the number of flags in the MRS container.

The rest of your changes work like a charm!
Thanks for your contributions,
Helge

Dear Helge,
I removed the image and created a new .mlapp and .m file. You find the updated files in my own copy of Osprey: https://github.com/P-VS/osprey/tree/develop/job
The reasoning that I added the braino protocol tag was that I encountered problems when I tried to analyse my phantom data. The program tried to analyse my data as brain data and since your tag wasn't present in the example job.m file I used, I defined a new tag. Later on, when I found your tag appearing in the code, I still got wrong results. Since I am hesitant to make changes in the code that can interfere with what other contributors are doing, I kept using my own flag for this version. Combining both flags and their use in a later stage seems logic of course.
Separating the processing streamline for phantoms from those from brain data as much as possible seemed to be the best approach to solve it. First of all because it is not only a matter of shifting the reference frequency, changing the filter settings and changing the metabolite list. In my code, the flag already changes the reading of the pfile (e.g. the definition of the ppm axis). I had to make new reference metabolite spectra simulations because the relative frequency shifts of the individual metabolites is different from those in brain spectra. Moreover, to my experience from previous studies, likely, also the T1 and T2 values of the metabolites used for the metabolite concentration quantifications differ between phantom and brain signals. But it is still on my to do list to check that with measured data. So, processing phantom data differs from the first till the last step and defining it as a separate protocol feels more natural. Secondly, by defining it as a separate protocol for this particular phantom, it will be more easy to add different protocols for other phantoms with different metabolite compositions (e.g. the MRS phantoms from Philips or Siemens) to the program.
Kind regards,
Peter

Dear Helge,
I removed the image and created a new .mlapp and .m file. You find the updated files in my own copy of Osprey: https://github.com/P-VS/osprey/tree/develop/job
The reasoning that I added the braino protocol tag was that I encountered problems when I tried to analyse my phantom data. The program tried to analyse my data as brain data and since your tag wasn't present in the example job.m file I used, I defined a new tag. Later on, when I found your tag appearing in the code, I still got wrong results. Since I am hesitant to make changes in the code that can interfere with what other contributors are doing, I kept using my own flag for this version. Combining both flags and their use in a later stage seems logic of course.
Separating the processing streamline for phantoms from those from brain data as much as possible seemed to be the best approach to solve it. First of all because it is not only a matter of shifting the reference frequency, changing the filter settings and changing the metabolite list. In my code, the flag already changes the reading of the pfile (e.g. the definition of the ppm axis). I had to make new reference metabolite spectra simulations because the relative frequency shifts of the individual metabolites is different from those in brain spectra. Moreover, to my experience from previous studies, likely, also the T1 and T2 values of the metabolites used for the metabolite concentration quantifications differ between phantom and brain signals. But it is still on my to do list to check that with measured data. So, processing phantom data differs from the first till the last step and defining it as a separate protocol feels more natural. Secondly, by defining it as a separate protocol for this particular phantom, it will be more easy to add different protocols for other phantoms with different metabolite compositions (e.g. the MRS phantoms from Philips or Siemens) to the program.
Kind regards,
Peter


Add brew dev as an alias for brew developer

Provide a detailed description of the proposed feature
Add brew dev as an alias for brew developer.
What is the motivation for the feature?
It would make it easier to type and I feel like it's pretty intuitive. The dev subcommand is currently unused.
How will the feature be relevant to at least 90% of Homebrew users?
It will probably only be used by those who decide to contribute to the project.
What alternatives to the feature have been considered?
Keeping it the way it is.

I suppose we could, though are you actually running the command that often?

     Is tab completion or using Homebrew Aliases insufficient here?  That's unexpected.

Oh, yeah. Homebrew Aliases works perfectly. Thanks


How to support multiple/dynamic store URLs in successRedirect?

The plugin requires the following configuration:
options: {
      auth0: {
        clientID: Auth0ClientId,
        clientSecret: Auth0ClientSecret,
        auth0Domain: Auth0Domain,

        admin: {
          callbackUrl: `${BACKEND_URL}/admin/auth/auth0/cb`,
          failureRedirect: `${ADMIN_URL}/login`,
          successRedirect: `${ADMIN_URL}/`,
          // authPath: '/admin/auth/auth0',
          // authCallbackPath: '/admin/auth/auth0/cb',
          // expiresIn: 24 * 60 * 60 * 1000,
          // verifyCallback: (container, req, accessToken, refreshToken, extraParams, profile) => {
          // implement your custom verify callback here if you need it
          //    // console.log(container, req, accessToken, refreshToken, extraParams, profile)
          // }
        },

      store: {
          callbackUrl: `${BACKEND_URL}/admin/auth/auth0/cb`,
          failureRedirect: `${STORE_URL}/login`,
          successRedirect: `${STORE_URL}/`,
          // authPath: '/admin/auth/auth0',
          // authCallbackPath: '/admin/auth/auth0/cb',
          // expiresIn: 24 * 60 * 60 * 1000,
          // verifyCallback: (container, req, accessToken, refreshToken, extraParams, profile) => {
          // implement your custom verify callback here if you need it
          //    // console.log(container, req, accessToken, refreshToken, extraParams, profile)
          // }
        },
}

Is it possible to support multiple successRedirect so that we can work with Vercel Preview environments for the store? Or a way to dynamically set this on a route handler?
Of course for production we'll have a single one and that's the most secure option, but for preview environments we don't really care about this and it makes development/testing much easier if we are not constrained to a single redirect URL.

How would you see it working? That you can pass a method that gets resolved and the result is the preview url to redirect to?

Alright, that sounds good. I ll integrate that soon

@eparizzi-clevertech you should be able to install the lates version that works with the latest medusa and support the dynamic redirectTo


Fix warnings in macOS sample app in Xcode 9.3

Changes in this pull request
Issue fixed: #1057
Checklist

[x] All tests pass. Demo project builds and runs.
[x] I added tests, an experiment, or detailed why my change isn't tested.
[ ] I added an entry to the CHANGELOG.md for any breaking changes, enhancements, or bug fixes.
[x] I have reviewed the contributing guide

Hello again everyone! I was having a play with integrating Realm with IGListKit again and I noticed this issue detailing the macOS app featuring several build warnings.
This PR:

Updates the settings of both the macOS example project, and the Pods project to the Xcode 9.3 defaults.
Removes any deprecated Swift APIs.
Updates some variable names to satisfy swiftlint's constraint that names should be >= 4 characters.

Please let me know if I need to change/add anything else. Thanks! :)

Welcome back @TimOliver! 😍
It looks like there are some build failures. I’m on mobile so haven’t dug into them yet. Mind taking a look?
Sent with GitHawk

G'day again @rnystrom! I hope you've been well! Thanks! I've been getting things set up slowly over here. Hopefully I can help out a bit more down the line!
Thanks! I noticed that too. It seemed to be only one device class test failing (iPad Air 2 running iOS 9.3) so I was guessing it wasn't related to my PR.
I just had a deep dive. It turns out the build is failing because of a build warning in IGListAdapter in the pod lint process.

I'm guessing the assert is being omitted in the build process before the pod lint is performed. Probably the easiest way to fix it would be to directly check if self.collectionView != nil without creating that variable, but I'm wondering if that was done for a reason.
Thanks!

Awesome, fixed in #1188, thanks for diving!


v1.26.0 CSI and CPI chart image versions

https://github.com/rancher/image-mirror/pull/362 is a Prerequisite to this PR
Pull Request Checklist

[X] Any new images or tags consumed by charts has been added here
[X] Chart version has been incremented (if necessary)
[X] That helm lint and pack run successfully on the chart.
[ ] Deployment of the chart has been tested and verified that it functions as expected.
[ ] Changes to scripting or CI config have been tested to the best of your ability

Types of Change
Added new Image version
CSI  2.7.1, 3.0.1
CPI 1.26.0 (note k8s v1.27.0 will pin to 1.26.0 as that is the latest release ATM)

Update image version:
CSI 2.6.2 -> 2.6.3
CSI 2.7.0 -> 2.7.1

Linked Issues

Additional Notes

After the PR is merged
Once the PR is merged, typically upon a new release, the necessary teams will be notified via Slack hook to perform the RKE2 Charts and RKE2 changes. Any developer working on this issue is not responsible for updating RKE2 Charts or RKE2.

Sounds good to me, I wasn't sure on the policy around the defaults.

@a-blender After several days of fighting vsphere, I have been unable to validate my changes and could use some help. I am not familiar with the rancher + vsphere setup process, so this has been quite the learning experience. I was able to provision a rke2 cluster with the external cloud provider via Rancher and then manually attempted to install the CSI/CPI charts. I filled out the values.yaml with the Fremont datacenter info, as described in the guide, and the chart seems to deploy just fine
root@rke2-try4-pool1-be5e6385-gtzr9:/home/ubuntu/vsphere-charts/charts# helm install -f ./rancher-vsphere-csi/values.yaml --kubeconfig=/etc/rancher/rke2/rke2.yaml csi ./rancher-vsphere-csi/
NAME: csi
LAST DEPLOYED: Tue May  9 17:24:31 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None

However, the taint does not go away, and many pods are still pending.
root@rke2-try4-pool1-be5e6385-gtzr9:/home/ubuntu/vsphere-charts/charts# kubectl get pods -A
NAMESPACE         NAME                                                     READY   STATUS             RESTARTS        AGE
calico-system     calico-kube-controllers-687bc88ddf-dbl52                 0/1     Pending            0               53m
calico-system     calico-node-4kghk                                        0/1     Running            0               53m
calico-system     calico-typha-689874cc98-5hw4f                            0/1     Pending            0               53m
cattle-system     cattle-cluster-agent-6c6c756b7c-jxglx                    0/1     Pending            0               54m
default           rancher-vsphere-cpi-cloud-controller-manager-cbwkz       0/1     CrashLoopBackOff   6 (2m11s ago)   8m21s
default           vsphere-csi-controller-7cd544bdc9-dxlcp                  0/5     Pending            0               4m52s
default           vsphere-csi-controller-7cd544bdc9-g56gr                  0/5     Pending            0               4m52s
default           vsphere-csi-controller-7cd544bdc9-skc29                  0/5     Pending            0               4m53s

Help validating this would be much appreciated!

@dereknola Both charts deploy fine on a vsphere rke single node downstream cluster for me. CSI pods run successfully, CPI removed the taint from the node. I've checked the logs as well to make sure everything looks normal for the 1.26 images, I will merge this PR.


Documentation: Breaking The Monolith

Amplication will soon have a new project-level tab titled Architecture that will allow users to create a microservice architecture for their projects more easily. It will do this by letting users move entities between services and connecting them in a microservice architecture.
There will also be an AI Helper button that will give recommendations to users on which entities to place in which services. Users will then be able to accept these suggestions, make modifications to the suggestions, or discard them.
The documentation should clearly explain how all aspects of this page work.

@PazYanoverr


fixes bug with waitForBalance signaling

The code that signals no balancing happened would sometimes skip balancing for user tablets and still signal no balancing happened. Modified to only signal when balancing had run on all levels.

Ah, I see. It would skip balancing user tables where there are unhosted tablets, then signal that everything was balanced.

FYI that WaitForBalanceIT is still failing in main


[#5321] Operate a guard for the provided application key from the url

Navigate them to the main page if the provided application key is non-existent

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Codecov Report

Merging #5322 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5322      +/-   ##
==========================================
+ Coverage   40.91%   40.91%   +<.01%
==========================================
  Files        2712     2712
  Lines       82292    82292
  Branches    10987    10987
==========================================
+ Hits        33669    33670       +1
+ Misses      45664    45663       -1
  Partials     2959     2959

Impacted Files
Coverage Δ

...nitor/DefaultDataSourceMonitorRegistryService.java
77.77% <0%> (-11.12%)
:arrow_down:

...corp/pinpoint/rpc/stream/StreamChannelManager.java
70.68% <0%> (-1.73%)
:arrow_down:

...point/rpc/client/DefaultPinpointClientHandler.java
72.61% <0%> (+0.31%)
:arrow_up:

...om/navercorp/pinpoint/rpc/codec/PacketDecoder.java
63.82% <0%> (+2.12%)
:arrow_up:

...ofiler/sender/StandbySpanStreamDataSendWorker.java
83.58% <0%> (+2.98%)
:arrow_up:

...rc/main/java/com/navercorp/pinpoint/test/Item.java
73.33% <0%> (+3.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b0a7b6a...454609f. Read the comment docs.


Uncaught Error: tunneling socket could not be established, cause=connect ECONNREFUSED <IP_ADDRESS>:80

[Enter steps to reproduce:]

I restarted Atom after installing the atom-beautify package
...

Atom: 1.24.1 x64
Electron: 1.6.16
OS: Mac OS X 10.12.6
Thrown From: Atom Core
Stack Trace
Uncaught Error: tunneling socket could not be established, cause=connect ECONNREFUSED <IP_ADDRESS>:80
At stream.js:74

Error: tunneling socket could not be established, cause=connect ECONNREFUSED <IP_ADDRESS>:80
    at ClientRequest.onError (/app.asar/node_modules/tunnel-agent/index.js:177:17)
    at Object.onceWrapper (events.js:290:19)
    at emitOne (events.js:96:13)
    at ClientRequest.emit (events.js:188:7)
    at Socket.socketErrorListener (_http_client.js:309:9)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at emitErrorNT (net.js:1281:8)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)

Commands

Non-Core Packages
atom-beautify 0.32.2
auto-update-packages 1.0.1
better-git-blame 0.4.0
busy-signal 1.4.3
file-icons 2.1.17
git-blame 1.6.0
git-plus 7.10.0
git-time-machine 1.5.9
gitlab 0.1.3-7
highlight-selected 0.13.1
intentions 1.1.5
linter 2.2.0
linter-eslint 8.4.1
linter-scss-lint 3.1.1
linter-ui-default 1.7.1
minimap 4.29.8
minimap-find-and-replace 4.5.2
minimap-git-diff 4.3.1
minimap-highlight-selected 4.6.1
open-recent 5.0.0
pigments 0.40.2
platformio-ide-terminal 2.8.0
split-diff 1.5.2

lmao

@InsertPunHere1 your comment was deleted as a violation of the Atom Code of Conduct as it was insulting or derogatory.

It only happened the one time. It just seemed like an odd thing to happen.

Yeah it could have just been a temporary network issue - thanks for confirming it happened the one time only.  We'll go ahead and close since it's not a reproducible issue.


FormData not being sent as body

Hello,
Sending formData as a request body using the Angular2TokenServicedoes not add anything to the body and simply sends an empty body.
Appending form data:
    add() {
        let formData: any = new FormData();
        formData.append('article[title]', this.article.title);
        formData.append('article[description]', this.article.description);
        formData.append('article[author]', this.article.author);
        if (this.article.image) {
            formData.append('article[image]', this.article.image);
        }

        this.httpService.sendData('articles', formData).subscribe((data: Response) => {
            this.router.navigate(['admin/articles']);
        });

Sending it to the server:
  sendData(url : string, data: any): Observable<any> {
    return this.tokenService.request({
      method: RequestMethod.Post,
      url: `http://localhost:3000/${url}`,
      body: data // <- Contains FormData
    });

How can I make this work? I'm using form-data primarily because I'm sending images, so using JSON won't work.
Thanks in advance!

+1 I'm trying to do the exact same thing.  I've been trying to hack https://github.com/valor-software/ng2-file-upload to work but since responses don't update my token, I basically get logged out every time I upload a file.

I made a workaround. I will update you on it later today

@Kaizeras were you able to get this working?

@davidjconnolly I  tried using ngx-uploader, which works if you set   config.change_headers_on_each_request = false, but thats a security risk!
UploadInput = {
            type:   'uploadAll',
            url:    environment.apiUrl + '/' + this.url,
            method: 'PUT',
            data: { foo: 'bar' },
            headers: {
                'token-type':   'Bearer',
                'access-token': this._tokenService.currentAuthData.accessToken,
                'client':       this._tokenService.currentAuthData.client,
                'expiry':       this._tokenService.currentAuthData.expiry,
                'uid':          this._tokenService.currentAuthData.uid
            },
            concurrency: 1
        }

Yes, I made an easy workaround:
Given that you want to make some requests to be multipart/form-data, not all of them, you can extend the Angular2TokenService and initialize it with different globalOptions for a particular request:

@Injectable()
export class FormDataService extends Angular2TokenService {

    addItem(data: FormData) {
        this.init({
            apiBase: 'http://localhost:3000',
            globalOptions: {
                headers: {
                    'Content-Type': 'multipart/form-data',  // <-- Set mutlipart/form-data only for this request
                    'Accept': 'multipart/form-data', // <-- Set mutlipart/form-data only for this request
                    "access_token_name": localStorage.getItem('accessToken'),
                    "client_name": localStorage.getItem('client'),
                    "uid_name": localStorage.getItem('uid'),
                    "expiry_name": localStorage.getItem('expiry'),
                    "token-type_name": 'Bearer'
                }
            }
        });
        let hdrs = this.currentAuthHeaders;
        hdrs.append('enctype', "multipart/form-data")
        let requestOptions = new RequestOptions({
            method: RequestMethod.Post,
            headers: hdrs,
            url: 'http://localhost:3000/articles',
            body: data
        });
        return this.request(requestOptions).map(
            (response) => response.json()
        );
    }

You can also move the init to a separate method call.
If you do it this way, the options that are set globally are done with application/json request headers, while the methods that have to send files are done with multipart/form-data requests.


Requesting to Join Maintainers

Open/Merged PRs
I'd like to join the maintainers to help Gitea and other FOSS projects flourish by providing software to assist in the development process.

Codecov Report

Merging #7541 into master will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #7541      +/-   ##
==========================================
+ Coverage   41.23%   41.25%   +0.02%
==========================================
  Files         469      469
  Lines       63620    63620
==========================================
+ Hits        26232    26246      +14
+ Misses      33965    33950      -15
- Partials     3423     3424       +1

Impacted Files
Coverage Δ

routers/repo/view.go
43.25% <0%> (+1.01%)
:arrow_up:

modules/git/git.go
37.34% <0%> (+3.61%)
:arrow_up:

models/unit.go
67.56% <0%> (+5.4%)
:arrow_up:

modules/setting/git.go
65.38% <0%> (+19.23%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e0d6d2f...6b6eca0. Read the comment docs.

@gary-kim welcome aboard! GitHub should have just sent you an email with a link to accept an invitation into this org. I will also add maintainer permissions to Gary#6885 Discord account (I assume that is you). Please also create an account on https://gitea.com and let me know your username, this is because we will soon use Gitea itself to develop gitea.

Welcome! :tada:


use FolioClient to fetch Acquisition unit name for display in the vendor UI

Currently, the vendor list page is displaying the AcqUnitId instead of the name, use the FolioClient to translate the ID into the display name and update the UI to show the name.
See: https://s3.amazonaws.com/foliodocs/api/mod-orders/p/acquisitions-units.html#acquisitions_units_units__id__get

@aaron-collier I've been trying different approaches to this (making the folio call right before rendering the template, overriding the AirflowPlugin's on_load method, or looking at ways to tie into the blueprint lifecycle) but they are either unsatisfactory in slowing the page load or add a lot of complexity to set-up and testing. Since this is just to get the acq unit names and I don't think those change (at all? rarely?) I think I'm just going to add this as a database field and figure out how to backfill. Let me know if you have concerns with that approach.


refactor(getCitations.js): add null check for adaptiveCards variabl…

…e before accessing its properties

thank you!

closes #231