[$500] mWeb/Chrome-Contact methods-Confirmation window moves when trying to remove secondary contact method

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: v1.4.5-7
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause-Internal Team
Slack conversation:
Action Performed:

Go to URL https://staging.new.expensify.com/
Login with any account
Go to Settings - Profile - Contact method - New contract method
Add a new contract method, but not validate
Open just added contact method
When the keyboard opens, tap on the remove button

Expected Result:
Confirmation window not moves
Actual Result:
Confirmation window moves
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[x] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
https://github.com/Expensify/App/assets/115492554/3a553cef-285c-4564-84d5-5281d1889f28

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~010d076df7ad5db3f2
Upwork Job ID:<PHONE_NUMBER>411456512
Last Price Increase: 2023-11-30

Proposal
Please re-state the problem that we are trying to solve in this issue.
Confirmation window moves
What is the root cause of that problem?
Because keyboard is closed During show the modal.
What changes do you think we should make in order to solve the problem?
we need update like this.
toggleDeleteModal(isOpen) {
        Keyboard.dismiss();
        InteractionManager.runAfterInteractions(() => {
                 this.setState({isDeleteModalOpen: isOpen});
        });
}

https://github.com/Expensify/App/blob/e2f421491cfe87878a21a212b927b1bd9107cfc6/src/pages/settings/Profile/Contacts/ContactMethodDetailsPage.js#L211-L215
What alternative solutions did you explore? (Optional)

Hey @allroundexperts we've got a few proposals here for review -- can you take a look please?

I think this is a trivial case which can be solved in a relatively simple way by using InteractionManager. Since its being used elsewhere as well, I would prefer this over other solutions which are more complex.
Since @dukenv0307 was the first to propose the above, I say we go with there proposal.
The assigned internal engineer can double confirm this though.
🎀 👀 🎀 C+ reviewed

Hey @grgia, can you help us confirm this comment?

I think this is a trivial case which can be solved in a relatively simple way by using InteractionManager. Since its being used elsewhere as well, I would prefer this over other solutions which are more complex.
Since @dukenv0307 was the first to propose the above, I say we go with there https://github.com/Expensify/App/issues/32262#issuecomment-1833680475.
The assigned internal engineer can double confirm this though.

@allroundexperts @dukenv0307 proposal looks good to me, assigned!

PR is merged and undergoing QA!

@stephanieelliott I think this one's ready for payment, let me know if you need anything from me!

This is all paid!

@stephanieelliott there was regression which was not handled by original author / reviewer

If this doesn't apply penalty, along with https://github.com/Expensify/App/issues/33410#issuecomment-1866784975, I will take this for future reference.

ok so I'd like to confirm that original folks don't take responsible for follow-up issues caused by their PR if they're not counted as regression.


added my Hack web3 Build submission file

@shriyadindi It should be there in the PR.
So that After merging your image should be there in repo. Create a New PR or edit this


⚠ Bofrost.at has degraded performance

In 88fe925, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1728 ms

Resolved: Bofrost.at performance has improved in 9ac85de after 309 days, 21 hours, 51 minutes.


Align build with react/solid/… instead of svelte | fix default error typing | add simple example

Hi :) Thanks for chiming in making Lit compatible with React Query!
This fixes build, so the root nx command, build:all, can succeed.

Added a working "lit simple" example + fixed the error typing. It looks like react-query is using Error | null instead of unknown.

Added a mutation observer controller, with correct typings (from react-query etc.)
Build is OK.
A very quick runtime test was done, at least onSuccess seems to work.
@customElement('child-element')
export class ChildElement extends LitElement {
  @state() todoId = 1

  private todoQuery = todoQueryController(this)

  private todoMutation = new MutationController(this, () => ({
    mutationFn: (newTodo: { id: string }) => {
      console.log('OK')

      return new Promise((r) => r({}))
    },
  }))

 render() {
    const { isPending, error, data, isFetching } = this.todoQuery.result

    if (error) return 'An error has occurred: ' + error.message
    this.todoMutation

    return html`

        <button
          @click=${() => {
            this.todoMutation.result.mutate(
              { id: 'string' },
              {
                onSuccess: () => {
                  console.log('OK')
                  queryClient.invalidateQueries({ queryKey: ['todo'] })
                },
              },
            )
          }}
        >
          MUTATE
        </button>
`
//...

}


Add migration to drop extra fields from permissions

Part of #7013 cleanup (1st step)
Note this migration does not reverse to the exact same database state, although it should be consistent.

Retest this, please

CR please, @juanignaciosl
I don't intend to merge this until some days pass with the current setup and we see no errors, as data will be dropped by this change. I do my best to recover on migration rollback, but it won't be perfect for unconsistent data.

When you rename or drop a column you must first tell ActiveRecord to forget it, since during the migration the servers that are still running will want it because of the inspection of the tables (ActiveRecord "remembers" the columns that the tables had at boot time). More details on this blogpost: https://blog.codeship.com/rails-migrations-zero-downtime/
You should first add this to Carto::Permission (please check it ;-) ):
  def self.columns
    super.reject { |c| ['entity_id', 'entity_type'].include?(c.name) }
  end

As a nice side effect this will check that the columns are not actually needed at all without deleting data.
cc @gfiorav , (I can't remember whether he's aware of this).

Thanks for pinging me! Nice to see this in context!

:+1: after my comment on disabling columns (which is a must) and the heads up on the deletion risk (up to you).

Good to know about ActiveRecord caching the schema. Saving the article.

Blocked by https://github.com/CartoDB/cartodb/pull/7098

#7098 merged. If nothing breaks, this is next.


Fix leaked ObjectRCs on object Variant reassignment

Bonus:

Add some (un``)likely magic
Use memdelete() instead of memfree() for ObjectRC  (not strictly needed, but more correct; hinted by @ThakeeNathees's comment on the bug report)

Fixes #39821.

Thanks!


Updating NumPy selector

We are rapidly approaching a NumPy 1.12 release. May want to update the selector so that it doesn't skip that accidentally.

Before we can do this, we need to make sure that our dependencies are built for numpy 1.12.

[x] scikit-learn (https://github.com/conda-forge/scikit-learn-feedstock/pull/33)
[ ] bottleneck (https://github.com/conda-forge/bottleneck-feedstock/pull/8)

Numpy 1.12 support was added in #21.


Types missing in Load Function

I have this endpoint:
src/routes/api/user/[user_id]/+server.ts
import type { RequestEvent } from './$types';
import { PrismaClient} from '@prisma/client';
import type { API } from 'sveltekit-zero-api';
import { NotFound, Ok, Unauthorized } from 'sveltekit-zero-api/http';

const prisma = new PrismaClient();

interface Get {
}

export async function GET(event: API<Get, RequestEvent>) {
	const { params, locals } = event;
	const session = await locals.getSession();

	if (!session || !session.user || session.user.id !== params.user_id) {
		return Unauthorized()
	}

	let user = await prisma.user.findFirst({
		where: {
			id: params.user_id
		},
		include: {
			profile: true,
			journals: {
				include: {
					entries: {
						where: {
							timestamp: {
								gte: new Date(new Date().toDateString())
							}
						},
						include: {
							ingredients: true
						}
					}
				}
			}
		}
	});

	if (!user) {
		return NotFound();
	}

	return Ok({
		body: user});
};

I'm calling it here:
src/routes/app/+page.server.ts
import { redirect } from '@sveltejs/kit';
import api from 'src/api'
import type { PageServerLoad } from './$types';

export const load: PageServerLoad = async ({ parent, fetch }) => {
	const { session } = await parent();
	if (!session?.user) throw redirect(303, '/auth');

	const response = await api.user.user_id$(session.user.id).GET({}, fetch)

	return {
		session: session,
		user: response.body
	};
};

the returned user: response.body object has an "any" type, how come?

Thanks for letting me know! Looks like I've made a mistake. I'll patch it ASAP

Hey!
Sorry, a slight overlook from my end actually, and I have to revert my commit again. My mind was somewhere else I suppose.
So what you see is actually intentional, and that's because when you await any given response, you might get a status 500, a status 400, 404 etc.
What ideally you should get, is
body = User | unknown
Or something like that.
Instead what you should do is return the body from an Ok response using $ (dollar symbol)
export const load: PageServerLoad = async ({ parent, fetch }) => {
	const { session } = await parent();
	if (!session?.user) throw redirect(303, '/auth');

	const user = await api.user.user_id$(session.user.id).GET({}, fetch).$.Ok(res => res.body)

	if(!user)
		return {} // The endpoint did not return Ok (200).

	return {
		session: session,
		user: response.body
	};
};

Sorry for my wrong previous response which was wrong. I'm going to reopen this issue until I reverted my last commit🤦


Make request methods generic over a new Client type

This has a few benefits:

Consumers of this crate are now capable of bringing their own HTTP Clients (so long as they produce futures that are Send)
Consumers can enable runtime support (tokio v async-std v sync) without being forced into a specific Client choice
Clients are no longer constructed for each request and can be re-used for multiple requests, meaning we can make use of a client's built in connection pooling instead of building new clients per request

Drawbacks:

Every method on Bucket now expects a &Client argument. This could be worked around by storing the client inside a bucket, but I personally prefer to keep them separate

re-using the same client between requests potentially helps this problem: https://github.com/durch/rust-s3/issues/212

Also

fixes: #152
fixes: #188

Very happy to see this being worked on

@asonix thanks for the PR, my bandwidth is a bit limited, but I'll try and get this merged soon-ish

Thanks @durch

Was this intended to be closed or was that a result of deleting the target branch

Its the target branch, lets remedy that

eagerly awaiting this being merged :crossed_fingers:

Closing for now, I'm not, branches have diverged too far, with all the features and different async backends this would be a tall order merge :/


(GH-161) Better async

Description

The GitReleaseManager is doing a lot of context switching when executing async methods.
Also calling .Result on Tasks is generally bad practice.
Related Issue

#161
Motivation and Context

Fixes bad practices when using Tasks. This PR tries to solve:

Stack traces being 💩 when there is an unhandled exception due to calling .Result on Tasks
Methods are context switching a lot, leading to potential issues, if thread a Task tries to switch back to is blocked, leading to dead-locks
Going async from entry point and all the way through the tool

How Has This Been Tested?

Ran the test suite and ran the CLI tool to verify it still works.
Screenshots (if appropriate):

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

Had to bump the test project to netcoreapp3.0 otherwise it wouldn't run. Complained about missing testhost.dll even with netcore 2.0 and net472 targeting pack installed.

@Cheesebaron thank you again for doing this!  I really appreciate it!


Expose appropriate docker run options to docker build

If a Dockerfile contains a RUN directive, it has to create a container of the current state of the image and then run the command in it.
I have hit an interesting case: I have configured the docker daemon to use the fluentd log driver and I run fluentd in a container. This is perfectly fine because I can manually specify --log-driver json-file when running my containers that have to be started before the fluentd container.
However, given my first statement, it is possible that if the log driver cannot initialize, you are unable to build images that contain RUN directives as docker build does not support the --log-driver option in order to override the daemon options and therefore the docker run to execute the RUN directive fails.
I don't know if there are also other options from docker run that should be exposed to docker build but it would be worth checking that out at the same time I guess.

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
For more information about reporting issues, see CONTRIBUTING.md.
You don't have to include this information if this is a feature request
(This is an automated, informational response)

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

Description of problem:
See original post.
docker version:
Client:
Version:      1.9.1
API version:  1.21
Go version:   go1.4.3
Git commit:   a34a1d5
Built:        Fri Nov 20 17:56:04 UTC 2015
OS/Arch:      darwin/amd64
Server:
Version:      1.9.1
API version:  1.21
Go version:   go1.4.2
Git commit:   a34a1d5
Built:        Fri Nov 20 13:12:04 UTC 2015
OS/Arch:      linux/amd64
docker info:
Containers: 46
Images: 1746
Server Version: 1.9.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 1844
Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: fluentd
Kernel Version: 3.13.0-65-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 6
Total Memory: 7.798 GiB
Name: derp
ID: derp?
Username: derp
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
uname -a:
Darwin derp 15.2.0 Darwin Kernel Version 15.2.0: Fri Nov 13 19:56:56 PST 2015; root:xnu-3248.20.55~2/RELEASE_X86_64 x86_64
Environment details (AWS, VirtualBox, physical, etc.):
docker and swarm running on Ubuntu VMs. Using TCP+TLS docker remote API.
How reproducible:
100%
Steps to Reproduce:

See
original
post.

Actual Results:
Building failed due to log driver failing to initialise and there is no way to override it with the docker build command.
Expected Results:
Building is always possible.
Additional info:
Please don't close my issue.

ping @cpuguy83 @duglin @tiborvass any thoughts on this?

I'd be fine with specifying the log driver here.
It's a weird situation, especially when probably most people don't need to log builds.

True, perhaps we can add a default setting in future, that allows setting a different driver for "build"

I was also going to suggest a specific default for run during build as JSON-file but it felt inconsistent. Then again, it might be more unexpected that building breaks.

Well, I was wondering if we should just default to json-file logs, not sure if there's people expecting other logs during build? Haven't given it a lot of thought yet though.

It's probably still indeed a problem. We need to default to json-file or none.
@thaJeztah @cpuguy83 thoughts?

@LK4D4 I agree, I think defaulting to none, or JSON-file is the best option

We see this as a bug, and it should ignore the daemon logging driver options

A pull request is opened in https://github.com/docker/docker/issues/29552 for those interested


Improper Command Name

Problem encountered on https://dotnet.microsoft.com/en-us/learn/aspnet/blazor-tutorial/run
Operating System: windows
The first command displayed to build and run the program, 'dotnet watch' is incorrecnt and should be corrected to 'dotnet watch run'

Did the dotnet new command run correctly as shown in the tutorial?
dotnet new blazorserver -o BlazorApp --no-https -f net6.0
The tutorial as written requires .NET 6 SDK. If you're getting an error message that the command should be dotnet watch run, then you're on a lower version of the SDK.


Errors in our evaluations

Currently, we have 6 ERROR_REPOS: PaddleClas', 'pyrallel', 'rltk', 't2wml-annotation', 'Calliar', 'Morph-OME. However, I think three of them are due because the annotation (perhaps) is not correct or complete (so I dont think that they are really errors of the code_inspector). And the last one, is something regarding with the evaluate_directory_type.py (again, so I dont think is an error of the code_inspector).

See bellow my comments :

ARBML/Calliar --- commented in  #issue194 It is annotated as a library. But in my opinion it should be just a script.

PaddlePaddle/PaddleClas is annotated in our csv as library, but it has an entry point. So, our code inspector "classifies" it as package. Furthermore, it also has several (independent) scripts with mains, which are detected by our classifier.

t2wml-annotation is annotated in our csv as package, but it does not have an entry point. So in my opinion we should classify it as a library.  Code_inspector classifies it as a library.

Morph-OME is classified as service and script, and our classifier classifies as service too. However, the evaluate_directory_type.py script count this as ERROR_REPOS.

I fixed the problem with pyrallel - so now code_inspector classifies as a library!.
However, rltk has an issue (error) with the "setup.py" - so our classifier can not inspect it (the mock does not work). And it guesses the wrong type.

Fixed problems with "pyrallel', 'rltk' (these were regarding with the code_inspector code)! so we have two errors less.

Ok! Found the error with Morph-OME -- it was missing and "and" in the CSV.
And I've found another (potential) error in the CSV- dig-sandpaper. Annotated as a package, but it doesnt have an entry point. So, it should be a library. I will change that later in the CSV.
I have added a note in all my changes of the CSV.

After reviewing annotations, now we have 8 main errors. It looks like before there were cases where "package, library" was returned, but as discussed in #201 now we return them just as "library". Hence the new errors. Some of these may be related to using other versions of python

I am going to close this issue, because it seems to have been addressed. Other issues have been uncovered with the newer evaluation :)


Add build for mariadb 1.0.7

I've only built it for GNU/Linux, some work is still required for macOS.


TOSS-common

in progress, for interface / code structure review only.
Add 2 new interface in thrift
ExecResponse addEdgesAtomic(1: AddEdgesRequest req);
ExecResponse processTransaction(1: TransactionReq txnCtx);
addEdgesAtomic is for client outside storage cluster.
processTransaction is used internal in storage cluster.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.liuyu seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Fix bug with encoding unpaired surrogates

Fix failing WPT cases for text encoding containing unpaired surrogates. Benchmarks show no change in performance. Uses similar logic to ::from_utf8_lossy but treats lone surrogates as having an error length of 3 instead of a separate error per byte so the correct number of replacement chars are inserted.

Looked at the CI failure. make tests passes for me locally — so I wonder if something is going wrong with cloning the submodule?


weather-mv will ingest data into BQ from Zarr much faster.

weather-mv bq's previous Zarr ingestion system only used one worker. This PR uses Xarray-Beam for Zarr ingestion, in order to distributed xr.Dataset chunks across beam workers. This improves ingestion into BQ.
Outstanding issues: I can't find a way to incrementally load rows into BQ from Zarr. While I've used windowing on fixed intervals to break up a large ingestion job into smaller parts, it seems like the actual writing to BQ gets stuck in a reshuffle step within the WriteToBigQuery transform. In this PR or a future PR, let's try to find a way to incrementally write rows to BQ once they've been processed, instead of having to wait for the entire dataset to be processed. CC: @dabhicusp.

Data after running pipelines for mentioned cases.

File Size
Before
After

~100Mb
58 min.
41min.

~1Gb
16h 30min.
1h 12min.

This changes are relatively fast than before for zarr batch ingestion.


Add github action for llvm-cov coverage

There has been discussion around generating coverage reports for tests. See #1853  and #2353.
This PR adds an action using llvm-cov and coveralls.
The action generates lcov coverage report using llvm-cov and uploads the generated report to coveralls. You can see sample reports for my fork here: https://coveralls.io/github/pool2win/rust-bitcoin
I am using the cargo-llvm-cov wrapper around llvm-cov.
I also use the coveralls official github action to push the coverage report to coveralls. It removes the need to deal with repository secrets etc, making the action easy to run for all contributors on their forks.
We can move this action later to rust.yml if we need to.

Neat! Looks like it's working. This PR's report appears here https://coveralls.io/github/rust-bitcoin/rust-bitcoin

Do you know if there's a way we can signal to coveralls to ignore the fuzz/ directory?

Do you know if there's a way we can signal to coveralls to ignore the fuzz/ directory?

Seems like we can: https://github.com/taiki-e/cargo-llvm-cov#exclude-file-from-coverage
Here is the change that makes it work: https://github.com/pool2win/rust-bitcoin/commit/47b476ec5bdc2f4d9a9fd28f30a7152e1bf45277
Here is a coverage report without the fuzz directory: https://coveralls.io/github/pool2win/rust-bitcoin?branch=ignore-fuzz-dir-for-coveralls
I can do a separate PR for the patch or push the commit to this PR - whatever you prefer.

Do you want to update this PR to have fuzz excluded?

~Do you want to update this PR to have fuzz excluded?~ (Sorry, my brain is missing some gears today.)
I think just updating the PR is simplest. ACK once the change is in.

Updated the PR to ignore the fuzz directory.


fix: hydrate in twind-v1-plugin

This PR is a slightly modified version of hydrate from twind-v1-plugin (#946).
In this plugin, Cssrules of SSR are resumed by cssom() of twind.
There are demos for each of #946 and this PR. (I added a lot of classes to compare the performance.)

#946 plugin's Demo (source)

This PR plugin's Demo (source)

What is the issue with #946 plugin?
Twind's getSheet returns the sheet created from cssom to which resume is assigned, but since SSR's HtmlStyleSheet cannot be targeted to this sheet, resume at setup doesn't make sense.
As a result, CSSRules of ClientSide that partially overlaps with ServerSide is inserted.
Demo

I think this should be improved.
What was changed?
I made a sheet created by cssom with SSR's HtmlStyleSheet as the target, and bind the resume of the sheet created by getSheet.
export default function hydrate(options: TwindConfig) {
  const elem = document.getElementById(STYLE_ELEMENT_ID) as HTMLStyleElement;
  const sheet = cssom(elem);

  sheet.resume = getSheet().resume.bind(sheet);
  document.querySelector('[data-twind="claimed"]')?.remove();

  setup(options, sheet);
}

Demo

The resume of the sheet is working, so there is no duplication.
What about performance?
I compared the performance of each plugin on chrome.

#946 plugin's Demo (source)

This PR plugin's Demo (source)

The Scripting time for each is about the same.

Thank you very much for reading through to the end.
Please let me know if my understanding is wrong.🙏

Thank you!

Should I create a test like fresh/tests/fixture_twind-pr1050?

just fresh/tests/fixture_twind is also fine!

Should the content of the test be such that this PR passes and https://github.com/denoland/fresh/pull/946 fails?(For example, compare stylesheets in islands)

Yes, it would be helpful!

@hashrock Thanks!
I'll write a test.

I added test!
The test contents are as follows

Ensure that twind compiles class in static page.
Ensure that twind does not cause duplication of cssrules in islands.
Ensure that twind compiles dynamically inserted classes in islands.

deno task test results would look like this

Test No.
#1050
#946
twind v0.16 plugin

1
:white_check_mark:
:white_check_mark:
:white_check_mark:

2
:white_check_mark:
:x:
:white_check_mark:

3
:white_check_mark:
:x:
:white_check_mark:

Please let me know if I am missing or wrong!

@ry Thanks for the review!
I have corrected the areas you reviewed.
But there is one thing I couldn't solve.
The textDecoderStream called here leaks.
https://github.com/denoland/fresh/blob/c4cfb201e72908ae6d0cc9993310a9baa78891dd/tests/twind_test.ts#L23-L25
I have tried calling cancel() of lines and  externally TextDecoderStream (ex. const decoder = TextDecoderStream();), but could't solve it.
For now, I got the textDecoder's rid from the resourceMap and manually close the resource.
https://github.com/denoland/fresh/blob/c4cfb201e72908ae6d0cc9993310a9baa78891dd/tests/twind_test.ts#L47-L63
Is there a correct cancel() call?
Or should I add sanitizeResources: false?

Without --trace-ops, a leak occurred, so I added delay(10) before t.step.

@hashrock Thanks for merging!


Bring back the baseball illustration

I'm a musician, and I even compose music, and I absolutely ABHOR the music illustration for consistency levels. With music, it is either strong consistency level, or it is noise - there is no other option. This is why this is such a bad illustration.
I understand that not everybody understands baseball, but it is a simple game. I use that illustration in class (I'm an MCT) all the time and have no problem explaining it. In fact, the baseball illustration is what made it originally make sense to me. It just makes more sense.
Please bring back the baseball illustration. If you don't want to get rid of the music illustration, then at least give us an alternative by adding the additional illustration.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2b3fb7da-5483-160d-1781-d4abff9d798d
Version Independent ID: d5e51b7e-7f34-c7df-ce4f-ee4bf5c8059a
Content: Consistency levels in Azure Cosmos DB
Content Source: articles/cosmos-db/consistency-levels.md
Service: cosmos-db
GitHub Login: @markjbrown
Microsoft Alias: mjbrown

@sthoward0914 We are assigning this to our author to evaluate and comment on.
@markjbrown Can you please check this feedback and help here.

Thank you for the reply @markjbrown . @sthoward0914 - Please let us know if you need any further info or we will go ahead and close this request.

Okay. Understood. Can we at least have something that will make sense in the context? As I said, with music, it is either strong consistency, or it is chaos and noise. What would provide a logical explanation to a diverse audience. I don't think music is it.
I'm open, but give me something shareable with a class.

I'm not sure why you think the others beside strong do not reflect the behavior of the consistency model. The sequence of the notes is accurate to the consistency model's behavior. Additionally, feedback on using the musical notes has been very positive and is in fact, what we show in the Azure portal.
This is the first instance of feedback asking it be changed. I'm open to changing it to use another metaphor but I will need considerably more customer feedback before I do so.
For now I'm fine noting this feedback but will leave as is for now.
Thanks again.

@markjbrown
In answer to your question, because musical notes are more than just an order. The notes indicate the pitch and the timing. When you change the order that the notes are played (arrive), you change the musical piece you are playing. It becomes even more chaotic as that can change harmonies/chords, and at the end, if you have anything other than strong consistency that can enforce not only the order, but the timing, then you're not playing the same piece of music. It does not maintain musical integrity.
I think it might be a good illustration of a case when strong consistency is required (or possibly bounded staleness since you would hear the same music, but just with latency), but logically, I don't think it makes real sense as an illustration of any other consistency model other than to say "if that happens, you have something entirely different." That's also why I don't use it as an illustration in class. It's why I always read it and say "Surely there is a better illustration than that which could be used." With the baseball metaphor, I could focus only on order and whether it reflected a state that actually existed in the ball game (eventually consistent may not, for example).

Thanks again for your feedback and thoughts on this.
As I stated previously, I'm always open to finding a metaphor that helps convey this concept as it can be challenging at first to grasp. That said, I have not received really any feedback beyond this that the musical notes are not a useful way to relate to this.
I will make note of your feedback and comments here and track for any future feedback. If I receive sufficient input that another metaphor is better to use, I will make that adjustment to our examples here.
Thanks again!

@sthoward0914 At this time we will go ahead and close this feedback. Thank you.


Workaround for displaying responses in local result browser

--> see load-testing/manual/440-result-browser/#response-content-tools --> TODO in markdown
Should we document a workaround for the problem that browsers say "Failed to load '[filename]'" when you try to open response in result browser? If yes, what is the 'official' workaround? (Suggestion: starting a local python server in results directory works.)

Already done, see /load-testing/manual/440-result-browser/#response-content-tools


Large Object Support?

Curious if there are any plans for large object support? Normally I'd just store in the filesystem with a checksum in the DB but I have a need for the streaming/seeking abilities that come with the large object api. I don't even need all that fancy support here only basic crud and from my other tools I can do the rest.
I see TEXT support but that's 1GB limited and I'm doing neural network stuff that easily eclipses that with larger models so I truly need large object support or perhaps another document db to go alongside.
Wanted to ask in case the docs were out of date. I also lurk in the Gitter chat and can ask there if collaboration is needed. Thanks!

Hello @acidtonic ,
I'm currently working on finishing arrays functionality.
Large objects can be added afterward. But most probably it'll happen (large objs) in the next release 1.3.0.
Regards,
Leonid

I seen arrays and 1.2.5 get released, just wanted to see how large object support is going.
Also if I use TEXT do I need to do my own base64 encoding?

@acidtonic ,
Yes, arrays were added in 1.2.5.

Also if I use TEXT do I need to do my own base64 encoding?

Yes, if your binary data can contain a '\0' char - then base64 encoding may be a good solution for you.

just wanted to see how large object support is going.

Yes, I plan to add it in 1.3.0

Hi @lganzzzo
Large Object Support is integrated in 1.3.0 or it is work still in progress ?
I couldn't find any example on how to use this in oatpp framework.


Textual Inversion training error

After update I kept getting this error using the usual setup for my training.
Loading weights [None] from /content/gdrive/MyDrive/sd/stable-diffusion-webui/models/Stable-diffusion/model.ckpt
Applying attention optimization: xformers... done.
Weights loaded in 7.5s (load weights from disk: 5.1s, apply weights to model: 0.8s, apply half(): 0.1s, move model to device: 1.5s).
Training at rate of 0.01 until step 100
Preparing dataset...
100% 200/200 [00:05<00:00, 34.42it/s]
0% 0/250 [00:00<?, ?it/s]*** Error training embedding
Traceback (most recent call last):
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 529, in train_embedding
scaler.scale(loss).backward()
File "/usr/local/lib/python3.10/dist-packages/torch/_tensor.py", line 487, in backward
torch.autograd.backward(
File "/usr/local/lib/python3.10/dist-packages/torch/autograd/init.py", line 200, in backward
Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1, 77, 768]], which is output 0 of MulBackward0, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

im facing the same issue. i was able to train embeddings yesterday, don't know what happened today that made it break

im facing the same issue. i was able to train embeddings yesterday, don't know what happened today that made it break. were you able to get it working?

No luck yet

I am also having this issue

Looks like the issue is with something in automatic1111 that hasn't been updated in the colab yet. If you want to fix it manually, find this file in your drive and make the change described here.

Looks like the issue is with something in automatic1111 that hasn't been updated in the colab yet. If you want to fix it manually, find this file in your drive and make the change described here.

I made the said changes and still didn't work for me unfortunately. Were you able to get it running?

yes, it's running for me. you need to restart the Start Stable-Diffusion step in the colab for it to work, but otherwise idk what it could be

I made a silly mistake a didn't delete the 2 lines as instructed. After deletion and addition it works fine now.
I do want to point out that even with the latest build, without making any changes, the problem persists. So if you want to run TI training you still have to make this manual change every time.
Thanks for the recommendation!

yes, it's running for me. you need to restart the Start Stable-Diffusion step in the colab for it to work, but otherwise idk what it could be


display for current viewed ghost

It's annoying to have hide all but the current ghost to see Dashboard while viewing records. This also applies for "hide PB ghost"

sorry not fully tested, pb ghost inputs are still shown when viewing record

This needs to be implemented in the VehicleState dependency plugin. Take a look at the discussion in this closed PR: https://github.com/openplanet-nl/vehiclestate/pull/3


starting server: creating filecoin host: couldn't connect to any of bootstrap peers

Trying to connect to the new calibrationnet

Hey @nathansenn . jsign/calib.8.13.0 is a WIP branch to add support to the new version, so isn't yet ready.
In the process of adding support in Powergate, I discovered a retrieval bug on some file sizes. They were working in a fix, so most probably using 8.13.0 won't be a usable version anyway.
Stay tuned since most prob today we'll have a PR ready for the new calibration.

#569 This would solve the bootstrap issue for the new calibrationnet

Replied on the PR!


Login error (1106): permission deny

So I created new home assistant installation and I can't login, i get error Login error (1106): permission deny, no more details.
I used same information for login as in my previous installation where it worked, is there some log where I can get more information for the error?

sadly changing password in the mobile app fixed this :(


Support for Speed entities/services

Curious why Speed controls were removed in a previous commit?
Would it be possible to add back in support for displaying the current speed as an entity, and exposing a service to set the speed?
(TP04)

Curious why Speed controls were removed in a previous commit?

That's a really great question. I wish I knew the answer, but that was done by Shen way before I adopted the project. It sure seems weird though.

Would it be possible to add back in support for displaying the current speed as an entity, and exposing a service to set the speed?

Fret not, because it is currently supported! It was added back, perhaps somewhere around 49a7aa4240b49747b09a207cdc260cce32a151b8? Or maybe Shen's commit was just part of a refactor? I'm not entirely sure.
Anyway, a fan entity is exposed 's speed can be displayed and controlled.

Let me know if you have any issues with it!

Home Assistant switched from named speeds to percentages for Fan entities. Auto mode was changed to a preset. These changes are not specific to the Dyson integration.

Wow! Total oversight on my part. I was searching through the entities list and debugging services and didn't see it. Now that I clicked on the device, I do see the same as your screenshot!
I am able to use the "set speed percentage" service just fine!
Thanks for the info!

Glad you got it set up!
In Lovelace, it takes like 2 or 3 cards to cover all of the features of entities Some day I kinda want to make a custom card that fits the Dyson fans.


nothing is seen in codepen/jsfiddle

when I open codepen/jsfiddle examples - nothing is seen, only empty space.

@diomed  thanks for your report, I had reproduced the error, that sees some css file loaded failed
I will fix it later, you can clone the project and run npm run dev to see the demos after installing dependencies at this time.

I got a question: what the IM tool you are using ? Slack or Wechat or Discord ?

Discord

got it

⁉

Just curious, because I am in China, use WeChat more.
If you have a more urgent need, you can also find me at discord.
Of course, the issue is actually a better way, leaving me some time to think.


Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots or Links
If applicable, add screenshots or links to help explain your problem. DO NOT include links to NSFW webpages. Preferably create code to reproduce using https://jsfiddle.net
Desktop (please complete the following information if relevant):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information if relevant):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Affected version
For example, v3.0.0
Additional context
Add any other context about the problem here.

Hi @Phamkhang135 - can you tell us more about this?
What is the bug you're facing? If you could provide us with a description of the issue, that would be great.
Any additional detail you can provide us with - such as the device you're using and the browser would also be really helpful!

Hi @Phamkhang135 - seeing as no additional details were provided we'll close this issue. If you're still having an issue, please feel free to re-raise it and provide further details so that we can look into it for you.
Thanks!

Hola, encontre un error en la V3, algunas m3u8 no se pueden reproducir con la V3, sin con la V2, concretamente el reproductor queda cargando y nunca termina de cargar. Note este error con una url en concreto, desconozco si es por compativilidad del explorador o que la V3 no proporciona alguna decodificacion que la V2 si.


Golangci-lint cache input key is wrong on re-runs

When I reran the golangci-lint job, there was a problem loading the cache
  /usr/bin/tar<EMAIL_ADDRESS>Cannot open: File exists
  Error: /usr/bin/tar<EMAIL_ADDRESS>Cannot open: File exists
  /usr/bin/tar: Exiting with failure status due to previous errors
  Warning: Failed to restore: Tar failed with error: The process '/usr/bin/tar' failed with exit code 2
  Cache not found for input keys: golangci-lint.cache-2766-764367f7551edc6ee1cd40c0c794ec829fc93f91, golangci-lint.cache-2766-
  Prepared env in 6800ms

job - https://github.com/statnett/image-scanner-operator/actions/runs/3845788279/jobs/6550500816

@padlar , is this still a problem? I tried to rerun a golangci-lint job now, and it worked. Is there any other way than rerunning a lint job that can reproduce this?
I notice that in the issue you created upstream got the label question, meaning Further information requested, but I can't see what kind of information is requested, do you know?

Looks like this has been fixed in the underlying github actions pkg https://github.com/actions/toolkit/pull/1353. We can close this issue and probably update https://github.com/golangci/golangci-lint-action/issues/640 with our observation?

Sounds like a plan to me :-)

This issue seems to have been fixed now.


Pages without a module cannot be pushed with the Directory or Module flags

Pages outside of a module should have been written to disk with an _ before the name (ex. _NonModularizedPage instead of NonModularizedPage. Since the original implementation does not do this, we should fix the push command to take this into account.

Closed by #5


Switch New York Times case study to use youtube shortcode

Closes #25124
xref #23660

/kind cleanup
/cc @sftim

Preview: https://deploy-preview-25138--kubernetes-io-master-staging.netlify.app/case-studies/newyorktimes/

/assign @kbhawkey

/approve


Make unexported fields panic (informatively) instead of silently passing

Closes #3928.

LGTM

Looks like this fails in places other than the serialization test. Will fix.

OK well that was a sequence of disturbing findings. Fixed now, I think.

LGTM


refactored canvas manipulation from engine into seperate file.

Cool, I don't think I moved any of this out of engine.js into my dom-manipulator file so hopefully there will be no conflicts! This is really helping out engine.js. :100:


ARROW-12814: [C++][Gandiva] Implements ABS, FLOOR, PI, SQRT, SIGN, LSHIFT, RSHIFT, CEIL, TRUNC, LN and LOG2 functions

Implements the ABS, FLOOR, PI, SQRT, SIGN, LSHIFT, RSHIFT, CEIL and TRUNC.
The RANDOM functions are already implemented in Gandiva.

#10274 handles abs operation well.

I included the CEIL function in the title because it was implemented too.

@kiszk I applied all suggestions you made, could you check if the corrections are fine? Thanks in advance for the review!

The LN function was added in this PR too.

The LOG2 function was added to the Pull Request

@augustoasilva please, could you review this Pull Request?

Regarding rshift and lshift, is it better to keep the same semantics as this? cc @pitrou

It would certainly be desirable to have the same semantics in Gandiva and in Arrow compute kernels.

@pitrou @kiszk I will check the reference that you sent in Arrow Compute and I will try to apply the same semantic in this Pull Request

It looks like the difference is that here the shift amount is reduced modulo the type precision (what x86 generally does but not what ARM does, e.g. see this post about MSVC on ARM; allows negative shifts) and in the other PR, such shifts (including negative shifts) are disallowed (following the C++ specification).
The Arrow Compute behavior reflects the presence of having checked/unchecked variants; it seems the Gandiva variants must always produce a result? But it would be good to have them agree.
Looking at some other systems:

T-SQL (MS SQL Server) has no bit-shift operators,
Postgres doesn't specify a behavior.
Python disallows negative shift values (overflow doesn't apply due to infinite-precision integers)

@lidavidm thanks for the reference about the shift operators in ARM architecture.
I used the Java implementation as a base for the SHIFT behavior.
I choose it just to keep the compatibility between the languages, but I can change to match the same specifications that the Compute module.

Well it's perhaps worth discussing - which of the various languages' behaviors we want to mimic. Though as Antoine noted in the other PR, since there's not really a consensus, it's safer to just error on out-of-range values.

@kiszk @pitrou and @lidavidm I follow your suggestions and I changed the shift functions to have the same behavior as the Compute module. Thanks for your help!
@kiszk I also applied your suggestions to add more cases in the tests, when you be available, could you take another look in the changes?

@kiszk I applied all suggestions you made in the PR. Could you take another look at the PR and see if it is ready to be merged?

Sorry for late review. It looks good to me. I would like to ask another review.
@pitrou Could you please review this?

@pitrou When you be free, could you take a look at this PR? I applied all changes that you and @kiszk suggested.

ping @pitrou

@pitrou can you take a look at this PR? Because I want to close it as soon as possible

@pitrou can you take a look at it?

I would rather if a Gandiva developer vetted this PR. @pravindra perhaps?

Closing because it has been untouched for a while, in case it's still relevant feel free to reopen and move it forward 👍


Configs CI Build Checker

Configs CI Build Checker to Project
Enable github actions to build checker.

It's OK ⭐


err: add PyErr::fetch_last_error

An alternative to #1957.

We decided not to proceed with this alternative.


bottle: check actual bottle block contents when --keep-old

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[x] Have you successfully run brew tests with your changes locally?
[x] Have you successfully run brew man locally and committed any changes?

This PR restores some of the brew bottle --merge --write --keep-old logic that was removed in https://github.com/Homebrew/brew/pull/10102
Instead of checking for default spec values, we can check which methods were being called in the old bottle block
Supersedes https://github.com/Homebrew/brew/pull/10135

Review period will end on 2021-01-01 at 17:58:13 UTC.

Review period ended.

Review period ended.

Merging now so @iMichka can continue work on the new cellar syntax—I can open a follow-up PR if any changes are requested after merge

Merging now so @iMichka can continue work on the new cellar syntax—I can open a follow-up PR if any changes are requested after merge


CNCF Projects : Docs Added

CNCF Projects : Docs Added are Open Policy Agent , TiKV , Linkerd,Argo, Buildpacks,CloudEvents,CNI,CRI-O,
FALCO, Flux, KubeEdge , Open Telemetry , SPIFFE , SPIRE , Thanos, Contour,Cortex, CrossPlane,Flagger,
gRPC, KEDA , Notary , Operator Framework , HELM , CoreDNS , CNCF

@Aryamanz29  i have added 26 files further in CNCF Projects


remove print from public API

motivation: reduce public API surface area given the plan is to abosrb this library in the language runtime eventually
changes:

make print function private
remove explicit swift-server file

as discussed on our last sswg call, removing print to reduce API surface area. will need a major version bump I assume

@tomerd yes, will need a major bump. There's one thing we could do instead: We could just deprecate it in a minor for now??

We could just deprecate it in a minor for now??

works for me
@ianpartridge @tanner0101 ?


chore: remove console.log from go pacmak gen

Removes a console.log statement from the go Documentation class that
was logging to Rosetta instance fo each emit invocation. This was
added to correct a lint error that the instances was not used that
wasn't easily surpressed.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: d9bef3647602cf4b69f9c0a99e20b2b1b34c09d2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-Blkkw9bQFn8A
Commit ID: 6747dc466e790267f3bd985cdd2726f2bdeecf31
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Update zuul role

Update containers to latest upstream versions
Add auth configuration
Fix variable names for nodepool images
Add zuul_user definition to role defaults

@osfrickler Is this already ready for a review?

@osfrickler Is this already ready for a review?

Yes, this is what we run in production. I'd made it a draft if it weren't ready.


Ability to use Kafka filtering and recordFilterStrategy

Hello,
I am building an application which produces and consumes messages to/from the same kafka topic (shared with other applications).
Therefore, I want to be able to filter out messages produced by the application itself.
To do that with efficiency, I want to filter messages as soon as possible.
I think that FilteringAcknowledgingMessageListenerAdapter is the solution.
But to use it, I need to be able to pass a RecordFilterStrategy to KafkaMessageDrivenChannelAdapter.
Would it be ok to add a specific kafka property describing a RecordFilterStrategy class name?
If not, how can I do that?

Spring Integration does not currently support wrapping its internal listener in another adapter such as the filtering adapter.
You would have to use spring-kafka's @KafkaListener instead of @StreamListener. There, you can inject a RecordFilterStrategy into the ConcurrentKafkaListenerContainerFactory.
Feel free to open an issue at spring-integration-kafka and we can look at adding the capability.

The `` wraps into FilteringAcknowledgingMessageListenerAdapter  its listener since 2.0.1:
https://github.com/spring-projects/spring-integration-kafka/blob/master/src/main/java/org/springframework/integration/kafka/inbound/KafkaMessageDrivenChannelAdapter.java#L219
The question is how to get a gain of that RecordFilterStrategy in the Kafka Binder for the KafkaMessageDrivenChannelAdapter.
Am I right?

That configuration seems to be tied to the retry functionality of the KafkaMessageDrivenChannelAdapter . Maybe it shouldn't be.
In any case, while we can allow the infrastructure of the Kafka binder to be more flexible, I wonder what about the benefits.  This seems to be a scenario where a Spring Integration Filter would be equally appropriate (and for that matter portable as well).
cc/ @artembilan @garyrussell

That configuration seems to be tied to the retry functionality of the KafkaMessageDrivenChannelAdapter.

I don't think so. Look, the logic there is based on the:
boolean filterInRetry = this.filterInRetry && this.retryTemplate != null
					&& this.recordFilterStrategy != null;

But if you still have some doubts, please, open an appropriate issue in SI Kafka.
Yes, SI filter in the target SCSt application for the INPUT channel is good and really portable compromise.

@artembilan What I meant was that in the KafkaMessageDrivenChannelAdapter, if this.retryTemplate is null (no retry configured) then filterInRetry will be false and  FilteringAcknowledgingMessageListenerAdapter will not be set. In any case, this is beyond the scope of the original issue.

@artembilan My mistake - there's an else branch that I missed, so you're right.
In any case, my original observation remains: I think we should rather use a Filter in a SCSt solution.

Hello,
Thank you for your answers.
I want to continue to use Spring Cloud Stream and be able to filter before payload deserialization.
I do not know the other solutions (SCSt, S-I-Kafka, Spring Kafka).
1- Does SCSt provide filtering before payload deserialization?
2- Is there a way to tweak KafkaBinderConfiguration.kafkaMessageChannelBinder() from spring-cloud-stream-binder-kafka to add a RecordFilterStrategy to kafkaMessageDrivenChannelAdapter? If not how should I do it? Could we add a way to tweak it?
3- Finally can you give me more details about the simplest way to reach my goal?
I am prone to contribute with PRs if needed.

Spring Cloud Stream is fully based on Spring Integration. The pring-cloud-stream-binder-kafka is fully based on the Spring Kafka and Spring Integration for Kafka. So, you still continue to use Spring Cloud Stream but pull a bit Spring Integration API, principles and design techniques to your application. Finally the application would be what you and customer expect.

No, SCSt doesn't provide filtering. More over  Spring Kafka doesn't do that. Even if with RecordFilterStrategy, we already have deserialization from Kafka before processing records.

No, there is no now way to tweak kafkaMessageChannelBinder for the RecordFilterStrategy. What we suggest is like filter EI pattern implementation from Spring Integration: http://docs.spring.io/spring-integration/reference/html/messaging-routing-chapter.html#filter.
And no, I don't think that we will add that option, just because it must be a bean, no just simple property.

I think something like simplest solution can be like:

@Filter(inputChannel = Processor.INPUT, outputChannel = "processRecord")
public boolean filterRecord(ConsumerRecord<?, ?> consumerRecord) {
    // filtering logic
}

@StreamListener("processRecord")
@SendTo(Processor.OUTPUT)
public Object processRecord(MyPojo data) {
   ....
}

We always welcome contribution, but you should be sure that you are familiar with all the API used by Spring Cloud Stream in particular Spring Integration.
That reminds me a story like: how can you use JPA not knowing JDBC?..

Ok thank you.
If there is no way to filter before payload deserialization and after headers deserialization, then I will use the @Filter solution.
I was proposing to contribute following your eventual guidance ;)

headers ???
Kafka doesn't have notion of message header.
What you see in SCSt is artificial solution based on convention.  Therefore we can't have headers before payload anyway. Because we have to deserialize Kafka message fully and only after that we apply our algorithm to extract headers and payload separately.

I see.
My second idea was to be able to set kafka record id myself from the producer and filter on it. Maybe it is also a chimera ...

That's fine. Spring Kafka populates the key to the headers like:
Map<String, Object> rawHeaders = kafkaMessageHeaders.getRawHeaders();
rawHeaders.put(KafkaHeaders.RECEIVED_MESSAGE_KEY, record.key());
rawHeaders.put(KafkaHeaders.RECEIVED_TOPIC, record.topic());
rawHeaders.put(KafkaHeaders.RECEIVED_PARTITION_ID, record.partition());
rawHeaders.put(KafkaHeaders.OFFSET, record.offset());

if (acknowledgment != null) {
	rawHeaders.put(KafkaHeaders.ACKNOWLEDGMENT, acknowledgment);
}

With many others as you see.

Ok thanks for the tips !
Closing this issue.

headers ???
Kafka doesn't have notion of message header.
What you see in SCSt is artificial solution based on convention.  Therefore we can't have headers before payload anyway. Because we have to deserialize Kafka message fully and only after that we apply our algorithm to extract headers and payload separately.

@artembilan In that case, does it really make a difference if we filter the message in KafkaConfiguration class (@filter) or we consume the message and do Programmatic filtration in Listener class

@ashankabiswas  Please do not comment on closed issues especially with questions.. Please us Stack Overflow channel.


Change SECRET_KEY  and get error ValueError: Invalid decryption key on K8s

When I try change SECRET_KEY = '\2\1thisismyscretkey\1\2\e\y\y\h'  (default), to
(for example):
SECRET_KEY = '\2\1888D237694D70843CDBC38195215E5623EDAA74C3D47E8C4F1156\1\2\e\y\y\h'

I getting the error ValueError: Invalid decryption key in sqlalchemy_utils when on execute superset-init.
To reproduce:
$ cd ./install/helm/superset
$ mkdir config/ && touch config/superset_config.py
$ nano config/superset_config.py
---
import os

# Your App secret key
SECRET_KEY = '\2\1<new long random string>\1\2\e\y\y\h'

# The SQLAlchemy connection string to your database backend
# This connection defines the path to the database that stores your
# superset metadata (slices, connections, tables, dashboards, ...).
# Note that the connection information to connect to the datasources
# you want to explore are managed directly in the web UI
# SQLALCHEMY_DATABASE_URI = 'sqlite:////var/lib/superset/superset.db'
SQLALCHEMY_DATABASE_URI = 'postgresql+psycopg2://XXXXX:XXXX@host/db'

# Flask-WTF flag for CSRF
WTF_CSRF_ENABLED = True
# Add endpoints that need to be exempt from CSRF protection
WTF_CSRF_EXEMPT_LIST = []

# Set this API key to enable Mapbox visualizations
MAPBOX_API_KEY = os.environ.get("MAPBOX_API_KEY", "")
---
$ helm upgrade --install superset .
$ kubectl exec -it superset-<pod_id> superset-init

Expected results
Actual results
Loaded your LOCAL configuration at [/etc/superset/superset_config.py]
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
2019-11-10 14:44:31,592:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
Loaded your LOCAL configuration at [/etc/superset/superset_config.py]
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
2019-11-10 14:44:38,625:INFO:root:Configured event logger of type <class 'superset.utils.log.DBEventLogger'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 124, in decrypt
    decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xca in position 1: invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/superset", line 31, in <module>
    cli()
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 586, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 426, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/superset/cli.py", line 51, in init
    utils.get_or_create_main_db()
  File "/usr/local/lib/python3.6/site-packages/superset/utils/core.py", line 948, in get_or_create_main_db
    get_main_database()
  File "/usr/local/lib/python3.6/site-packages/superset/utils/core.py", line 971, in get_main_database
    return get_or_create_db("main", conf.get("SQLALCHEMY_DATABASE_URI"))
  File "/usr/local/lib/python3.6/site-packages/superset/utils/core.py", line 956, in get_or_create_db
    db.session.query(models.Database).filter_by(database_name=database_name).first()
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3228, in first
    ret = list(self[0:1])
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3018, in __getitem__
    return list(res)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 105, in instances
    util.raise_from_cause(err)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 85, in instances
    rows = [proc(row) for row in fetch]
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 85, in <listcomp>
    rows = [proc(row) for row in fetch]
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 572, in _instance
    populators,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 693, in _populate_full
    dict_[key] = getter(row)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/type_api.py", line 1247, in process
    return process_value(impl_processor(value), dialect)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 409, in process_result_value
    decrypted_value = self.engine.decrypt(value)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 126, in decrypt
    raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key

Screenshots
If applicable, add screenshots to help explain your problem.
How to reproduce the bug

Go to '...'
Click on '....'
Scroll down to '....'
See error

Environment
(please complete the following information):

superset version: superset version
0.34.1
python version: python --version
3.6.1
node.js version: node -v
npm version: npm -v

Checklist
Make sure these boxes are checked before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[x] I have reproduced the issue with at least the latest released version of superset.
[ ] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context
Add any other context about the problem here.

Hi @newtonjose,
This is probably because you already have database connections that were encrypted with the old key. You have to recreate them or update

This is probably because you already have database connections that were encrypted with the old key. You have to recreate them or update

how to recreate or update ,this page get error

This is probably because you already have database connections that were encrypted with the old key. You have to recreate them or update

how to recreate or update ,this page get error

i got an solution,  add a config DB_SECRET_KEY in config.py it's value is origin SECRET_KEY,  then use config["DB_SECRET_KEY"] replace config["SECRET_KEY"] in superset/models/core.py  Database model

I used a different approach:
I could only find 1 place that the DB_SECRET_KEY was being used - which was in the dbs table under password. After rotating the SECRET_KEY, i set that field to null in the database. After that the database screen will work again (it crashes because it can't decrypt the passwords) - and then reset the passwords.

The databases page would not load for me after changing the SECRET_KEY. Here is how I fixed it:
psql -d superset -U superset -W
# Password for user superset:
select database_name, password from dbs;
update dbs set password = null;

Now the databases page will load but clicking edit on certain databases still didn't work and caused this stack trace:
Unable to load dialect <class 'sqlalchemy.dialects.mssql.adodbapi.MSDialect_adodbapi'>: type object 'MSDialect_adodbapi' has no attribute 'dbapi'
2021-09-17 18:29:43,812:WARNING:superset.db_engine_specs:Unable to load dialect <class 'sqlalchemy.dialects.mssql.adodbapi.MSDialect_adodbapi'>: type object 'MSDialect_adodbapi' has no attribute 'dbapi'
2021-09-17 18:29:43,819:ERROR:root:Invalid decryption key
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 126, in decrypt
    decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 2: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 85, in wraps
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 155, in wraps
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 1448, in get
    return self.get_headless(pk, **kwargs)
  File "/app/superset/utils/log.py", line 242, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 390, in get_headless
    duration, response = time_function(super().get_headless, pk, **kwargs)
  File "/app/superset/utils/core.py", line 1461, in time_function
    response = func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 1344, in get_headless
    item = self.datamodel.get(pk, self._base_filters, self.show_select_columns)
  File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/models/sqla/interface.py", line 935, in get
    query, _filters, select_columns=select_columns
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
    ret = list(self)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
    cursor.close()
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
    with_traceback=exc_tb,
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
    rows = [proc(row) for row in fetch]
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
    rows = [proc(row) for row in fetch]
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 588, in _instance
    populators,
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/loading.py", line 725, in _populate_full
    dict_[key] = getter(row)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/type_api.py", line 1278, in process
    return process_value(impl_processor(value), dialect)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 469, in process_result_value
    value = super().process_result_value(value=value, dialect=dialect)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 414, in process_result_value
    decrypted_value = self.engine.decrypt(value)
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt
    raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key

Create a new database (e.g. name = "New DB") from the UI. You can get the SQLAlchemy URI using this SQL:
select sqlalchemy_uri from dbs where database_name = 'Original DB';

Once the new database is created, back in psql:
select database_name, id from dbs where database_name = 'New DB' or database_name = 'Original DB';
# 5 = New DB
# 2 = Original DB
update tables set database_id = 5 where database_id = 2;
update tab_state set database_id = 5 where database_id = 2;
update query set database_id = 5 where database_id = 2;
update saved_query set db_id = 5 where db_id = 2;
update table_schema set database_id = 5 where database_id = 2;
delete from dbs where database_name = 'Original DB';
update dbs set database_name = 'Original DB' where database_name = 'New DB';

I've unfortunately encountered this and here's what I've done to fix this:

Change SECRET_KEY on superset_config.py
Access psql client and connect to the superset DB
update dbs set password = null; (thanks @jhult), exit out
superset db upgrade
superset init
Update password on Superset Data > Databases (I'm using 1.2 btw)

I've encountered the same issue when upgraded superset to 1.4.1 from 1.4.0.
I resolved the issue @andtlopez comment with one more step.

Change SECRET_KEY on superset_config.py
Access psql client and connect to the superset DB (\connect db_name;)
3 update dbs set password = null, encrypted_extra where id=?;  <-  encrypted_extra should be set to null
superset db upgrade
superset init
superset run
Update password on Superset: Data > Databases (I'm using 1.2.0 btw)

just sharing my observation that having same secret_key and connecting the from the different host fails with invalid decryption key. would like to know views from community.( i think this by design)

I had the same issue when I use docker-compose to install superset on MacOS.
Due to it is my local test, it is easy for me just to remove old containers and old data and start fresh.

docker-compose down -v (or docker-compose down and docker volume prune), this will remove the old containers and old data
docker-compose up, to start fresh

The key point is you need to remove old volumes.

just sharing my observation that having same secret_key and connecting the from the different host fails with invalid decryption key. would like to know views from community.( i think this by design)

Same for me, upgrading (in my case from 1.3.2 to 1.4.2rc1) breaks the encryption, even if the SECRET_KEY stays exactly the same (Superset even warns about using the default SECRET_KEY).

I've encountered the same issue when upgraded superset to 1.4.1 from 1.4.0. I resolved the issue @andtlopez comment with one more step.

Change SECRET_KEY on superset_config.py
Access psql client and connect to the superset DB (\connect db_name;)
update dbs set password = null, encrypted_extra where id=?;  <-  encrypted_extra should be set to null
superset db upgrade
superset init
superset run
Update password on Superset: Data > Databases (I'm using 1.4.1 btw)

In step 3 I'm getting a syntax error.
Would update dbs set password = null; update dbs set encrypted_extra = null; work?

@andtlopez thanks for the hint.
Same issue. But what I also wanted was to change SECRET_KEY from default one to new one. I couldn't find anything about how to decrypt paswords with the SECRET_KEY. I found useful to decrypt passwords from working superset before upgrade. To do that:

Run a shell in the app context

superset shell

Paste a snippet

from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

from superset import config
from superset.models.core import Database

engine = create_engine(config.SQLALCHEMY_DATABASE_URI)
connection = engine.connect()
Session = sessionmaker(bind=connection)
session = Session()

db_ids = session.query(Database.id).all()  # -> [(1,), (2,), (3,), (6,), (4,), (5,)]
db_ids = [db_id[0] for db_id in db_ids]  # Flatten

for db_id in db_ids:
    db = session.query(Database).get(db_id)
    if db.password is not None:
        print(db.id, db.database_name, db.password)

session.close_all()
connection.close()
engine.dispose()

Hope it'll help!

I have upgraded superset from 1.3.0 to 1.4.2, When i run the command superset init, i get the same error which is ValueError: Invalid Decryption key. I have set it up on ubuntu machine(from scratch). Even though i am able to login into superset portal and view charts that were created in the previous version, the init command throws the error.
Let me know how to resolve it.
NOTE: For db's we are using managed database, and sqlite for superset which is default one.

In step 3 I'm getting a syntax error. Would update dbs set password = null; update dbs set encrypted_extra = null; work?

I just ran update dbs set password = null;  without any reference to encrypted_extra and it worked.  When I restarted Superset, I could access the list of Databases, and just had to update the password in the SQLALCHEMY_DATABASE_URI for the connection to work again.

@andtlopez thanks for the hint.
...
for db_id in db_ids:
    db = session.query(Database).get(db_id)
    if db.password is not None:
        print(db.id, db.database_name, db.password)
 ...

And I get
Traceback (most recent call last):
  File "/path/to/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt
    decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 1: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<console>", line 2, in <module>
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 1018, in get
    return self._get_impl(ident, loading.load_on_pk_identity)
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 1135, in _get_impl
    return db_load_fn(self, primary_key_identity)
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 286, in load_on_pk_identity
    return q.one()
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3490, in one
    ret = self.one_or_none()
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
    ret = list(self)
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
    cursor.close()
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
    compat.raise_(
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
    rows = [proc(row) for row in fetch]
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
    rows = [proc(row) for row in fetch]
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 579, in _instance
    _populate_full(
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 725, in _populate_full
    dict_[key] = getter(row)
  File "/path/to/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line 1278, in process
    return process_value(impl_processor(value), dialect)
  File "/path/to/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 477, in process_result_value
    value = super().process_result_value(value=value, dialect=dialect)
  File "/path/to/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 422, in process_result_value
    decrypted_value = self.engine.decrypt(value)
  File "/path/to/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 130, in decrypt
    raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key

Nope, this does not help.

What I do not understand is this : I can open the SQLite database with sqlite3, I can query tables, I can see data, etc. When I go in the admin interface (Superset), I can vew dashbaords, queries, etc. but the Databases list is empty!

even if there is actual data in the dbs table :
sqlite> select id, database_name FROM dbs;
1|examples
2|mysql_db_1
3|mysql_db_2

When I create a new database, using Superset, the database is created in the SQLite database, but it is still not listed.
Why is it complaining about an "invalid decryption key" when the database is fully readable! How to fix this?

This workaround https://github.com/apache/superset/issues/8538#issuecomment-1046238602, assumes that the meta-data db is using postgres; in my case it was sqlite. So the steps in my case where:

Change SECRET_KEY on superset_config.py
Access sqlite3 client and connect to the sqlite db file ( sqlite3 superset.db )
update dbs set password = null, encrypted_extra where id=?; <- encrypted_extra should be set to null
superset db upgrade
superset init
superset run
Update password on Superset: Data > Databases (I'm using 1.4.1 btw)

I only changed step 2.

Do note that superset has a cli for this:
https://superset.apache.org/docs/installation/configuring-superset/#secret_key-rotation

This workaround #8538 (comment), assumes that the meta-data db is using postgres; in my case it was sqlite. So the steps in my case where:

Change SECRET_KEY on superset_config.py
Access sqlite3 client and connect to the sqlite db file ( sqlite3 superset.db )
update dbs set password = null, encrypted_extra = null where id=?; <- encrypted_extra should be set to null
superset db upgrade
superset init
superset run
Update password on Superset: Data > Databases (I'm using 1.4.1 btw)

I only changed step 2.

i use
1/  sqlite3 superset.db
2/  delete from dbs;
delete dbs data
is ok;

As noted here and here above, there is a cli for this.

@andy-clapson superset re-encrypt-secrets doesn't help. I'm still getting the error:
2023-02-27 14:25:28,532:INFO:superset.utils.encrypt:Collecting info for re encryption
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 126, in decrypt
    decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8a in position 0: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/superset/utils/encrypt.py", line 146, in _re_encrypt_row
    unencrypted_value = previous_encrypted_type.process_result_value(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 479, in process_result_value
    value = super().process_result_value(value=value, dialect=dialect)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 424, in process_result_value
    decrypted_value = self.engine.decrypt(value)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py", line 128, in decrypt
    raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/superset", line 33, in <module>
    sys.exit(load_entry_point('apache-superset', 'console_scripts', 'superset')())
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/flask/cli.py", line 567, in main
    return super().main(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/flask/cli.py", line 407, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/app/superset/cli/update.py", line 119, in re_encrypt_secrets
    secrets_migrator.run()
  File "/app/superset/utils/encrypt.py", line 190, in run
    self._re_encrypt_row(conn, row, table_name, columns)
  File "/app/superset/utils/encrypt.py", line 163, in _re_encrypt_row
    raise Exception from exc
Exception

I had the same issue when I use docker-compose to install superset on MacOS.
Due to it is my local test, it is easy for me just to remove old containers and old data and start fresh.

docker-compose down -v (or docker-compose down and docker volume prune), this will remove the old containers and old data
docker-compose up, to start fresh

The key point is you need to remove old volumes.

this worked for me

This is probably because you already have database connections that were encrypted with the old key. You have to recreate them or update

how to recreate or update ,this page get error

i got an solution, add a config DB_SECRET_KEY in config.py it's value is origin SECRET_KEY, then use config["DB_SECRET_KEY"] replace config["SECRET_KEY"] in superset/models/core.py Database model

hi sir, can you tell me where is config["SECRET_KEY"] in superset/models/core.py Database model, really i dont find it, thankyou

This worked for me
https://superset.apache.org/docs/installation/configuring-superset/#rotating-to-a-newer-secret_key

How can I solve this problem?

  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/app/pythonpath/keycloak_security_manager.py", line 41, in login
    return handle_login()
  File "/usr/local/lib/python3.9/site-packages/flask_oidc/__init__.py", line 487, in decorated
    return view_func(*args, **kwargs)
  File "/app/pythonpath/keycloak_security_manager.py", line 38, in handle_login
    login_user(user, remember=False)
  File "/usr/local/lib/python3.9/site-packages/flask_login/utils.py", line 169, in login_user
    if not force and not user.is_active:
AttributeError: 'bool' object has no attribute 'is_active'

i used the oidc connector the docker is https://github.com/HiWay-Media/Superset-Keycloak

select database_name, id from dbs where database_name = 'New DB' or database_name = 'Original DB';
'# 5 = New DB'
'# 2 = Original DB'
update tables set database_id = 5 where database_id = 2;
update tab_state set database_id = 5 where database_id = 2;
update query set database_id = 5 where database_id = 2;
update saved_query set db_id = 5 where db_id = 2;
update table_schema set database_id = 5 where database_id = 2;
delete from dbs where database_name = 'Original DB';
update dbs set database_name = 'Original DB' where database_name = 'New DB';

This solved it for me!
also in the new superset versions, need to add:
update sl_datasets set database_id = 5 where database_id = 2;
update sl_tables set database_id = 5 where database_id = 2;

Just to help anyone else coming across this.
Trying to migrate / upgrade superset using backup of postgres metadata, shifting to new version.
I went from superset 2.0.x to 2.10.
Then seeing problems with invalid decryption key, so you see the datasets in superset ui but the database connections barf.
First, fix secret keys in superset_config.py
PREVIOUS_SECRET_KEY = 'CHANGE_ME_TO_A_COMPLEX_RANDOM_SECRET'
SECRET_KEY = 'some-really-random-key'
<= determined original key from other (original) installation by getting session on superset container:
$ docker exec -it superset_worker bash
then:
superset shell
then:

from flask import current_app; print(current_app.config["SECRET_KEY"])

<= generate from:
bash$ openssl rand -base64 42
Second, ensure to specify a version in your docker-compose config:
$ vi docker-compose-non-dev.yml
x-superset-image: &superset-image apachesuperset.docker.scarf.sh/apache/superset:2.1.0
Third, as per comments above, update the password and encrypted_extra values for your db connections in the restored metadata db:
psql: update dbs set password = null, encrypted_extra = null
Fourth, you should now be able to login to superset ui, go to database connections, and just re-enter the password(s) for your database(s) - then the dataset(s) for those databases come back to life.

modify in config.py :
SECRET_KEY = "your now value"
PREVIOUS_SECRET_KEY = "your pre value"
then exec:  superset re-encrypt-secrets

I've unfortunately encountered this and here's what I've done to fix this:

Change SECRET_KEY on superset_config.py
Access psql client and connect to the superset DB (\connect db_name;)
update dbs set password = null; (thanks @jhult), exit out
superset db upgrade
superset init
superset run
Update password on Superset: Data > Databases (I'm using 1.2.0 btw)

step 7, what uri for that?
postgresql://examples:<what_value>@db:5432/examples

<what_value> would be the password I believe.

我不明白的是：我可以打开SQLite数据库，我可以查询表，我可以查看数据等等。当我进入管理界面（Superset）时，我可以查看破折号、查询等，但数据库列表是空的！sqlite3

即使表中有实际数据：dbs
sqlite> select id, database_name FROM dbs;
1|examples
2|mysql_db_1
3|mysql_db_2

当我使用Superset创建一个新数据库时，该数据库是在SQLite数据库中创建的，但仍未列出。
为什么当数据库完全可读时，它抱怨“解密密钥无效”！如何解决这个问题？

I also met this problem, may I ask how you finally solved it?

此解决方法 #8538 （comment） 假定元数据数据库正在使用 postgres;就我而言，它是SQLITE。因此，在我的情况下，步骤是：

更改SECRET_KEY superset_config.py
访问 sqlite3 客户端并连接到 sqlite db 文件 （sqlite3 superset.db )
update dbs set password = null， encrypted_extra = null where id=？;<- encrypted_extra应设置为 null
Superset DB 升级
超集初始化
超集运行
在 Superset 上更新密码：数据>数据库（顺便说一句，我使用的是 1.4.1）

我只更改了第 2 步。

How does step 2 here access the sqlite3 client and connect to the sqlite.db file?

I unfortunately cannot remember my previous secret key (My config file got deleted, somehow), please help!!!

I unfortunately cannot remember my previous secret key (My config file got deleted, somehow), please help!!!

You can try to reset your secret key by usingopenssl rand -base64 42 andexport SUPERSET_SECRET_KEY='(just generated secret key)'.

@yjcsivsuk Can you please elaborate a little more on the steps? where should I export the key? Is the only step needed? restart?
I'm in the same case as @Teesmo, I don't have access to my previous secret key.

I'm not sure we have docs on "what to do if you lose your secret key" - is there a standard procedure? CC @dpgaspar @sfirke

Ok so I spent all day on this and finally got it so I'm going to add some more detail to help others out.
We had a previous deployment running on K8s deployed from Helm. I upgraded to 3.0.0 and tried setting the SUPERSET_SECRET_KEY in a variety of ways using the openssl rand -base64 42 with no luck.
A couple things I found to be true for me was that setting:
  extraSecretEnv:
    SUPERSET_SECRET_KEY: SomelongBase64String

did not in fact set the SECRET_KEY in the superset_config.py like the docs say it will.
Another thing that seems to be true is if you have a previous deployment you have to set a PREVIOUS_SECRET_KEY which you can only get from the previous version.
After all that, these are the steps I followed to get it to work:

From 3.0.0 I downgraded to 2.1.1 (along with helm to 0.7.7) and deployed (we use ArgoCD so it syncs with any update)
Shell logged into the running web superset running and ran cat pythonpath/superset_config.py. At the bottom there should be a SECRET_KEY='thisISaSECRET_56789' or something basic like that. Grab that string.
Upgraded to 3.0.0 and set values.yaml like this:

  extraSecretEnv:
    SUPERSET_SECRET_KEY: <some string you generate by using `openssl rand -base64 42`>
    # old default secret goes here
    PREVIOUS_SECRET_KEY: thisISaSECRET_56789

   # you could just hardcode them here but this is was what I did when I figured out `extraSecretEnv` were not being used.
  configOverrides:
    secret: |
      # Secret key for signing cookies
      SECRET_KEY = os.getenv('SUPERSET_SECRET_KEY')
      PREVIOUS_SECRET_KEY = os.getenv('PREVIOUS_SECRET_KEY')

deploy these changes.
4. shell into the web superset again and run superset re-encrypt-secrets
Hopefully this helps someone not spend the time I did figuring this out.


Use dotenv-expand

#3310

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi @bre7 😃
Thanks for the PR, I'll bring this feature request to the team.
Could you motivate this feature request maybe?
Maybe just a small description of why you want to see it happen inside Prisma? (You could do it outside Prisma maybe?)

I just saw that you linked an issue 🙈

Sure, have a look at #3310. It's mostly to use env vars without the need to configure PrismaClient manually and avoid multiple contact points.

Hi @bre7, thanks for the PR!
We're getting quite a few issues around .env. We want to spend some time understanding the problem better before jumping to a solution. We may end up going with this but we want to explore the possible solutions first.
Thanks for your patience!

I like this, just a few weeks back I was looking at how to do variable in .env files to find it is not possible.
I had another library that needed to connect to DB. But it wanted the parts separate. (I ended up using new URL(dbString) as the solution but would love to have it separate.

@bre7 can you please rebase to master? Then we can merge this :)

I fixed the conflict, PR needs to be reviewed, specifically this comment https://github.com/prisma/prisma/pull/3330/#discussion_r472607729

We could add a test so we know this works too

missing package.json updates for testing

Example usage from a .env file
USERNAME="postgres"
DATABASE_URL="postgres://${USERNAME}@localhost:5432/repro"

@bre7 Sorry about that but it looks like you will need to sign the Contributor License Agreement again here https://cla-assistant.io/prisma/prisma?pullRequest=3330
It's because we just updated it with the new company name (content is the same).

👍

👏
One last thing: might be worth it to add a comment under https://github.com/prisma/prisma/blob/master/README.md#the-prisma-schema:
datasource db {
  provider = "postgresql"
-  url      = env("DATABASE_URL")
+  url      = env("DATABASE_URL") /// variable expressions allowed (see dotenv-expand)
}

Issues in real app. Prisma isn't parsing the schema correctly (assumes public)

Missing: add test case with real connection string. Seeing some issues at the moment

Error validating datasource db: The URL for datasource db must start with the protocol postgresql://.


feat(16258): Show which protocol the user is creating/editing and adapter for

See https://hivemq.kanbanize.com/ctrl_board/57/cards/16258/details/
The PR adds the name and the icon of the protocol being edited (for both creation or modification), in the header of the side panel.
It allows the user to immediatly see which protocol the form is associated with.
Before

After

The icons are still small - but we can look to standardise these with some custom sizes moving forward.


Question: How does Pact contract interact with consensus layer?

I think of the contract as db application replicated in cluster and it exposes its functions that insert/update its protected data in tables. So in private blockchain, each function is a command that needs the the majority nodes to agree to commit, and then the function is applied  to the contract (db application) by the leader and other followers. Am I right?
Does the consensus layer know anything about the function?
Does it need to understand Pact?

So in private blockchain, each function is a command that needs the the majority nodes to agree to commit, and then the function is applied to the contract (db application) by the leader and other followers. Am I right?

Yes. There is the potential to use the output of commitTx to achieve "output consensus" but this is done in a lazy fashion, unlike the public/block context where the output/receipt is encoded into the ledger itself.

Does the consensus layer know anything about the function?
Does it need to understand Pact?

Not per se, but it does need to implement the Pact REST API to provide hashes, payload data and signatures.


Getting syntax errors while using Vite with vue3/router4

Reproduction
Details on code
Steps to reproduce the bug
<router-view v-slot="{ Component }">
    <component
      :is="Component"
      class="container"
      ref="componentRef"
    />
  </router-view>

This is my router code.
Getting below error:
[plugin:vite:vue] v-slot can only be used on components or
Tried to rewrite syntax:
<router-view>
  <template v-slot="{ Component }">
    <component
      :is="Component"
      class="container"
      ref="componentRef"
    />
  </template>
</router-view>

Getting different error:
Codegen node is missing for element/if/for node. Apply appropriate transforms first.
Expected behavior
As per error, if v-slot is not allowed on , there should be alternate approach which should work without errors.
Actual behavior
Getting error: [plugin:vite:vue] v-slot can only be used on components or
Additional information
No response

This doesn’t seem to come from the router but from the vite plugin.
BTW follow the issue guidelines next times please


[cloud_functions] Correct method channel name on iOS

Description
There was a bug fix raised here: https://github.com/FirebaseExtended/flutterfire/issues/1866. The fix was merged in 0.4.1+8, but then undid by https://github.com/FirebaseExtended/flutterfire/pull/1707.
This MR is to rename the method channel on iOS to match with the interface.
Related Issues
https://github.com/FirebaseExtended/flutterfire/issues/1866
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] If the pull request affects only one plugin, the PR title starts with the name of the plugin in brackets (e.g. [cloud_firestore])
[x] My PR includes unit or integration tests for all changed/updated/fixed behaviors (See Contributor Guide).
[x] All existing and new tests are passing.
[x] I updated/added relevant documentation (doc comments with ///).
[x] The analyzer (flutter analyze) does not report any problems on my PR.
[x] I read and followed the Flutter Style Guide.
[x] I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy.
[x] I updated CHANGELOG.md to add a description of the change.
[x] I signed the CLA.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require plugin users to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (please indicate a breaking change in CHANGELOG.md and increment major revision).
[x] No, this is not a breaking change.

@googlebot I signed it!

@collinjackson Can you have a look please? And maybe investigate why it passed the checks again?

Superseded by https://github.com/FirebaseExtended/flutterfire/pull/2051


Support the dynamic config nemesis

Feature Request
Is your feature request related to a problem? Please describe:

Describe the feature you'd like:

refs: https://github.com/pingcap/docs-cn/pull/2384/files
Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

we will move to matrix.


Crash when dismissing a modal QLPreviewController in iOS8 (ipad)

Basically, present a QLPreviewController modally in iOS8 and crash when user dismisses it.
Traced to swizzle st_parentViewController.
I 'fixed' it by doing this in UIViewController+STPopup.m (but maybe there's a better way?):

(UIViewController *)st_parentViewController
{
@try {
if (!self.popupController) {
return [self st_parentViewController];
}
}
@catch (NSException *exception) {
}
@finally {
return [[self.popupController valueForKey:@"containerViewController"] parentViewController];
}
}

That was a hard one to find:)

@ijwelch Actually, the swizzling of parentViewController is unnecessary, will delete it.