[design:suggest] Lack of space in "Create Chart" GUI

Screenshot

Description
There are a lot of cases when a chart has quite a lot of tags and a long description. When you select a chart in the gallery, an info panel appears and eats a significant chunk of space from the gallery itself. On a 13'' display there will be too little space left to browse other charts or to read the description comfortably. Especially it affects localizations for languages that are not as brief as English.
Design input
I suggest to review the layout of "Create chart" screen and probably take advantage of vertical scroll for the whole page.

I remember that we tried vertical scrolling, but having the "Create New Chart" button being offscreen caused even more significant problems. Hopefully @kasiazjc has some ideas.

I've been thinking about this for a while, and it seems we could get rid of this page altogether. Full chart editor already has the UI allowing to choose dataset and visualization type. When user hits "+ chart" button we could take them there and highlight the dataset selector.

As for chart type, we could pick flat table as the default. If another chart type is desired, user can always change it.

This modal window has a bit more space here and provides the same functionality.


please publish a new version

The last time the version was updated was 10 days ago.  The es6 rewrite was 8 days ago.  I'm trying to install the latest and I'm running into a bug that was fixed with the rewrite.

Sorry about that! merged & published.

thanks!


Router adds trailing slash to the 'base'

Version
4.0.12
Reproduction link
codesandbox.io
Steps to reproduce

Configure router to use base /base;
open your app https://your.app/base (reproduction — https://8yvwn.csb.app/base);
observe how URL changes to https://your.app/base/ (https://8yvwn.csb.app/base/);
trailing slash was added and now browser history contains 2 items.

What is expected?
Router doesn't add trailing slash to the base if URL doesn't contain one.
What is actually happening?
Router adds trailing slash and pollutes browser history.

Browsers still save history entry despite using replaceState.
Also this behavior means that if you configure router to use redirect from / to /redirected and then open app without trailing slash, you get 3 history entries:

https://your.app/base
https://your.app/base/
https://your.app/base/redirected

The history length is 1:

Note that adding the trailing slash is expected, having 2 history entries wouldn't be though but this is not currently the case.

Firefox:

Edge:

Video:

Clear demonstration of the issue:

That's expected native browser behavior because you are manually creating a new navigation. It's unrelated and uncontrollable by the router


Development roadmap

RT.js looks like an exciting project! I'm interested in exploring RT.js to orchestrate the handling of multiple data streams that are used to update multiple WebGL plots concurrently. I look forward to sharing any high-level results with you.
Are there plans to continue the development of RT.js? Are there any specific features that the authors are interested in adding to the framework?
Thanks!

Currently, we have no urging further ideas to extend RT.js. I think that such ideas could arise from the integration with a larger project, like th eone you mentioned with WebGL Plots. So, if you feel any shortcomings, feel free to make a suggestion how we can improve RT.js.


requets - Empty reply on more than 16 headers

Cause we need to give httparse a fixed size of headers, I want to get rid of it. So let's build a custom implementation of HTTP Request parsing.
It should work as the following:

Just make a new bytes::BytesMut object out of the headers
when a header is requested, search in the BytesMut and return an array of result (why an array? Headers can be defined more than one times, alternative: Keep first header value, ignore others)

This can lead to a performance improvement!?

Solving this by splitting request into pure ByteMut objects. Work in progress

Solved in branch master


播放数放回来

大概需要revert https://github.com/dd-center/vtbs.moe/commit/a31a1a2c83bdc716c9992be3a6e0b7fdfdb8f1ce#diff-14c2529eb4498c5d1ffd6915d05bf58a91bdda796af59f41d480d11c099d0479

closed in 9197b5cbf811a77d3c044a6c4b47a8e824890906


Error handling

If an error happens the plugin doesn't provide any event

You can use FGrpcResult.Result.Code in response lambda to check occurred an error

@savasozer in case of error, the lambda function is not reached.
template<typename T, typename R> class GrpcContext_Ping_Stream : public TGrpcContext<T, R> { 	typedef TGrpcContext<T, R> Super; protected: 	void OnRpcEventInternal(bool Ok, const void* EventTag, typename Super::FRpcCallbackFunc RpcCallbackFunc) 	{ 		if (!Ok) 		{ 			Super::RpcReaderWriter->Finish(&(Super::RpcStatus), Super::ReadTag); 			Super::UpdateState(EGrpcContextState::Done); 			return; 		}
As a test I deleted the authorization token from my service, in case of error it seems to go through here, but the callback is not called and for some reason Super::RpcStatus turns out to be ok

Sorry, I'm very busy with work recently and don't have time to take care of this project. I will take a look at this issue when I have time.

I think I realized how it should work in Turbolink context)
all you need to comment out one line.

when Ok=false we shouldn't call Super::UpdateState(EGrpcContextState::Done) in the OnRpcEventInternal (appropriate GrpcContext, TurboLinkGrpcContext.h)

for example:
if (!Ok)
{
	Super::RpcReaderWriter->Finish(&(Super::RpcStatus), Super::ReadTag);
	// Don't update state here because we want to wait "Finish" above
	// Super::UpdateState(EGrpcContextState::Done);

	return;
}

for better separation, maybe in the Finish function better use a different tag (FinishTag), but ReadTag works as well.

on the next AsyncNext you will receive updated grpcContext with relative RpcStatus with code_ and error_message_

finally in the OnRpcEventInternal (which will be called after AsyncNext in the relative context): Ok=true will be received (after previous Finish call) and no need to touch anything, because updated RpcStatus will be not Ok and your callback will be proceeding with error data as expected;

if (Super::RpcStatus.ok())
{
	// status isn't ok after "Finish" is being processed
	// and code removed for brevity
}
else
{
	UE_LOG(LogTurboLink, Error, TEXT("CallRpcError: %s"), *result.GetMessageString());

	if (RpcCallbackFunc)
	{
		RpcCallbackFunc(result, nullptr);
	}

	Super::UpdateState(EGrpcContextState::Done);
	return;
}

thanks again for the plugin - it's a lifesaver!

@thejinchao @MarcoForlivesi
I think I realized how it should work in Turbolink context)

when Ok=false we shouldn't call Super::UpdateState(EGrpcContextState::Done) in the OnRpcEventInternal (TurboLinkGrpcContext.h). because we need to wait when Super::RpcReaderWriter->Finish will be processed (another AsyncNext in the TurboLinkGrpcManager.cpp).
only after next processing, proper status will be written in relevant context.

also we need to implement another tag for filtering the Finish call
if (!Ok)
{
	Super::RpcReaderWriter->Finish(&(Super::RpcStatus), Super::FinishTag);

	// Don't Update State to Done here, because need to wait when this operation finished on the next AsyncNext which updates Super::RpcStatus
	// Super::UpdateState(EGrpcContextState::Done);
	return;
}

on the next AsyncNext you will receive updated grpcContext with relative RpcStatus with code_ and error_message_

finally, in the OnRpcEventInternal (which will be called after AsyncNext in the relative context):
we need to filter FinishTag after updating status and in case error push callback with wrong status

// Finish operation processed - handle error if exists and Update State to Done
if (EventTag == Super::FinishTag)
{
	if (RpcCallbackFunc && !Super::RpcStatus.ok())
	{
		UE_LOG(LogTurboLink, Error, TEXT("CallRpcError: %s"), *result.GetMessageString());

		if (RpcCallbackFunc)
		{
			RpcCallbackFunc(result, nullptr);
		}
	}

	Super::UpdateState(EGrpcContextState::Done);
	return;
}

also, in case if we sure that receiving wrong status only possible after AsyncNext isn't Ok then could omit further branch ```if (Super::RpcStatus.ok())``
slightly modified GrpcContext_Ping_Stream example:
template<typename T, typename R>
class GrpcContext_Ping_Stream : public TGrpcContext<T, R>
{
	typedef TGrpcContext<T, R> Super;
protected:
	void OnRpcEventInternal(bool Ok, const void* EventTag, typename Super::FRpcCallbackFunc RpcCallbackFunc)
	{
		if (!Ok)
		{
			Super::RpcReaderWriter->Finish(&(Super::RpcStatus), Super::FinishTag);

			// Don't Update State to Done here, because need to wait when this operation finished on the next AsyncNext which updates Super::RpcStatus
			// Super::UpdateState(EGrpcContextState::Done);
			return;
		}

		FGrpcResult result = GrpcContext::MakeGrpcResult(Super::RpcStatus);

		// Finish operation processed - handle error if exists and Update State to Done
		if (EventTag == Super::FinishTag)
		{
			if (RpcCallbackFunc && !Super::RpcStatus.ok())
			{
				UE_LOG(LogTurboLink, Error, TEXT("CallRpcError: %s"), *result.GetMessageString());

				if (RpcCallbackFunc)
				{
					RpcCallbackFunc(result, nullptr);
				}
			}

			Super::UpdateState(EGrpcContextState::Done);
			return;
		}

		// In case if we sure that receiving wrong status only possible after AsyncNext isn't Ok then could omit further branch
		if (Super::RpcStatus.ok())
		{
			if (Super::GetState() == EGrpcContextState::Initialing)
			{
				check(EventTag == Super::InitialTag);

				Super::RpcReaderWriter->Read(&(Super::RpcResponse), Super::ReadTag);
				Super::UpdateState(EGrpcContextState::Busy);
			}
			else
			{
				if (RpcCallbackFunc)
				{
					RpcCallbackFunc(result, &(Super::RpcResponse));
				}
				Super::RpcReaderWriter->Read(&(Super::RpcResponse), Super::ReadTag);
			}
		}
		else
		{
			UE_LOG(LogTurboLink, Error, TEXT("CallRpcError: %s"), *result.GetMessageString());

			if (RpcCallbackFunc)
			{
				RpcCallbackFunc(result, nullptr);
			}
			Super::UpdateState(EGrpcContextState::Done);
			return;
		}
	}
public:
	GrpcContext_Ping_Stream(FGrpcContextHandle _Handle, UGrpcService* _Service, UGrpcClient* _Client)
		: Super(_Handle, _Service, _Client)
	{
	}
};

thanks again for the plugin - it's a lifesaver!

Yes, it seems works, but you miss the part of the tag handling
    //async tag
    void* InitialTag = nullptr;
    void* WriteTag = nullptr;
	void* ReadTag = nullptr;
	void* FinishTag = nullptr;

/
void GrpcContext::UpdateState(EGrpcContextState NewState)
{
	if (GetState() == NewState) return;

	ContextState = NewState;
	if (ContextState == EGrpcContextState::Initialing)
	{
		InitialTag = Service->TurboLinkManager->GetNextTag(AsShared());
		WriteTag = Service->TurboLinkManager->GetNextTag(AsShared());
		ReadTag = Service->TurboLinkManager->GetNextTag(AsShared());
		FinishTag = Service->TurboLinkManager->GetNextTag(AsShared());
	}
	else if (ContextState == EGrpcContextState::Done)
	{
		Service->TurboLinkManager->RemoveTag(InitialTag);
		Service->TurboLinkManager->RemoveTag(WriteTag);
		Service->TurboLinkManager->RemoveTag(ReadTag);
		Service->TurboLinkManager->RemoveTag(FinishTag);
	}

	if (Client->OnContextStateChange.IsBound())
	{
		Client->OnContextStateChange.Broadcast(Handle, NewState);
	}
}

yeap, you are right, I forget to note this.
also in this case, need to add logic above to another context behaviors

@thejinchao is it possible to integrate the changes proposed by 6r0m?

I was beating my head against a wall trying to figure out what I was doing wrong, only to spend hours going down this rabbit hole. I don't know C++ GRPC very well, but this does seem like the best solution. Is someone working on a PR? That's probably the easiest way to get this fixed. I have another couple of bugs to report too, but those have (sort of) easy workaround. This doesn't.


React group project: day 3

Hi Dear Reviewer, We are a total of two persons working on this React group project: day 3.

@Kaiserabbas
@tajemouti
A Link to Space-Travelers-Hub

Hi Team,
Your project is complete!  There is nothing else to say other than... it's time to merge it :shipit:
Congratulations! 🎉
Highlights 💯

Descriptive info ✔
Tasks distributed equally ✔

Optional suggestions
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.

As described in the Code reviews limits policy  you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.


Data race on getAnsiPattern

Hello,
I have a program that is running a pool of goroutine that connect to devices to fetch configuration.
While troubleshooting another issue, I ran my program with the -race flag, and got a notice from golang:
==================
WARNING: DATA RACE
Read at 0x0000030df390 by goroutine 33:
  github.com/scrapli/scrapligo/util.getAnsiPattern()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/util/bytes.go:14 +0x3c
  github.com/scrapli/scrapligo/util.StripANSI()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/util/bytes.go:23 +0x93
  github.com/scrapli/scrapligo/channel.(*Channel).read()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/channel/read.go:39 +0x29e
  github.com/scrapli/scrapligo/channel.(*Channel).Open.func1()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/channel/channel.go:127 +0x39

Previous write at 0x0000030df390 by goroutine 47:
  github.com/scrapli/scrapligo/util.getAnsiPattern()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/util/bytes.go:15 +0x68
  github.com/scrapli/scrapligo/util.StripANSI()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/util/bytes.go:23 +0x93
  github.com/scrapli/scrapligo/channel.(*Channel).read()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/channel/read.go:39 +0x29e
  github.com/scrapli/scrapligo/channel.(*Channel).Open.func1()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/channel/channel.go:127 +0x39

Goroutine 33 (running) created at:
  github.com/scrapli/scrapligo/channel.(*Channel).Open()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/channel/channel.go:127 +0x211
  github.com/scrapli/scrapligo/driver/netconf.(*Driver).Open()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/driver/netconf/driver.go:210 +0x175
  stash.ovh.net/network/netconf-proxy/pkg/application.(*NetconfProxy).driverFactory()
      /Users/netixx/dev/netconf-proxy/pkg/application/proxy.go:213 +0x8f5
  stash.ovh.net/network/netconf-proxy/pkg/application.(*NetconfProxy).query()
      /Users/netixx/dev/netconf-proxy/pkg/application/proxy.go:131 +0xe5
  stash.ovh.net/network/netconf-proxy/pkg/application.(*NetconfProxy).processQuery()
      /Users/netixx/dev/netconf-proxy/pkg/application/proxy.go:110 +0x4be
  stash.ovh.net/network/netconf-proxy/pkg/application.(*NetconfProxy).Querier()
      /Users/netixx/dev/netconf-proxy/pkg/application/proxy.go:93 +0x1a6
  main.(*Application).Start.func2()
      /Users/netixx/dev/netconf-proxy/cmd/app.go:218 +0x39

Goroutine 47 (running) created at:
  github.com/scrapli/scrapligo/channel.(*Channel).Open()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/channel/channel.go:127 +0x211
  github.com/scrapli/scrapligo/driver/netconf.(*Driver).Open()
      /Users/netixx/.gvm/pkgsets/go1.20.3/global/pkg/mod/github.com/scrapli/scrapligo@v1.1.7/driver/netconf/driver.go:210 +0x175
  stash.ovh.net/network/netconf-proxy/pkg/application.(*NetconfProxy).driverFactory()
      /Users/netixx/dev/netconf-proxy/pkg/application/proxy.go:213 +0x8f5
  stash.ovh.net/network/netconf-proxy/pkg/application.(*NetconfProxy).query()
      /Users/netixx/dev/netconf-proxy/pkg/application/proxy.go:131 +0xe5
  stash.ovh.net/network/netconf-proxy/pkg/application.(*NetconfProxy).processQuery()
      /Users/netixx/dev/netconf-proxy/pkg/application/proxy.go:110 +0x4be
  stash.ovh.net/network/netconf-proxy/pkg/application.(*NetconfProxy).Querier()
      /Users/netixx/dev/netconf-proxy/pkg/application/proxy.go:93 +0x1a6
  main.(*Application).Start.func2()
      /Users/netixx/dev/netconf-proxy/cmd/app.go:218 +0x39

I replaced :
func getAnsiPattern() *regexp.Regexp {
	if ansiPattern == nil {
		ansiPattern = regexp.MustCompile(ansi)
	}

	return ansiPattern
}

with :
var ansiPattern *regexp.Regexp = regexp.MustCompile(ansi) //nolint: gochecknoglobals

func getAnsiPattern() *regexp.Regexp {
	return ansiPattern
}

And it seems good now (while running the same test). Any reason for the "getAnsiPattern" to be the way it was ?
We could also remove the getAnsiPattern altogether and use ansiPattern directly.
I also had a memory leak, which I think could be related to that (because some reference to ansiPattern may get lost by the data race ?). The image below shows part of the heap dump, where we see that channel read is not attached to anything.

I think this was done to avoid having global vars, but I'll let @carlmontanari to confirm that.
For a similar function we did a Once trick -- https://github.com/scrapli/scrapligo/blob/e35f8e521d19187d75b2cfe7db33a647e2288aae/response/netconf.go#L31-L40
but since ansiPatter is still a global var already, maybe there is no need for that and your fix is a legit one.

ahh yeah good find!
this should probably live behind a do once like @hellt posted. I guess also just as a global is totally fine, but just to try to be consistent, and also so that (in this case) its never compiled until needed the first time we can roll w/ the once thing.
I'll get a branch together this afternoon to share and if everyone is happy with that we can get it merged! thanks for sharing this @netixx !

okidokie, does #146 look reasonable (I should hope I didnt mess that up, all one line change and all! 🤣)

looks good to me :)
thanks for the fix!!


Change validation message

Is there any way to change this message, may be confusing for users

You can pass whatever you want to the error messages, don't have to output the pydantic validation error exactly, this was just an example.

Hi @RenaudLN
Sorry I am struggling with exactly how to do this, any guidance would be great, first time with pydantic.
Just want to say "Invalid email" instead of "String should match pattern.."
Have tried both model_validator and field_validator, but nothing happens and in the model_validator, there are no messages.
class LoginData(BaseModel):
    email: str = Field(
        title="Email",
        pattern=email_regex,
    )
    password: str = Field(
        title="Password",
        min_length=6,
    )

    @field_validator("email")
    @classmethod
    def validate_x(cls, v: str) -> str:
        if not re.match(email_regex, v):
            raise ValueError("Custom error: Invalid email format.")
        return v

    @model_validator(mode="before")
    @classmethod
    def validate_email(cls, data: Any) -> Any:
        if not re.match(email_regex, data.get("email")):
            raise ValueError("Custom error: Invalid email format.")
        return data

I assume you're using the example in usage.py, you can change the message that is surfaced here
errors = {SEP.join([str(x) for x in error["loc"]]): error["msg"] for error in e.errors()}
#                                                                       ^ This here comes from pydantic validation
# You could do something like:
errors = {SEP.join([str(x) for x in error["loc"]]): f"Invalid {error['loc'][-1]}" for error in e.errors()}

Awesome thank you!


Remove unnecessary platform specific code

Fixes #17

thanks. reviewing. will merge

When are you able to merge it? This blocks production on https://github.com/NearHuscarl/flutter_login
Thanks in advance

give me 10mins let me merge it


[devops]: Remove references to internal/vendor packages

Related Ticket(s)
Closes #12008
Description
Adjusts all internal/vendor imports to use the package imports instead.
Changelog
Changed

All imports of the form [...]/internal/vendor/@carbon/[package][rest] changed to @carbon/[package]/es[rest]

Removed

vendor build steps

Passes all the tests, and I don't see any stragglers still importing from internal/vendor.
Even after yarn clean yarn install and yarn build, I am still seeing built files importing from those paths, but maybe that can be a followup?

I see one place where those are coming from. Fix incoming.

Passes all the tests, and I don't see any stragglers still importing from internal/vendor.
Even after yarn clean yarn install and yarn build, I am still seeing built files importing from those paths, but maybe that can be a followup?

I see one place where those are coming from. Fix incoming.

Fixed in https://github.com/carbon-design-system/carbon-for-ibm-dotcom/pull/12010/commits/86ef4f5dbbdc49119b28beb53eabdb7ba9c90484. There was a build script for icons that was automating creation of modules that used an internal/vendor import path.

Looks good - should we remove the documentation around the vendor dirs?
https://github.com/carbon-design-system/carbon-for-ibm-dotcom/blob/main/packages/web-components/IMPLEMENTATION_NOTES.md#vendor-directory

Looks good - should we remove the documentation around the vendor dirs?
https://github.com/carbon-design-system/carbon-for-ibm-dotcom/blob/main/packages/web-components/IMPLEMENTATION_NOTES.md#vendor-directory

Addressed in https://github.com/carbon-design-system/carbon-for-ibm-dotcom/pull/12010/commits/e47e8c17a1faef713e3a51384ded928293744149

@kennylam There was a big merge conflict here after the CWC removal PR landed. It was caused by a lot of overlap in adjusting import statements. I've addressed the conflicts and this should now be ready to go.


Add documentation

Update version to 1.0.0.
Resolves #31

Coverage remained the same at 93.827% when pulling d28f0a5a0bf319af5f62a9e1ee3c30a3ab4fdad1 on initial-documentation into f93d9305c64766e422d15364f2554b5e3796b5ad on master.

Coverage remained the same at 93.827% when pulling 51ac36ae57e5bb245fcf0c5d263360f858dde40b on initial-documentation into f93d9305c64766e422d15364f2554b5e3796b5ad on master.


403 while Downloading a new version of Bun

Run oven-sh/setup-bun@v1
Downloading a new version of Bun: https://bun.sh/download/latest/linux/x64?avx2=true&profile=false
Error: Error: Unexpected HTTP response: 403

Does https://bun.sh/download has some kind of WAF or rate limiting mechanism? When I manually run curl on the runner it displayed <title>Just a moment...</title> but I coudn't see the reason.
Also I can confirm that directly curl https://github.com/oven-sh/bun/releases/download/bun-v1.0.33/bun-linux-x64-baseline.zip is ok.
P.S. We use self-hosted runners and utilize the cloudflare WARP, but I think it shouldn't be a problem to setup bun.

i get the same but with a 404 and only on our windows build environment

i get the same but with a 404 and only on our windows build environment

https://github.com/oven-sh/setup-bun/pull/72

@networkhermit are you still experiencing this issue? If you want to try curl, don't forget to add -L as the API redirects directly to github:
curl -O -L "https://bun.sh/download/latest/linux/x64?avx2=true&profile=false"

@networkhermit are you still experiencing this issue? If you want to try curl, don't forget to add -L as the API redirects directly to github:
curl -O -L "https://bun.sh/download/latest/linux/x64?avx2=true&profile=false"

No. We switched to another ci runner group since the 403 error.
I know the -L flag of curl, but instead of 302 it got 403 from cloudflare at that time, meaning it could never redirect to github.
We will try to use the previous runner group in the the next few weeks. I will report here once the issue happens again.

@networkhermit are you still experiencing this issue? If you want to try curl, don't forget to add -L as the API redirects directly to github:
curl -O -L "https://bun.sh/download/latest/linux/x64?avx2=true&profile=false"

No. We switched to another ci runner group since the 403 error.
I know the -L flag of curl, but instead of 302 it got 403 from cloudflare at that time, meaning it could never redirect to github.
We will try to use the previous runner group in the the next few weeks. I will report here once the issue happens again.

Okay, great!

Are you still experiencing this issue?

No. I think this issue can be closed.


Release Checklist 2.11.0-RC2

OpenShift Pipeline

[x] cStor
[x] Jiva
[ ] cStor-csi
[ ] Jiva-csi
[x] LocalPV

Konvoy Pipeline

[x] cStor
[x] Jiva
[x] cStor-csi
[x] Jiva-csi
[x] LocalPV

Native K8s Pipeline

[x] LVM-LocalPV
[x] ZFS LocalPV

Upgrade Pipelines [spc/cstor/jiva]

[x] 1.10.0 to 2.11.0-RC2
[x] 1.11.0 to 2.11.0-RC2
[x] 1.12.0 to 2.11.0-RC2
[x] 2.0.0 to 2.11.0-RC2
[x] 2.1.0 to 2.11.0-RC2
[x] 2.2.0 to 2.11.0-RC2
[x] 2.3.0 to 2.11.0-RC2
[x] 2.4.0 to 2.11.0-RC2
[x] 2.5.0 to 2.11.0-RC2
[x] 2.6.0 to 2.11.0-RC2
[x] 2.7.0 to 2.11.0-RC2
[x] 2.8.0 to 2.11.0-RC2
[x] 2.9.0 to 2.11.0-RC2
[x] 2.10.0 to 2.11.0-RC2

cspc/cstor csi volume upgrade

[x] 2.0.0 to 2.11.0-RC2
[x] 2.1.0 to 2.11.0-RC2
[x] 2.2.0 to 2.11.0-RC2
[x] 2.3.0 to 2.11.0-RC2
[x] 2.4.0 to 2.11.0-RC2
[x] 2.5.0 to 2.11.0-RC2
[x] 2.6.0 to 2.11.0-RC2
[x] 2.7.0 to 2.11.0-RC2
[x] 2.8.0 to 2.11.0-RC2
[x] 2.9.0 to 2.11.0-RC2
[x] 2.10.0 to 2.11.0-RC2

ZFS Upgrade Pipelines

[x] 1.0.0 to 1.9.0-RC2
[x] 1.0.1 to 1.9.0-RC2
[x] 1.1.0 to 1.9.0-RC2
[x] 1.2.1 to 1.9.0-RC2
[x] 1.3.0 to 1.9.0-RC2
[x] 1.4.0 to 1.9.0-RC2
[x] 1.5.0 to 1.9.0-RC2
[x] 1.6.0 to 1.9.0-RC2
[x] 1.7.0 to 1.9.0-RC2
[x] 1.8.0 to 1.9.0-RC2

LVM upgrade pipeline

[x] 0.4.0 to 0.7.0-RC2
[x] 0.5.0 to 0.7.0-RC2
[x] 0.6.0 to 0.7.0-RC2

Migration spc pool to cspc and cstor volume to cstor csi volume

[x] 2.0.0 to 2.11.0-RC2
[x] 2.1.0 to 2.11.0-RC2
[x] 2.2.0 to 2.11.0-RC2
[x] 2.3.0 to 2.11.0-RC2
[x] 2.4.0 to 2.11.0-RC2
[x] 2.5.0 to 2.11.0-RC2
[x] 2.6.0 to 2.11.0-RC2
[x] 2.7.0 to 2.11.0-RC2
[x] 2.8.0 to 2.11.0-RC2
[x] 2.9.0 to 2.11.0-RC2
[x] 2.10.0 to 2.11.0-RC2

While removing the cstor pools BlockDevice stuck into Released state.  https://github.com/openebs/openebs/issues/3413


The README file in this repo has a bad link - [404:NotFound]

The markup version of the readme that is displayed for the main page in this repo contains the following bad link:
"Context argument naming/positioning" & “Context propagation”
Status code [404:NotFound] - Link: https://github.com/apple/swift-distributed-tracing/tree/wip-readme-release#context-propagation-by-explicit-loggingcontext-passing
They should just be: #context-propagation-by-explicit-loggingcontext-passing
Extra
This bad link was found by a tool I recently created as part of an new experimental hobby project: https://github.com/MrCull/GitHub-Repo-ReadMe-Dead-Link-Finder
Re-check this Repo using the tool’s website: http://githubreadmechecker.com/Home/Search?SingleRepoUri=https%3A%2F%2Fgithub.com%2Fapple%2Fswift-distributed-tracing
If this has been helpful, or if you have any feedback on the tool itself, then please feel free to share your thoughts by adding a comment here, or adding to a “Discussion” in the tool’s Repo.

Thanks :-) hah yeah the never ending battle to fix dead links... definitely toolable, thanks I'll have a look at your project too :)

@swift-server-bot test this please


Regular user do not need login

In stead of login and user role based access control, a better solution can be verification of email
if the email address it not verified, the votes and comments are not recorded
if the email address is verified, the votes and comments are saved into database and the client do not need to provide email next time accessing the website
sequenceDiagram
    client->>server: vote
    client->>server: comment
    client->>server: vote
    client->>server: email address
    server->>client: API-key
    client->>server: vote
    client-->>server: [access verification link]
    server->>server: commit above votes and comments into database
    server->>server: remember the API-key for the same client

Need to test and verify the implementation


Import host should not be shown in the local cluster option

Internal reference: SURE-3614
Reported in 2.5.9.
Issue description:
This "import host" option is for displaying the command again for adding other clusters.
It should not be in the local cluster options list
Business impact:
The user was confused about what this option will do.
Repro steps:
just log in to the UI
Actual behavior:
"import host" is in the options list of the local cluster.
Expected behavior:
"import host" should not be in the options list of the local cluster.

@nwmac to find out what this is really supposed to do.

✅ Test Passed on Rancher version "v2.7-f9291747e2cd869c468a2d93fbff92b6c54b4cb3-head"

Component
Version

Rancher
v2.7-f9291747e2cd869c468a2d93fbff92b6c54b4cb3-head

User Interface
vmaster-dev

Dashboard
master "113e1d30a"

Validation Steps:

Install Rancher version v2.7-head
Login to Ember UI: https://<rancher_IP>/g/clusters
Verify that the option to "Import host" is NOT available for the local cluster.


Odysee Autoencoding Compression Issues

This is actually a Odysee bug, but because there's no Odysee repo, I thought I'd put it here.
The problem is this. If you are a channel that has been granted auto encoding, you also lose the ability to share the raw file on Odysee - where the majority of views watch the content.
For example. I recently published, via Odysee, a 3D animation which was set in a dark sense. This made it very susceptible to artefacts when compressed. As a result, to get it looking good and steamtable at the same time, I had to adjust some of the more complex compression options (as suggested in the YouTube tech docs), such as encoding with CBR, keyframes under 2, High Profile, and 2 consecutive B frames.
I tweaked each setting, frame rate and data rate multiple times until I got the best video quality vs streamability.
However, when uploaded to Odysee, my already compressed file was compressed again, but without the correct settings, which meant the artefacts returned leaving the video looking blocky and low quality.
While in most cases the auto encoring is great, it would also be nice if there was an option to use the original file too, which channels that don't have the auto encoding feature already use.

I uploaded a bigger file, so when it gets compressed, it'll compress better (a common trick used on other social media platforms). However, now it's not auto encoding. Now the viewers can only use a file that doesn't steam well.

This is something we're discussing on the transcoder side for odysee services. I've made sure to bring it up a few times. Right now we compress to heavily.


Update sbt-ci-release to 1.5.4

Updates com.geirsson:sbt-ci-release from 1.5.3 to 1.5.4.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.geirsson", artifactId = "sbt-ci-release" } ]

labels: sbt-plugin-update, semver-patch

Codecov Report

Merging #2 (dd42279) into master (5e76e1e) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #2   +/-   ##
=======================================
  Coverage   97.05%   97.05%
=======================================
  Files          23       23
  Lines         884      884
  Branches        5        5
=======================================
  Hits          858      858
  Misses         26       26

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e76e1e...dd42279. Read the comment docs.


Validation issue with<PHONE_NUMBER> (IN)

Please read the "guidelines for contributing" (linked above) and fill in the template below.
Country/region affected (e.g., "US"): IN
Example number(s) affected ("+1<PHONE_NUMBER>"):<PHONE_NUMBER>
The phone number range(s) to which the issue applies ("+1 555 555-XXXX"):
The type of the number(s) ("fixed-line", "mobile", "short code", etc.):
The cost, if applicable ("toll-free", "premium rate", "shared cost"):
Supporting evidence (for example, national numbering plan, announcement from mobile carrier, news article): IMPORTANT - anything posted here is made public. Read the guidelines first!
link to demo

@Deepakjaikalyani777,
Thank you for using libphonenumber!
The mentioned issue is already fixed in our library and will be released within this week.

@kkeshava ,
Thanks for the update.

Fixed in v8.3.0

Okay, Thanks.


remove deprecated custom_transforms code

This went in as 35fcf2e, but for some reason our source-management bot didn't mark this PR as merged. I'll close it manually!


Conflict with...  cursor="rayOrigin: mouse"

First of all, congratulations for your great work, it's awesome.
The latest a-frame version 0.6.1 now includes Mayognaise "aframe-mouse-cursor-component" which works fine on its own.  I'm trying to integrate this component into aframe-ar by adding: cursor="rayOrigin: mouse"  on the a-marker-camera but I get this error:

three.js:15179 Uncaught TypeError: Cannot read property 'count' of undefined
at Lt.Object.assign.raycast (three.js:15179)
at Xn (three.js:39252)
at jn.intersectObjects (three.js:39329)
at i.tick (raycaster.js:167)
at a-scene.js:517
at Array.forEach ()
at HTMLElement.value (a-scene.js:515)
at HTMLElement.value (a-scene.js:562)
at HTMLElement.render (bind.js:12)
at e (a-scene.js:459)

This is the code I'm using:

Everything works fine until I add... cursor="rayOrigin: mouse". I guess this is not currently supported because it's a new aframe component, but I was wondering if anyone else has tried to use it with success. And if not, where can I start digging into to try to fix this issue?
Thanks in advance
Keep up the awesomeness !

I cam to post the exact same issue so +1 on this. Happens both on desktop chrome and iOS11 Safari.

here this is due to the camera having a custom projection matrix. and this being trouble with three.js. aka this is a complex hairy problem to fix.
that said here people just want to be able to click on object. and this one could be provided more easily! let me reread this issue under this angle :)

@dreinstein10 sorry i dunno the cursor= stuff. can you provide me a simple example of it working in webvr. Just for me to quickly learn more about this component. i have the feeling it will be faster for resolving this issue

from here - https://mayognaise.github.io/aframe-mouse-cursor-component/basic/index.html

    <link rel="stylesheet" type="text/css" href="../common.css"/>

ok so i got a lead...
Currently aframe-mouse-cursor-component use the rendering camera to do the raycasting. This is perfect in a normal case. But here it trigger an issue in three.js raycasting and how it interfere with the AR camera calibration. So we need to work around that. and this can be done if aframe-mouse-cursor-component can use the camera we would specify and not the rendering camera.
hello @mayognaise, are you following all this ? :)

at first sight, it seems around here
https://github.com/mayognaise/aframe-mouse-cursor-component/blob/master/index.js#L396

@jeromeetienne This has slightly changed (hence that cursor= part) as A-Frame 0.6.1 from 15 days ago has rolled the functionality in with a similar, but not duplicate implementation 0.6.1 Release notes
See this pull request

Hi @dmarcos, do you know about this ?

Hi,
I also got this error..
Is this problem solved?

Also got bit by this. Dear @jeromeetienne, could you make any progress thus far?

This is afflicting me also

Having issues with this as well. Got to the end of my MVP and realized I have to choose between click functionality on mobile and the camera lol. @jeromeetienne let me know if I can help in anyway.

any progress here, need to be able to detect a click on an element urgently, but so far clicks are detected across the entire canvas  and screen, how could we detect a click on an specific element triggered by the marker?

I was able to workaround this issue by using event-set-component in addition to mouse-cursor-component. Was able to set visual properties, although no luck with running functions from clicks.
Ex using:
a-frame 0.7.1,
event-set-component (4.0.0)
mouse-cursor-component
a-frame-ar
I use javascript to create a box with the mouse click functionality
var box = document.createElement('a-box');
plane.setAttribute('event-set__down','_event: mousedown; scale: 2 2 2')
scene.appendChild(box);

Hello, I am facing the same problem, I want to click on the object, but it is not work. Here are my code.
`

  `

and the click js is
`AFRAME.registerComponent('changescene', {
schema: {
event: {type: 'string', default: ''},
linkto: {type: 'string', default: ''}
},
multiple: true,
init: function () {
var self = this;
},
update: function () {
var data = this.data;
var el = this.el;
if (data.event) {
console.log("googd");
el.addEventListener(data.event, function () {
console.log("click successful");
alert("clicked");
});
}
}
});
`
Thanks.

Hello,
I search an solution for this problem, and I have try to use a Three.js raytracer for get a clickable element. But I' noob, so I grope and I fail...
this is my function :  (in orign I code in Coffee Script, but I translate for you in es6 (more and less))
AFRAME.registerComponent( 'clickable-ar', {
init: () => {

		document.addEventListener('click', () => {

			raycaster = new THREE.Raycaster()
			mouse = new THREE.Vector2()

			mouse.x = ( event.clientX / window.innerWidth ) * 2 - 1
			mouse.y = - ( event.clientY / window.innerHeight ) * 2 + 1

			# raycaster.setFromCamera( mouse.clone(), AFRAME.scenes[0].camera )
			# objects = raycaster.intersectObjects( AFRAME.scenes[0].sceneEl.children )

			# position = {x: 0, y: 0, z: 0} and  direction = {x: 0, y: 0, z: -1} )
			raycaster.set( AFRAME.scenes[0].camera.position, AFRAME.scenes[0].camera.getWorldDirection() )
			elts = raycaster.intersectObject( this, true )

			for( elt in elts ) {
				elt.click();
			}
	});
}

});
I' want to know if I'm in a good way, or it's ridiculus ? And any suggestion..
Sorry for my very bad English.. I'm French :P
And thank you for your contribution !
Ygg.

Hi guys.
After a real nightmare session I've managed to get AFrame fuse cursor working with marker-camera. It stopped disappearing on marker found.
It's kind of hack/exploit, but works really well. Below only the parts that matter (please note there is no hyphen in a-marker-camera):

and for the event, in my case, this:

If @enquel's workaround/hack was working before, it isn't now!

This PR has been closed due to inactivity. Please open a new, updated issue if the problem or the question is still relevant.

Everything became extremely slow, when adding rayOrigin: mouse
(I need to handle click event on 3D-model - that's why I use rayOrigin: mouse)

so it looks like no one has really figured this out still. I got it working in react web ar, but does not work well. Triggering animations by clicking entities has some issues with recognizing the click. the raycaster is not fine tuned enough. Please I hope someone finally figures this out, or updates ar.js. I wish I could be of more help. I will post my react example when I have some time to do a write up.

I got it working with a hacky solution. It is far from perfect but test users didn't notice anything.
The problem
As far as I understand the problem is that arjs uses a different projection matrix than the actual rendering camera, so when you initiate a mouse raycast it is not aimed properly as it uses a camera with different properties.
None of the solutions I found online seemed to work on a mobile.
The ugly solution
My logic was that a raycast originating from the cursor (center) should aim at the same direction no matter the camera projection matrix. So when a user clicks anywhere on the screen we can send a raycast from the cursor and then calculate the offset using the collision point, distance and angle.

Create a large plane as a child of the <a-marker and give it the .collidable class
When the user clicasdks on the screen send a raycast from the cursor
Detect the collision point on the plane
Calculate the actual desired collision position (get where the user clicked on the screen, multiply it by a distance factor and place it on the plane accordingly - this is your raycast hit)
Cycle through your real collidable objects and check if the distance between the new hitpoint and the the collidbale is bellow your threshold.

HTML:

JS:
AFRAME.registerComponent("raycaster-listen", {
  init: function () {
      let el = this.el
      let myRaycaster
      let canvas = this.el.sceneEl.canvas;
      let clickPosUpdated = new THREE.Vector3();
      let distanceMultiplierX = .6;
      let distanceMultiplierY = .45;
      let isMobile = this.el.sceneEl.isMobile;
      this.el.addEventListener("raycaster-intersected", evt => {
          this.raycaster = evt.detail.el;
          myRaycaster = this.raycaster;
      });
      this.el.addEventListener("raycaster-intersected-cleared", evt => {
          this.raycaster = null;
          myRaycaster = null
      });

      canvas.addEventListener("touchstart", function (evt) {
          console.log(evt)
          // GET MOUSE XY
          var _canvasSize = canvas.getBoundingClientRect(),
              w = _canvasSize.width,
              h = _canvasSize.height,
              offsetW = _canvasSize.left,
              offsetH = _canvasSize.top;
          var cx = void 0,
              cy = void 0;
          if (isMobile) {
              var touches = evt.touches;
              if (!touches || touches.length !== 1) {
                  return;
              }
              var touch = touches[0];
              cx = touch.clientX;
              cy = touch.clientY;
          } else {
              cx = evt.clientX;
              cy = evt.clientY;
          }
          cx = cx - offsetW;
          cy = cy - offsetH;
          var mouseX = (cx / w) * 2 - 1;
          var mouseY = -(cy / h) * 2 + 1;

          // RAYCAST
          let intersection = myRaycaster.components.raycaster.getIntersection(el);
          if (!intersection) {
              return;
          }

          // CALCULATE UDPATED POS
          clickPosUpdated.x = intersection.point.x + intersection.distance * distanceMultiplierX * mouseX;
          clickPosUpdated.y = intersection.point.y + intersection.distance * distanceMultiplierY * mouseY;
          clickPosUpdated.z = intersection.point.z; // ToDo - calculate the Z offset based on the angle of the collision

          // SET THE DEBUG BOX TO SEE WHERE IT HIG
          debugBox.setAttribute("position", clickPosUpdated);

      });
  }
});

Left to do: calculate the Z offset based on the angle of the collision / rotation of the marker.

What’s the difference about the ar.js camera? Is not a perspective one with
a pose? Has the cursor ever worked? When did it break?
Diego
On Tue, Mar 31, 2020 at 1:12 AM Nikola Georgiev<EMAIL_ADDRESS>wrote:

I got it working with a hacky solution. It is far from perfect but test
users didn't notice anything.
The problem
As far as I understand the problem is that arjs uses a different
projection matrix than the actual rendering camera, so when you initiate a
mouse raycast it is not aimed properly as it uses a camera with different
properties.
None of the solutions I found online seemed to work on a mobile.
The ugly solution
My logic was that a raycast originating from the cursor (center) should
aim at the same direction no matter the camera projection matrix. So when a
user clicks anywhere on the screen we can send a raycast from the cursor
and then calculate the offset using the collision point, distance and angle.

Create a large plane as a child of the <a-marker and give it the
.collidable class
When the user clicasdks on the screen send a raycast from the cursor
Detect the collision point on the plane
Calculate the actual desired collision position (get where the user
clicked on the screen, multiply it by a distance factor and place it on the
plane accordingly - this is your raycast hit)
Cycle through your real collidable objects and check if the
distance between the new hitpoint and the the collidbale is bellow your
threshold.

HTML:

JS:
AFRAME.registerComponent("raycaster-listen", {
init: function () {
let el = this.el
let myRaycaster
let canvas = this.el.sceneEl.canvas;
let clickPosUpdated = new THREE.Vector3();
let distanceMultiplierX = .6;
let distanceMultiplierY = .45;
let isMobile = this.el.sceneEl.isMobile;
this.el.addEventListener("raycaster-intersected", evt => {
this.raycaster = evt.detail.el;
myRaycaster = this.raycaster;
});
this.el.addEventListener("raycaster-intersected-cleared", evt => {
this.raycaster = null;
myRaycaster = null
});
  canvas.addEventListener("touchstart", function (evt) {
      console.log(evt)
      // GET MOUSE XY
      var _canvasSize = canvas.getBoundingClientRect(),
          w = _canvasSize.width,
          h = _canvasSize.height,
          offsetW = _canvasSize.left,
          offsetH = _canvasSize.top;
      var cx = void 0,
          cy = void 0;
      if (isMobile) {
          var touches = evt.touches;
          if (!touches || touches.length !== 1) {
              return;
          }
          var touch = touches[0];
          cx = touch.clientX;
          cy = touch.clientY;
      } else {
          cx = evt.clientX;
          cy = evt.clientY;
      }
      cx = cx - offsetW;
      cy = cy - offsetH;
      var mouseX = (cx / w) * 2 - 1;
      var mouseY = -(cy / h) * 2 + 1;

      // RAYCAST
      let intersection = myRaycaster.components.raycaster.getIntersection(el);
      if (!intersection) {
          return;
      }

      // CALCULATE UDPATED POS
      clickPosUpdated.x = intersection.point.x + intersection.distance * distanceMultiplierX * mouseX;
      clickPosUpdated.y = intersection.point.y + intersection.distance * distanceMultiplierY * mouseY;
      clickPosUpdated.z = intersection.point.z; // ToDo - calculate the Z offset based on the angle of the collision

      // SET THE DEBUG BOX TO SEE WHERE IT HIG
      debugBox.setAttribute("position", clickPosUpdated);

  });

}
});
Left to do: calculate the Z offset based on the angle of the collision
/ rotation of the marker.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jeromeetienne/AR.js/issues/148#issuecomment-606470700,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAJTLTNWYIPRSSWW76IBZTRKGQXJANCNFSM4DVKNSBA
.

What’s the difference about the ar.js camera? - I never found out. I based my assumption on jeromme etiene's comment saying that this is "due to the camera having a custom projection matrix.".
Has the cursor ever worked? - the cursor seems to work fine if it is not setup with rayOrigin: mouse
When did it break? - I've only tried it in the past month
I had a job for a client where users had to tap on moving objects in arjs. The default a-frame implementation (cursor="rayOrigin: mouse") did not work with arjs. I looked all over the Internet for solutions and did not find anything working on mobile (there was smth for desktop though). This topic was the closest I got to an explanation or solution to the issue.
In one of the earlier comments jeromme etiene says that this is "due to the camera having a custom projection matrix.". Based on this I tried to inspect the three scene and see if I can do anything about the camera. I failed and never really got to the bottom of the problem.
Pressed by the project's timeline I tried a few workarounds and the one I've written above gave decent results so I shared it :)

@kvabakoma Hi There! I tried your code.
There is debugBox  variable not specified when trying to access to it (I think a querySelector is missed). I have to dig on it better, in general. Did you have finished it?
Also, I suggest you all to continue discussing this on this PR, on the new Repo, this one has been deprecated: https://github.com/AR-js-org/AR.js/issues/66

@nicolocarpignoli thanks for pointing out the code ain't working. It seems I've put a wrong version. I've updated my previous post with what should be working.
You can also see a demo here: link - it uses ontouchstart so you should test it on mobile or in dev tools by emulating a phone.
Will share it on the other thread as well. Cheers :)


generating WACZ without Docker - wacz not working

Getting a proxy connection failed on the  wacz_archiver_enricher on all urls.
First time I've set this up, so probably something simple / maybe I've missed something.
Next step for me is to setup a local dev version and debug it.. but this issue may be useful for others at the same stage as me.
I have the profile setup in secrets/profile.tar.gz which I did via
# create a new profile
docker run -p 6080:6080 -p 9223:9223 -v $PWD/crawls/profiles:/crawls/profiles/ -it webrecorder/browsertrix-crawler create-login-profile --url "https://twitter.com/"

Output of the run is:
docker run --rm -v $PWD/secrets:/app/secrets -v $PWD/local_archive:/app/local_archive bellingcat/auto-archiver --config secrets/orchestration.yaml

2023-08-22 10:50:17.450 | INFO     | auto_archiver.core.config:parse:111 - FEEDER: gsheet_feeder
2023-08-22 10:50:17.450 | INFO     | auto_archiver.core.config:parse:112 - ENRICHERS: ['hash_enricher', 'wacz_archiver_enricher']
2023-08-22 10:50:17.450 | INFO     | auto_archiver.core.config:parse:113 - ARCHIVERS: ['wacz_archiver_enricher']
2023-08-22 10:50:17.450 | INFO     | auto_archiver.core.config:parse:114 - DATABASES: ['gsheet_db']
2023-08-22 10:50:17.450 | INFO     | auto_archiver.core.config:parse:115 - STORAGES: ['local_storage']
2023-08-22 10:50:17.450 | INFO     | auto_archiver.core.config:parse:116 - FORMATTER: html_formatter
2023-08-22 10:50:24.319 | INFO     | auto_archiver.feeders.gsheet_feeder:__iter__:48 - Opening worksheet ii=0: wks.title='Sheet1' header=1
2023-08-22 10:50:26.275 | WARNING  | auto_archiver.databases.gsheet_db:started:28 - STARTED Metadata(status='no archiver', metadata={'_processed_at': datetime.datetime(2023, 8, 22, 10, 50, 26, 274503), 'url': 'https://twitter.com/dave_mateer/status/1505876265504546817'}, media=[])
2023-08-22 10:50:26.916 | INFO     | auto_archiver.core.orchestrator:archive:85 - Trying archiver wacz_archiver_enricher for https://twitter.com/dave_mateer/status/1505876265504546817
2023-08-22 10:50:26.916 | DEBUG    | auto_archiver.enrichers.wacz_enricher:enrich:52 - generating WACZ without Docker for url='https://twitter.com/dave_mateer/status/1505876265504546817'
2023-08-22 10:50:26.916 | INFO     | auto_archiver.enrichers.wacz_enricher:enrich:96 - Running browsertrix-crawler: crawl --url https://twitter.com/dave_mateer/status/1505876265504546817 --scopeType page --generateWACZ --text --screenshot fullPage --collection 5e60e6e9 --id 5e60e6e9 --saveState never --behaviors autoscroll,autoplay,autofetch,siteSpecific --behaviorTimeout 120 --timeout 120 --profile /app/secrets/profile.tar.gz
{"logLevel":"info","timestamp":"2023-08-22T10:50:27.983Z","context":"general","message":"Browsertrix-Crawler 0.10.3 (with warcio.js 1.6.2 pywb 2.7.4)","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:50:27.983Z","context":"general","message":"Seeds","details":[{"url":"https://twitter.com/dave_mateer/status/1505876265504546817","include":[],"exclude":[],"scopeType":"page","sitemap":false,"allowHash":false,"maxExtraHops":0,"maxDepth":1000000}]}
{"logLevel":"info","timestamp":"2023-08-22T10:50:27.983Z","context":"general","message":"With Browser Profile","details":{"url":"/app/secrets/profile.tar.gz"}}
{"logLevel":"info","timestamp":"2023-08-22T10:50:30.205Z","context":"worker","message":"Creating 1 workers","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:50:30.205Z","context":"worker","message":"Worker starting","details":{"workerid":0}}
{"logLevel":"info","timestamp":"2023-08-22T10:50:30.263Z","context":"browser","message":"Disabling Service Workers for profile","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:50:30.269Z","context":"worker","message":"Starting page","details":{"workerid":0,"page":"https://twitter.com/dave_mateer/status/1505876265504546817"}}
{"logLevel":"info","timestamp":"2023-08-22T10:50:30.270Z","context":"crawlStatus","message":"Crawl statistics","details":{"crawled":null,"total":null,"pending":1,"failed":0,"limit":{"max":0,"hit":false},"pendingPages":["{\"seedId\":0,\"started\":\"2023-08-22T10:50:30.206Z\",\"url\":\"https://twitter.com/dave_mateer/status/1505876265504546817\",\"added\":\"2023-08-22T10:50:28.119Z\",\"depth\":0}"]}}
{"logLevel":"info","timestamp":"2023-08-22T10:50:32.373Z","context":"general","message":"Awaiting page load","details":{"page":"https://twitter.com/dave_mateer/status/1505876265504546817","workerid":0}}
{"logLevel":"error","timestamp":"2023-08-22T10:51:02.379Z","context":"general","message":"Page Load Error, skipping page","details":{"msg":"net::ERR_TIMED_OUT at https://twitter.com/dave_mateer/status/1505876265504546817","page":"https://twitter.com/dave_mateer/status/1505876265504546817","workerid":0}}
{"logLevel":"error","timestamp":"2023-08-22T10:51:02.379Z","context":"worker","message":"Unknown exception","details":{"type":"exception","message":"net::ERR_TIMED_OUT at https://twitter.com/dave_mateer/status/1505876265504546817","stack":"Error: net::ERR_TIMED_OUT at https://twitter.com/dave_mateer/status/1505876265504546817\n    at navigate (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/common/Frame.js:98:23)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Deferred.race (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/util/Deferred.js:79:20)\n    at async Frame.goto (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/common/Frame.js:64:21)\n    at async CDPPage.goto (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/common/Page.js:578:16)\n    at async Crawler.loadPage (file:///app/crawler.js:1062:20)\n    at async Crawler.default [as driver] (file:///app/defaultDriver.js:3:3)\n    at async Crawler.crawlPage (file:///app/crawler.js:451:5)\n    at async PageWorker.timedCrawlPage (file:///app/util/worker.js:151:7)\n    at async PageWorker.runLoop (file:///app/util/worker.js:192:9)","workerid":0}}
{"logLevel":"warn","timestamp":"2023-08-22T10:51:02.380Z","context":"pageStatus","message":"Page Load Failed","details":{"loadState":0,"page":"https://twitter.com/dave_mateer/status/1505876265504546817","workerid":0}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:02.386Z","context":"worker","message":"Worker exiting, all tasks complete","details":{"workerid":0}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:02.483Z","context":"crawlStatus","message":"Crawl statistics","details":{"crawled":1,"total":1,"pending":0,"failed":1,"limit":{"max":0,"hit":false},"pendingPages":[]}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:02.483Z","context":"general","message":"Waiting to ensure pending data is written to WARCs...","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:02.485Z","context":"general","message":"Generating WACZ","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:02.485Z","context":"general","message":"Num WARC Files: 0","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:02.485Z","context":"general","message":"Crawl status: done","details":{}}
2023-08-22 10:51:02.489 | WARNING  | auto_archiver.enrichers.wacz_enricher:enrich:108 - Unable to locate and upload WACZ  filename='collections/5e60e6e9/5e60e6e9.wacz'
2023-08-22 10:51:02.490 | DEBUG    | auto_archiver.enrichers.hash_enricher:enrich:31 - calculating media hashes for url='https://twitter.com/dave_mateer/status/1505876265504546817' (using SHA3-512)
2023-08-22 10:51:02.490 | DEBUG    | auto_archiver.enrichers.wacz_enricher:enrich:52 - generating WACZ without Docker for url='https://twitter.com/dave_mateer/status/1505876265504546817'
2023-08-22 10:51:02.490 | INFO     | auto_archiver.enrichers.wacz_enricher:enrich:96 - Running browsertrix-crawler: crawl --url https://twitter.com/dave_mateer/status/1505876265504546817 --scopeType page --generateWACZ --text --screenshot fullPage --collection c851aa3f --id c851aa3f --saveState never --behaviors autoscroll,autoplay,autofetch,siteSpecific --behaviorTimeout 120 --timeout 120 --profile /app/secrets/profile.tar.gz
{"logLevel":"info","timestamp":"2023-08-22T10:51:03.099Z","context":"general","message":"Browsertrix-Crawler 0.10.3 (with warcio.js 1.6.2 pywb 2.7.4)","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:03.100Z","context":"general","message":"Seeds","details":[{"url":"https://twitter.com/dave_mateer/status/1505876265504546817","include":[],"exclude":[],"scopeType":"page","sitemap":false,"allowHash":false,"maxExtraHops":0,"maxDepth":1000000}]}
{"logLevel":"info","timestamp":"2023-08-22T10:51:03.100Z","context":"general","message":"With Browser Profile","details":{"url":"/app/secrets/profile.tar.gz"}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:03.653Z","context":"worker","message":"Creating 1 workers","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:03.653Z","context":"worker","message":"Worker starting","details":{"workerid":0}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:03.703Z","context":"browser","message":"Disabling Service Workers for profile","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:03.710Z","context":"worker","message":"Starting page","details":{"workerid":0,"page":"https://twitter.com/dave_mateer/status/1505876265504546817"}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:03.710Z","context":"crawlStatus","message":"Crawl statistics","details":{"crawled":null,"total":null,"pending":1,"failed":0,"limit":{"max":0,"hit":false},"pendingPages":["{\"seedId\":0,\"started\":\"2023-08-22T10:51:03.654Z\",\"url\":\"https://twitter.com/dave_mateer/status/1505876265504546817\",\"added\":\"2023-08-22T10:51:03.157Z\",\"depth\":0}"]}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:05.392Z","context":"general","message":"Awaiting page load","details":{"page":"https://twitter.com/dave_mateer/status/1505876265504546817","workerid":0}}
{"logLevel":"error","timestamp":"2023-08-22T10:51:05.398Z","context":"general","message":"Page Load Error, skipping page","details":{"msg":"net::ERR_PROXY_CONNECTION_FAILED at https://twitter.com/dave_mateer/status/1505876265504546817","page":"https://twitter.com/dave_mateer/status/1505876265504546817","workerid":0}}
{"logLevel":"error","timestamp":"2023-08-22T10:51:05.399Z","context":"worker","message":"Unknown exception","details":{"type":"exception","message":"net::ERR_PROXY_CONNECTION_FAILED at https://twitter.com/dave_mateer/status/1505876265504546817","stack":"Error: net::ERR_PROXY_CONNECTION_FAILED at https://twitter.com/dave_mateer/status/1505876265504546817\n    at navigate (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/common/Frame.js:98:23)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async Deferred.race (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/util/Deferred.js:79:20)\n    at async Frame.goto (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/common/Frame.js:64:21)\n    at async CDPPage.goto (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/common/Page.js:578:16)\n    at async Crawler.loadPage (file:///app/crawler.js:1062:20)\n    at async Crawler.default [as driver] (file:///app/defaultDriver.js:3:3)\n    at async Crawler.crawlPage (file:///app/crawler.js:451:5)\n    at async PageWorker.timedCrawlPage (file:///app/util/worker.js:151:7)\n    at async PageWorker.runLoop (file:///app/util/worker.js:192:9)","workerid":0}}
{"logLevel":"warn","timestamp":"2023-08-22T10:51:05.399Z","context":"pageStatus","message":"Page Load Failed","details":{"loadState":0,"page":"https://twitter.com/dave_mateer/status/1505876265504546817","workerid":0}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:05.409Z","context":"worker","message":"Worker exiting, all tasks complete","details":{"workerid":0}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:05.540Z","context":"crawlStatus","message":"Crawl statistics","details":{"crawled":1,"total":1,"pending":0,"failed":1,"limit":{"max":0,"hit":false},"pendingPages":[]}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:05.540Z","context":"general","message":"Waiting to ensure pending data is written to WARCs...","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:05.542Z","context":"general","message":"Generating WACZ","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:05.543Z","context":"general","message":"Num WARC Files: 0","details":{}}
{"logLevel":"info","timestamp":"2023-08-22T10:51:05.543Z","context":"general","message":"Crawl status: done","details":{}}
2023-08-22 10:51:05.549 | WARNING  | auto_archiver.enrichers.wacz_enricher:enrich:108 - Unable to locate and upload WACZ  filename='collections/c851aa3f/c851aa3f.wacz'
2023-08-22 10:51:05.549 | DEBUG    | auto_archiver.formatters.html_formatter:format:37 - [SKIP] FORMAT there is no media or metadata to format: url='https://twitter.com/dave_mateer/status/1505876265504546817'
2023-08-22 10:51:05.549 | SUCCESS  | auto_archiver.databases.gsheet_db:done:46 - DONE https://twitter.com/dave_mateer/status/1505876265504546817
2023-08-22 10:51:06.365 | SUCCESS  | auto_archiver.feeders.gsheet_feeder:__iter__:79 - Finished worksheet Sheet1

and orchestation.yaml is:
steps:
  # only 1 feeder allowed
  feeder: gsheet_feeder # defaults to cli_feeder
  archivers: # order matters, uncomment to activate
    # - vk_archiver
    # - telethon_archiver
    # - telegram_archiver
    # - twitter_archiver
    #- twitter_api_archiver
    # - instagram_tbot_archiver
    # - instagram_archiver
    # - tiktok_archiver
    # - youtubedl_archiver
    # - wayback_archiver_enricher
    - wacz_archiver_enricher
  enrichers:
    - hash_enricher
    # - metadata_enricher
    # - screenshot_enricher
    # - thumbnail_enricher
    # - wayback_archiver_enricher
    - wacz_archiver_enricher
    # - pdq_hash_enricher # if you want to calculate hashes for thumbnails, include this after thumbnail_enricher
  formatter: html_formatter # defaults to mute_formatter
  storages:
    - local_storage
    # - s3_storage
    # - gdrive_storage
  databases:
    #- console_db
    # - csv_db
    - gsheet_db
    # - mongo_db

configurations:
  gsheet_feeder:
    sheet: "AA Demo Main"
    header: 1
    service_account: "secrets/service_account.json"
    # allow_worksheets: "only parse this worksheet"
    # block_worksheets: "blocked sheet 1,blocked sheet 2"
    use_sheet_names_in_stored_paths: false
    columns:
      url: link
      status: archive status
      folder: destination folder
      archive: archive location
      date: archive date
      thumbnail: thumbnail
      timestamp: upload timestamp
      title: upload title
      text: textual content
      screenshot: screenshot
      hash: hash
      pdq_hash: perceptual hashes
      wacz: wacz
      replaywebpage: replaywebpage
  instagram_tbot_archiver:
    api_id: "TELEGRAM_BOT_API_ID"
    api_hash: "TELEGRAM_BOT_API_HASH"
    # session_file: "secrets/anon"
  telethon_archiver:
    api_id: "TELEGRAM_BOT_API_ID"
    api_hash: "TELEGRAM_BOT_API_HASH"
    # session_file: "secrets/anon"
    join_channels: false
    channel_invites: # if you want to archive from private channels
      - invite: https://t.me/+123456789
        id:<PHONE_NUMBER>
      - invite: https://t.me/+123456788
        id:<PHONE_NUMBER>

  twitter_api_archiver:
    # either bearer_token only
    # bearer_token: "TWITTER_BEARER_TOKEN"

  instagram_archiver:
    username: "INSTAGRAM_USERNAME"
    password: "INSTAGRAM_PASSWORD"
    # session_file: "secrets/instaloader.session"

  vk_archiver:
    username: "or phone number"
    password: "vk pass"
    session_file: "secrets/vk_config.v2.json"

  screenshot_enricher:
    width: 1280
    height: 2300
  wayback_archiver_enricher:
    timeout: 10
    key: "wayback key"
    secret: "wayback secret"
  hash_enricher:
    algorithm: "SHA3-512" # can also be SHA-256
  wacz_archiver_enricher:
    profile: secrets/profile.tar.gz
  local_storage:
    save_to: "./local_archive"
    save_absolute: true
    filename_generator: static
    path_generator: flat
  s3_storage:
    bucket: your-bucket-name
    region: reg1
    key: S3_KEY
    secret: S3_SECRET
    endpoint_url: "https://{region}.digitaloceanspaces.com"
    cdn_url: "https://{bucket}.{region}.cdn.digitaloceanspaces.com/{key}"
    # if private:true S3 urls will not be readable online
    private: false
    # with 'random' you can generate a random UUID for the URL instead of a predictable path, useful to still have public but unlisted files, alternative is 'default' or not omitted from config
    key_path: random
  gdrive_storage:
    path_generator: url
    filename_generator: random
    root_folder_id: folder_id_from_url
    oauth_token: secrets/gd-token.json # needs to be generated with scripts/create_update_gdrive_oauth_token.py
    service_account: "secrets/service_account.json"
  csv_db:
    csv_file: "./local_archive/db.csv"

Thank you for opening the issue, I've recently stumbled upon the same problem.
I'll try to look into it soon since it effectively blocks wacz collections when using a docker deployment.

After investigating the bug was introduced in this commit: https://github.com/bellingcat/auto-archiver/commit/987bbcaad083310791dda98687c24b9748089cfe
What happened? we removed pywb dependency from Pipfile thinking it was not being used but it is required so that browsertrix-crawler can work, it is installed in their own Dockerfile but since we use pipenv instead of the default pip installation it was not being accessed, and hence needs to be added explicitly to the Pipfile.

Thank you @msramalho  it is working for me now!


Graceful fall-back handling of imperfect codable concepts

Describe the bug
Incoming data can be messy.  Especially race encoding has missing values, assumed encoding schemes, and presumed encoded values.  The system should fall-back to a best case encoding and not destroy the original input.
To Reproduce
See samples below of incomplete CWE's and how they should be handled.  Currently using them will produce inconsistent or missing results.
Expected behavior
Details of what should happen:
|2106-3^WHITE^CDCREC| should produce.
{
        "url": "http://ibm.com/fhir/cdm/StructureDefinition/local-race-cd",
        "valueCodeableConcept": {
          "coding": [ {
            "system": "http://terminology.hl7.org/CodeSystem/v3-Race",
            "code": "2106-3",
            "display": "White"
          } ],
          "text": "WHITE"
        }
      }

Because:  CDCREC is a known system, 2106-3 is a valid code in that system, display is looked up in the system as White; the custom text WHITE is kept from the original CWE.2 Value as Text.
|White^WHITE^L| should produce.
{
        "url": "http://ibm.com/fhir/cdm/StructureDefinition/local-race-cd",
        "valueCodeableConcept": {
          "coding": [ {
            "system": "urn:id:L",
            "code": "White",
            "display": "WHITE"
          } ],
          "text": "WHITE"
        }
      }

Because:  L is and unknown system, White is a supposedly valid code in that system, display: WHITE is kept from CWE.2 as display Value;  text uses display value
|2106-3^^CDCREC| should produce.
{
        "url": "http://ibm.com/fhir/cdm/StructureDefinition/local-race-cd",
        "valueCodeableConcept": {
          "coding": [ {
            "system": "http://terminology.hl7.org/CodeSystem/v3-Race",
            "code": "2106-3",
            "display": "White"
          } ],
          "text": "White"
        }
      }

Because:  CRCREC is valid system, 2106-3 is a valid code, display: White is looked up in system for display Value;  because CWE.2 is empty, text uses calculated display.
|5234-5^hobbit^CDCREC| invalid code with valid system should produce:
{
        "url": "http://ibm.com/fhir/cdm/StructureDefinition/local-race-cd",
        "valueCodeableConcept": {
          "coding": [ {
         "system":  "system": "http://terminology.hl7.org/CodeSystem/v3-Race",
          "display": "Invalid input: code: 5234-5 for system:  "http://terminology.hl7.org/CodeSystem/v3-Race" original display: hobbit";
          } ]
        }
      }

Because:  CRCREC is valid system, 2186-5 is an invalid code, display:  no display value can be found;  because CWE.2 is empty, and no display to use as backup,  text is empty and therefore not shown.
Additional context
Display value is calculated:
1.  If code is in system-known-to-FHIR, use that display value.getValue
2.  Otherwise, if code is not in system-known-to-FHIR, it's a bad code, put an error message in the display string
3.  Otherwise if the system is known-to-us (in CodingSystemMapping ), then we can't tell if the code is valid or not, use the original display text

This was added in https://github.com/LinuxForHealth/hl7v2-fhir-converter/pull/169

In Release 1.0.10


Dynamically move the marker on map

I am making an indoor positioning application, for which I need to update the location of the android device every second. The position of the marker should change every second.
final ImageView logo = new ImageView(
logo.setImageResource(R.drawable.logo);
tileView.addMarker(logo, x, y, -0.5f, -1.0f);
Need to update x and y every second.

TileView.moveMarker http://moagrius.github.io/TileView/com/qozix/tileview/TileView.html#moveMarker(View, double, double)

resolved?

Ya, I got it. Can you please tell me how to move the marker smoothly from one position to another over the map?

There's no built in way to do that.  You'd have to use your own animation routine.


Missing requirements.txt

Please could you add a requirements.txt.

For what dependency exactly? Both the ajpy library and the helper tomcat.py rely only on built-in modules.

For what dependency exactly? Both the ajpy library and the helper tomcat.py rely only on built-in modules.


Remove unused focusedOpacity prop and function

The focusedOpacity prop is only used inside _opacityFocused which is not used anywhere. This pr removes this unused code.
The code was added in https://github.com/facebook/react-native/pull/10427 but it does not appear to be used in the final version of the pr.

@dlowder-salesforce Do we still need this?

@shergin No I don't think we need this any more.

@levsero Please add trailing commas back.


Cannot create a new function app

Does this occur consistently? 100% of the time (tried once)
Repro steps:
Deploy to a new function app (advanced) using Visual Studio Code.
Looks like the whole cloud explorer is broken, refresh does nothing and I do not see how to sign out - all subs look like this.

Restarting VS Code fixes this.
Action: azureFunctions.deploy
Error type: Error
Error Message: Entry not found in cache.
Version: 0.24.0
OS: win32
OS Release: 10.0.19041
Product: Visual Studio Code
Product Version: 1.49.1
Language: en

Call Stack
s.createError extension.js:16:58035
b.<anonymous> extension.js:393:42004
extension.js:393:36599extension.js:393:36599
extension.js:243:74686extension.js:243:74686
extension.js:243:72667extension.js:243:72667
extension.js:243:72287extension.js:243:72287
i.find extension.js:243:106819
extension.js:243:53526extension.js:243:53526
processTicksAndRejections task_queues.js:94:5

Just a note that I deployed to an existing app successfully, broke it somehow, removed the app in Portal and tried to deploy from VS Code to a new app.

As a workaround you could try reloading your window or signing out and signing back in to your Azure Account using the command palette.
Unfortunately there's not much we can do in this extension, but this is being tracked upstream here: microsoft/vscode-azure-account#53


Mouseover on nameplates?

Hey man, I used to macro everything in TBC on mouseover specially for using it on nameplates. It seems when using Shaguplates or nothing as nameplates it doesn't work with @mouseover. Could you take a look at it? I can't figure it out on my own >_< haha

With the release of Elysium PvP, this bug suddenly got fixed itself. I have no idea why.

With the latest Shaguplates this issue is solved. However ''Auto Shot'' and Mouseover doesn't work together. (/cast [@mouseover] Auto Shot). Any idea to work-around this one?

I'm unable to test this on Nostlysium since I'm not willing to sit through a 5-hour queue, but at least on the server I play on it'll interrupt your Auto Shot when you deselect your current target. Therefore it will either just switch back to your current target or, if you don't have one, interrupt Auto Shot altogether. So this might be a server side limitation.
Could you check if this is the case? Just select a target, cast Auto Shot, and then deselect your target and see if your Auto Shot gets interrupted. If so, then there's nothing I can do about this.

[@mouseOver] on pfUI unitframes doesn't work either. Can you please check it on Kronos3, dennis?


New version: Lilith v0.3.0

Registering package: Lilith
Repository: https://github.com/dfdx/Lilith.jl
Created by: @dfdx
Version: v0.3.0
Commit: 03b974f24944bbdde6fe3ce77ef164bd31665737
Git reference: master
Release notes:

new tracer
migration from CuArrays to CUDA.jl
new minimal Julia version is now v1.4

Replaced with #19623, feel free to close.


Tests for APIs are missing

Hello,
The tests for various api endpoints are missing currently.
Since there are a lot of endpoints, it'd be better if they are broken into separate issues - one for each

user
story
project
org
forum
auth
admin

I am willing to work on this issue myself.

@singhayushh  Please connect with @gauravdas014 to complete this task. He can guide to if you have any doubts. 👍

@singhayushh Let the Issue be a single issue, you can create multiple PRs or a single pr with subtasks, whatever you prefer. On completion of all the sub-issues under this issue, I will close this issue. React out for any other clarification

@singhayushh Are you working on it?

@gauravdas014
I've been having exams in college and I couldn't work on it..

@singhayushh May I unassign you or you would like to work in a couple of days?

Can we break it into multiple assignees?
I'll write tests for say, user and admin routes
Someone else continues the tests for other routes?

@singhayushh That's also fine. You can directly give PR, just mention that PR is a part of this issue. Or you can create a separate issues also.

Hey, sorry to inform but I'm unable to give enough time to the issue rn. I'd opt to be unassigned for now and let others work on this.
Sorry for the inconvenience, if caused.

Hey, sorry to inform but I'm unable to give enough time to the issue rn. I'd opt to be unassigned for now and let others work on this.
Sorry for the inconvenience, if caused.

That's fine.

please assign me this.

@subhangi2731 Please explain how you will proceed? What tools of testing you are planning to use?

I will use postman to test API endpoints or the citrus as I am familiar with them

@subhangi2731 Have you used supertest before? It is used for API testing

yes I have It is used to test HTTP endpoints and for that we need supertest module and install its dependecies

@subhangi2731 go ahead with the issue. First, write tests for the users and give PR. After its reviewed please continue with the rest

sure @gauravdas014

@gauravdas014 can u please reply so that i can send a pr

@gauravdas014 since there are a lot of endpoints to cover all should come under one issue then what endpoints should I test??can u be more specific.

You have to write tests for all the endpoints.

okay done

@gauravdas014 https://github.com/girlscript/feminist-bible-phase-2/blob/master/backend/.env.example in this how to bring all these four??
JWT_EXPIRES_IN=JWT expiry time
JWT_SECRET=JWT Secret
JWT_COOKIE_EXPIRES_IN=cookie expiry time
JWT_COOKIE_DOMAIN=.example.com

@gauravdas014 please help so that I can make a pr

@subhangi2731
JWT_EXPIRES_IN=86400000
JWT_SECRET=123xyz.xyz123
JWT_COOKIE_EXPIRES_IN=8640000
JWT_COOKIE_DOMAIN=http://localhost

@gauravdas014 @singhayushh while doing npm run dev http://localhost:8080/api/user
this is coming Unauthorized why please tell me

@subhangi2731
The route requires the request to be coming from an authenticated user
So you need to first hit the login / sign in route and then use these routes.

@singhayushh @gauravdas014 how to authorize and where is the signup route

can u please tell me in detail @singhayushh

@singhayushh @gauravdas014 how to authorize and where is the signup route

Please check the api-docs for the required routes. The sign up route is in authRoutes.js

ok @gauravdas014 what ill be the endpoint for signing up in this route

@subhangi2731 please go through the code. Just knowing the endpoint isn't enough.

@gauravdas014 please check signup is not happening by any route

@subhangi2731 it's a post request and not a get request. Please go through the code thoroughly before writing tests.


PredNet config file missing

Hi, it seems the config file of PredNet is missing under openstl/configs/kitticaltech/. Could you please add it? Thanks!

We found the implementation of this model may have some issues, so currently, we removed this model.

I will close this issue if there are no more questions. Please feel free to open a new issue when you have new questions.


V3: Using nodefs by default for now

The native filesystem is missing functionality required to resolve atlaspackrc#extends. I have set it to use the Nodejs filesystem implementation to move to the next step and discuss this later.
We can't ship using the Nodejs fs from Rust due to performance issues so this is an essential "todo".
I think the best way to proceed is to implement the Flow FileSystem interface in Rust and make it available to Nodejs via napi bindings then use that in both Nodejs and Rust. I think that a symmetrical interface will make it easier going forward.
I'll leave that up for discussion next week.

No longer required


Update istio.io/client-go version to 1.4.1

Update istio.io/client-go and drop our patch for it since it has been upstreamed.
No changes in behavior intended (besides dropping hack/istio-client-missing-group-name.patch).
/assign @mattmoor
cc @tcnghia

/ok-to-test


Bug: UI doesn't reflect that the snapshot exists.

Describe the bug
UI doesn't show title of snapshot when aborted 'add' command is followed by an update.
The image below shows some snapshot titles as "Loading..." with a throbber, and some without. The ones without you can see have a "singlepage" archive saved. So they shouldn't say "Loading" nor "not yet archived" on hover.

Steps to reproduce

archive add < bookmarks.json
ctrl-c
archive update --extract singlefile

And lots of other playing/experimenting, so who knows how I got the db into this state.

I don't know what's up with the errors above - singlefile works as does readability, and they don't error out when saving a URL.

Merging with this as it's the same bug, will get it fixed in the next few rc releases, thanks for reporting: https://github.com/ArchiveBox/ArchiveBox/issues/1178


⚠ Bofrost.de has degraded performance

In da451bb, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1120 ms

Resolved: Bofrost.de performance has improved in 87f5de3 after 2 minutes.


Revert "Updates Swagger YAML for new added APIs."

Reverts vmware/harbor#1143

Coverage increased (+0.007%) to 51.126% when pulling 96cf5ef866a5c1bea327d2ce7f6b85ff9b727a81 on revert-1143-master-update-swagger into 44205610ec4b84fa3dd1cc94b81b7bc319a564b0 on master.


Make cargo forward pre-existing CARGO if set

Currently, Cargo will always set $CARGO to point to what it detects its own path to be (using std::env::current_exe). Unfortunately, this runs into trouble when Cargo is used as a library, or when current_exe is not actually the binary itself (e.g., when invoked through Valgrind or ld.so), since $CARGO will not point at something that can be used as cargo. This, in turn, means that users can't currently rely on $CARGO to do the right thing, and will sometimes have to invoke cargo directly from $PATH instead, which may not reflect the cargo that's currently in use.
This patch makes Cargo re-use the existing value of $CARGO if it's already set in the environment. For Cargo subcommands, this will mean that the initial invocation of cargo in cargo foo will set $CARGO, and then Cargo-as-a-library inside of cargo-foo will inherit that (correct) value instead of overwriting it with the incorrect value cargo-foo. For other execution environments that do not have cargo in their call stack, it gives them the opportunity to set a working value for $CARGO.
One note about the implementation of this is that the test suite now needs to override $CARGO explicitly so that the user's $CARGO does not interfere with the contents of the tests. It could remove $CARGO instead, but overriding it seemed less error-prone.
Fixes #10119.
 Pushed a test change that should hopefully fix the issue.

Huzzah, looks like that did it!

:pushpin: Commit 724a1977cea200f76d76b9868b5c4179252582e3 has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit 724a1977cea200f76d76b9868b5c4179252582e3 with merge 352175f8107e7a304cb8407a9c298fe2c8cd3d29...


822 upgrade dependencies

Replaced Newtonsoft.Json with System.Text.Json
Setup static HttpClient in DwollaClient

Build warnings on the client can be addressed here also. I don't think the hiding was intentional so these Links properties can just be removed
Some tests are failing

I pushed commits to fix the HttpClient to be what I think MS guidance says to do. Kind of a bummer that their advice for framework is to use their DI system that was introduced in core... which means most framework projects aren't using it.
DateOnlyConverter needs test coverage and then run the Example app (in docker if needed, they have a dockerfile provided) just to check that everything still works as expected. I think then if that all looks good then you can close this PR and open a PR in the Dwolla repo.


How can i fine tuning the model?

i am using the model model-20170511-185253.ckpt-80000.data-00000-of-00001 to fine tuning on my own dataset to improve the accuracy. but i found the code doesn't using the slim of tensorflow so it is not that convenient and i should write the code to exclude the Logits scope by hand.
i found the issue #70 (https://github.com/davidsandberg/facenet/issues/70)
and follow what the asker did:
if args.checkpoint_exclude_scopes:
exclusions = [scope.strip()
for scope in args.checkpoint_exclude_scopes.split(',')]
else:
exclusions = ["InceptionResnetV1/Logits"]
restore_vars = []
update_gradient_vars = []
if args.pretrained_model:
for var in tf.all_variables():
excluded = False
for exclusion in exclusions:
# skip this variable
if var.op.name.startswith(exclusion):
excluded = True
if not excluded:
print(var.op.name)
if not 'Embeddings/' in var.op.name:
restore_vars.append(var)
else:
update_gradient_vars.append(var)
    else:
        restore_vars = tf.all_variables()
        update_gradient_vars = tf.all_variables()

and modified the code to restore the ckpt file:
saver = tf.train.Saver(restore_vars, max_to_keep=3)
# saver = tf.train.Saver(tf.trainable_variables(), max_to_keep=3)
but it does not work. i only found many error log about Not found: Key centers not found in checkpoint something like:
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
W tensorflow/core/framework/op_kernel.cc:993] Not found: Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
Traceback (most recent call last):
File "src/train_softmax.py", line 483, in
main(parse_arguments(sys.argv[1:]))
File "src/train_softmax.py", line 243, in main
saver.restore(sess, pretrained_model)
File "/usr/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1428, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/usr/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 767, in run
run_metadata_ptr)
File "/usr/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 965, in _run
feed_dict_string, options, run_metadata)
File "/usr/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1015, in _do_run
target_list, options, run_metadata)
File "/usr/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1035, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
[[Node: save/Assign_492/_1148 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_4449_save/Assign_492", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]]
Caused by op u'save/RestoreV2_493', defined at:
File "src/train_softmax.py", line 483, in
main(parse_arguments(sys.argv[1:]))
File "src/train_softmax.py", line 224, in main
saver = tf.train.Saver(restore_vars, max_to_keep=3)
File "/usr/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1040, in init
self.build()
File "/usr/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1070, in build
restore_sequentially=self._restore_sequentially)
File "/usr/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 675, in build
restore_sequentially, reshape)
File "/usr/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 402, in _AddRestoreOps
tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
File "/usr/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 242, in restore_op
[spec.tensor.dtype])[0])
File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/gen_io_ops.py", line 668, in restore_v2
dtypes=dtypes, name=name)
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
op_def=op_def)
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1226, in init
self._traceback = _extract_stack()
NotFoundError (see above for traceback): Key centers not found in checkpoint
[[Node: save/RestoreV2_493 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_493/tensor_names, save/RestoreV2_493/shape_and_slices)]]
[[Node: save/Assign_492/_1148 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_4449_save/Assign_492", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]]
what should i do? can anyone help?

I have the same problem!Have you solved  it? @BKZero

I have the same problem!Have you solved it?

Check your parameters which should be compatible with the given pre-trained model such as:
--model_def, --pretrained_model ./model/model-xxx.ckpt-xxxx, ...
Also, changing  --gpu_memory_fraction can help to solve GPU related errors.


Improve CAPTCHA support by implementing PrivacyPass

https://privacypass.github.io/
https://www.petsymposium.org/2018/files/papers/issue3/popets-2018-0026.pdf

For example, Privacy Pass is supported by Cloudflare to enable users to redeem passes instead of having to solve CAPTCHAs to visit Cloudflare-protected websites.

https://support.cloudflare.com/hc/en-us/articles/115001992652-Can-I-use-Privacy-Pass-with-Cloudflare-

The extension generates 30 tokens for each CAPTCHA solution and thus can be used to reduce CAPTCHA pages for each user by a similar factor.

Outdated but from one of Cloudflare's github repos: https://github.com/cloudflare/challenge-bypass-specification/blob/master/captcha-bypass-internet-draft.txt
Browser add-on source: https://github.com/privacypass/challenge-bypass-extension
@VeNoMouS

@pro-src
How to make use this PrivacyPass with cloudscraper?
I am using version  v4.0.1 and meanwhile all are working good.
But it seems like the wack-a-mole game with cloudflare team going faster chase each other.
I am looking for a more permanent solution.

How to make use this PrivacyPass with cloudscraper?

Read the links. It's something that we would have to implement. It's not something to be used with Cloudscraper.

I am using version v4.0.1 and meanwhile all are working good.

Glad to hear it. :+1:

But it seems like the wack-a-mole game with cloudflare team going faster chase each other.

I have no proof of such, although you might be correct.

I am looking for a more permanent solution.

I think CF has been a big proponent of PrivacyPass from the beginning. I don't think they would break that so maybe forcing a CAPTCHA, using a service such as anti-captcha.com to solve the initial challenge, and with PrivacyPass implemented, might be the best permanent solution.
The reCAPTCHA challenge hasn't changed at all and the anti-bot code on those pages isn't even being validated server side. Whereas the JS challenge has changed 3-4 times within the past couple of months. However, there has been changes that affect both of those challenge types and some pretty tricky ones that aren't causing problems for everybody yet.
I things will eventually calm down. Just stick it out with us while we develop new ways to bypass CF.

The browser extension code is really nice and doesn't need to be reproduced for this project. I'll release a modified version that pulls in the reusable parts of the browser-extension and bundle it with the necessary adaptions. Cloudscraper can think list it as a peer dependency and perform an optional require to make use of it. So it is something to be used with cloudscraper since we don't want to reproduce and maintain a complete fork.
@codemanki How does that sound to you?

@pro-src sounds very cool!

I'm working on it now and making great progress! A little background on the PrivacyPass core team, they are all cryptographers, one is a Golang core developer for Google, and at least one is a developer for Cloudflare. So it is a promising project indeed.
Since the tokens are blinded and can be redeemed for domains other than the one used to obtain them, it's possible to make an initial request to https://captcha.website to solve and get tokens to redeem for any domain! https://captcha.website is dedicated to the PrivacyPass project.

I'll have this working soon without a single change to cloudscraper. Just taking my time. :smiley:
const cloudscraper = require('cloudscraper').defaults({
  requester: require('privacy-pass')
});

I haven't finished this yet but when I get around to it, there is little left that needs to be done. Since no changes need to be made to cloudscraper, this is not a cloudscraper issue. I will update this issue upon publishing the node-privacy-pass package.

I don't know how many people were looking forward to the node-privacy-pass but if the thumbs are any indication, I'm not disappointing too many people. I've changed my mind after putting it off a lot lately. I don't currently plan to finish developing it.


ERROR = sendto failed: EPIPE (Broken pipe)

I`m using go tusd with java tusd(APP), these errors occurs.

when i upload file, sometimes 413 error occurs.
When we upload small file it works good, but for large file EPIPE ( Broken pipe ) error occurs

Please help me

Is this the same report as in https://github.com/tus/tus-java-client/issues/21? From the package name it looks like so. Please avoid reporting duplicate issues


Napari seg improve content

Copy-editing the Bravo Cohort (https://chanzuckerberg.github.io/napari-segmentation-workshop)
Fix PartSeg instructions

Fill holes
Pre-processing
Post-processing

@dgmccart Please review this draft PR.

@dgmccart Please review this PR.


LabsPhi3_sk_rag01 fails

System.MissingMethodException   HResult=0x80131513   Message=Method not found: 'System.ValueTuple3<System.ReadOnlyMemory1<Int64>,System.ReadOnlyMemory1,System.ReadOnlyMemory1<Int64>> FastBertTokenizer.BertTokenizer.Encode(System.String, Int32, System.Nullable1)'.
Source=SmartComponents.LocalEmbeddings
StackTrace:
at SmartComponents.LocalEmbeddings.LocalEmbedder.Embed[TEmbedding](String inputText, Nullable1 outputBuffer, Int32 maximumTokens)    at SmartComponents.LocalEmbeddings.LocalEmbedder.Embed(String inputText, Int32 maximumTokens)    at SmartComponents.LocalEmbeddings.SemanticKernel.LocalTextEmbeddingGenerationService.GenerateEmbeddingsAsync(IList1 data, Kernel kernel, CancellationToken cancellationToken)
at Microsoft.SemanticKernel.Embeddings.EmbeddingGenerationExtensions.d__02.MoveNext()    at Microsoft.SemanticKernel.Memory.SemanticTextMemory.<SaveInformationAsync>d__3.MoveNext()    at phi3vision_aiconsole.Program.<Phi3MemoryRAG>d__2.MoveNext() in O:\Windows For Programming\Projects\Visual Studio\Console\Ai\phi3vision_aiconsole\Program.cs:line 86    at phi3vision_aiconsole.Program.<Main>d__0.MoveNext() in O:\Windows For Programming\Projects\Visual Studio\Console\Ai\phi3vision_aiconsole\Program.cs:line 24
occurs at the following line
await memory.SaveInformationAsync(MemoryCollectionName, id: "info1", text: "Gisela's favourite super hero is Batman");

Hi @Barshan-Mandal  are you taking the semantic Kernel samples from https://github.com/elbruno/phi3-labs by @elbruno?
If so please raise issues on that repo?

Hi
I'm working on update this sample to make it work again. In the meantime, you can refer to this post from Arafat, who performs a very similar scenario doing RAG in local mode.
https://arafattehsin.com/ai-copilot-offline-phi3-semantic-kernel/
Best

semantic text cant handle long books .try using kernel memory


Correct results?

I am running this against a very large (~1.8GB) .com repo and only see two lines in the CSV.
type	message
repository-dependabot-alerts	This repository has one or more Dependabot alerts which will not be migrated
watchers	One or more users are watching this repo. Watchers will not be transferred as part of a migration.

On the command line I see:
2023-11-10T19:08:29.507Z info: GitHub REST rate limit: 1/5000 used - resets at 2023-11-10T19:55:02.000Z
2023-11-10T19:08:29.918Z info: GitHub GraphQL rate limit: 3/5000 used - resets at 2023-11-10T19:55:03Z
2023-11-10T19:08:29.919Z info: Running in GitHub.com mode...
2023-11-10T19:08:29.919Z info: Auditing ORGNAME/LARGEREPO...
2023-11-10T19:08:34.610Z warn [ORGNAME/LARGEREPO]: Auditor `repository-codespaces-secrets` failed for ORGNAME/LARGEREPO with error: Not Found
2023-11-10T19:08:37.300Z info: Successfully wrote audit CSV to ORGNAME_LARGEREPO_1699643308905.csv

For a large Org I thought there would be more. Is this expected?
Thanks
Todd

You were just checking one repo, right? 😊
If so, I'm not surprised particularly by the low number of warnings.
We don't check everything - just the things listed in the readme which we know can't be migrated.
At the moment, that doesn't include any checks for large files (which would require us to clone the repo and run git-sizer) or large repos (which I will add today, as that's a good feature idea!).

Large repo size auditor added in v0.3.0 ✅

@toddoconnor Is there anything more I can do to help on this?

Thanks @timrogers . I'm closing this for now. Look fwd to the new feature for large repos.