instascrape.exceptions.exceptions.InstagramLoginRedirectError: Instagram is redirecting you to the login page instead of the page you are trying to scrape. This could be occuring because you made too many requests too quickly or are not logged into Instagram on your machine. Try passing a valid session ID to the scrape method as a cookie to bypass the login requirement

Thanks a lot for the great API. Do you know how I can fix this error?
(insta) [jalal@goku insta-scrape]$ python test.py
/scratch3/venv/insta/lib/python3.8/site-packages/instascrape/core/_static_scraper.py:134: MissingCookiesWarning: Request header does not contain cookies! It's recommended you pass at least a valid sessionid otherwise Instagram will likely redirect you to their login page.
  warnings.warn(
Traceback (most recent call last):
  File "test.py", line 9, in <module>
    google_post.scrape()
  File "/scratch3/venv/insta/lib/python3.8/site-packages/instascrape/scrapers/post.py", line 73, in scrape
    return_instance = super().scrape(
  File "/scratch3/venv/insta/lib/python3.8/site-packages/instascrape/core/_static_scraper.py", line 144, in scrape
    return_data = self._get_json_from_source(self.source, headers=headers, session=session)
  File "/scratch3/venv/insta/lib/python3.8/site-packages/instascrape/core/_static_scraper.py", line 265, in _get_json_from_source
    self._validate_scrape(json_dict)
  File "/scratch3/venv/insta/lib/python3.8/site-packages/instascrape/core/_static_scraper.py", line 301, in _validate_scrape
    raise InstagramLoginRedirectError
instascrape.exceptions.exceptions.InstagramLoginRedirectError: Instagram is redirecting you to the login page instead of the page you are trying to scrape. This could be occuring because you made too many requests too quickly or are not logged into Instagram on your machine. Try passing a valid session ID to the scrape method as a cookie to bypass the login requirement
(insta) [jalal@goku insta-scrape]$ cat test.py
from instascrape import *

google = Profile('https://www.instagram.com/tresorbutik/')
google_post = Post('https://www.instagram.com/p/CSCNBuljxLv/')
google_hashtag = Hashtag('https://www.instagram.com/explore/tags/fashion/')

# Scrape their respective data
google.scrape()
google_post.scrape()
google_hashtag.scrape()

print(google.followers)
print(google_post['hashtags'])
print(google_hashtag.amount_of_posts)

You should get your login sessionid cookie from instagram (Using a cookie editor of some sort) and then add it to the headers when you scrape your data:
headers = {
  'cookie': 'sessionid=YOUR_SESSIONID_HERE'
}
google = Profile('https://www.instagram.com/tresorbutik/')
google.scrape(headers=headers)

You should get your login sessionid cookie from instagram (Using a cookie editor of some sort) and then add it to the headers when you scrape your data:
headers = {
  'cookie': 'sessionid=YOUR_SESSIONID_HERE'
}
google = Profile('https://www.instagram.com/tresorbutik/')
google.scrape(headers=headers)

This is still not working for me. but still trying


Do SSL better

#23 added a bunch of node "--openssl-legacy-provider" flags everywhere we use webpack because it bumped Node to v17 which included security fixes to the default ssl provider, which was a break.
Instead of using the legacy ssl provider we can fix this by updating the webpack output hashFunction: https://webpack.js.org/configuration/output/#outputhashfunction

@tpetracca I granted you write access so pushing to a branch should trigger CI

#26


ULT_SetMediaItemNote not working on newly ReaScript created item

im using Lua, SWS 2.6.1 with Reaper 5 pre 13
If i create a new item with reaper.AddMediaItemToTrack() then attempt to run ULT_SetMediaItemNote() on it, nothing seems to happen, no notes are added.

just did a simple test
item = reaper.GetSelectedMediaItem(0, 0)

reaper.ULT_SetMediaItemNote(item, "test")

ran it on a single selected empty item, did nothing. is this function broken in Lua?

Fixed for the next pre-release, thanks for the report.
Note about refresh: your script has to take care of it with UpdateArrange()


Si570 programmable oscillator

Basil driver, firmware and tests written by Matthieu Batista (CPPM).

Coverage decreased (-1.0%) to 38.033% when pulling 710c018e84219f6ca9dfa4dfa655da84bc8ee7a1 on MarcoVogt:development into d28515b226b8a6678a6288f9346b6da131626f87 on SiLab-Bonn:development.

Coverage decreased (-1.0%) to 38.033% when pulling 71486dec7b3c3fca8380ec7144a77b1d764bd71b on MarcoVogt:development into d28515b226b8a6678a6288f9346b6da131626f87 on SiLab-Bonn:development.

Needs some rework, I guess


updated go mod

Description
This PR fixes #
Notes for Reviewers
Signed commits

[ ] Yes, I signed my commits.

Yay, @acald-creator 🙌
@Abhishek-kumar09 @Azanul @abdullah1308

@acald-creator The commit message and PR title is "updated go mod" but only changed file is go.sum.
Is this a mistake or on purpose? I don't know of any case where go.sum changes without change in go.mod.

Thanks @acald-creator


Kubernetes environment stuck at "Starting services (1 of 10)"

Rancher Version:
1.1.4
Docker Version:
1.12.1
OS and where are the hosts located? (cloud, bare metal, etc):
Ubuntu 16.04 on AWS
Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)
Single node HA (ext. db [rds])
Environment Type: (Cattle/Kubernetes/Swarm/Mesos)
Kubernetes (3 R3.8XL AWS instances)
Steps to Reproduce:
Setup HA
Run rancher's kubernetes bootstrap command on the nodes
Results:
stuck at 1/9
Expected:
Functional cluster

Did you add hosts into your kubernetes environment?
Also, Rancher doesn't officially support Docker 1.12 on 1.1.4.

@deniseschannon the process won't start without hosts being added, so that not the case.
which docker ver. do you recommend ?
thanks :)

We recommend Docker 1.10.3 with Rancher 1.1.4

I've tried that out yesterday (1.10.3)
but unfortunately that didn't helped.
This time the HA server (single node) didn't came up at all.
I've run the initial rancher HA generator,
downloaded the script and removed the generator container.
Ran the script, got a UI after 10-20 minutes but in HA I couldn't see my host listed.
I've decided to give up on rancher and run kubernetes directly.
Thanks for the support so far.

Ok, I'll close this out then. A couple of options are to run a single instance of Rancher to get k8s up. There are lots of complexity in HA that could be preventing you from getting set up, but we do have users who have HA and k8s running together.
We are updating HA in our 1.2 release so please come try us again. :)

I am also facing similar issue it hangs up in service 2/10, my docker version is 1.12.6

@pakuni youre commenting in closed tickets. Can you open a new ticket with rancher deployment details?


Support for non-global cloud environments

Currently, the CMDlets call hard-coded graph.microsoft.com. This should be parameterizable, so other cloud environments can use the module, too. E.g. for GCC High, you should pass -GraphBaseUri 'https://graph.microsoft.us'.

There is still an issue where .azurewebsites.net is hard-coded, while GCC High uses .azurewebsites.us


PublishContainer doesn't traverse the project tree (Microsoft.Build.Traversal)

Hey!
I use a .proj file in the root (generated by dotnet-affected tooling) to build and publish the affected projects in my tree.
I'm now exploring the PublishContainer target for my CI pipeline. And it seems to be checking isPublishable and EnableSdkContainerSupport props in the said .proj file (in the root) which are false by default and then skips publishing the containers altogether without traversing the tree.
Publishing the projects inside the tree works just fine. Is it an intended behavior?
Thanks for looking into this!
--
MItya

You'd need to ask the traversal SDK folks why the target isn't forwarded along to the child projects.

Great! Interested in hearing what they think - in general if an action works for a solution we should be able to support it for a traversal project as well.

@jeffkl I'm guessing this is because the Traversal targets have explicit support for a key set of MSBuild targets that they forward along to ProjectReferences, and the PublishContainer target isn't one of those. That sound about right?

@baronfel that is correct, only a handful of targets are implemented in Traversal: https://github.com/microsoft/MSBuildSdks/blob/main/src/Traversal/Sdk/Traversal.targets#L135-L284
When building a solution, MSBuild generates a metaproj on-the-fly and any target not already declared it dynamically generates one for.  I wonder if MSBuild should have a feature to do this for Traversal projects as well?
But it wouldn't be that hard to add the PublishContainer target to Traversal either.  Someone just needs to do it.


Fixing TypeError on parse_level method from Parser

If you write something like this:
BoxLayout:
    orientation: 'vertical'
        Button:

You will get this error:
  File "/.venv/lib/python3.10/site-packages/kivy/lang/parser.py", line 756, in parse_level
    if current_property[:3] == "on_":
TypeError: 'NoneType' object is not subscriptable

Because current_property is None, and on line 756 we try to subscript it. But this error is not informative and does not allow the developer to know where the error comes from. In this case we should show in which line there is a invalid declaration,  by raising a ParserException.
After the changes, this is the result:
kivy.lang.parser.ParserException: Parser: File "main.kv", line 3:
...
    1: BoxLayout:
    2:     orientation: 'vertical'
>>  3:        Button:
    4:
    5:
...
Invalid declaration

Maintainer merge checklist

[x] Title is descriptive/clear for inclusion in release notes.
[ ] Applied a Component: xxx label.
[ ] Applied the api-deprecation or api-break label.
[ ] Applied the release-highlight label to be highlighted in release notes.
[ ] Added to the milestone version it was merged into.
[ ] Unittests are included in PR.
[ ] Properly documented, including versionadded, versionchanged as needed.

Ping @FilipeMarch

I'm sorry for this long time without answering you. I will add a test.

I'm catching an extra indentation declaration; added two tests, one for each case. @misl6

Hi @FilipeMarch !
The PEP8 check is failing, can you please take care of it?
Feel free to ping me when you're ready for another review!

@misl6 Done!


Update 404 -page

Update the page according to the latest figma design

Noticed that there is some "state update on unmounted component" warning on the roadmap router component, when coming to the 404 page from http://localhost:3000/roadmap/1/not-found. This seems to be from the useEffect.

Good catch, it is definitely from roadmap router's useEffect. This (https://github.com/facebook/react/issues/14369#issuecomment-468267798) seems to be the most common fix, yet in our case it has unexpected side-effects. Working on it...
I added the requested changes though

Hmm, looks like the "state update on unmounted component" warning might not be as significant as it sounds in this case, as it will be removed in the next version of React.
A comment  from the same thread sums up our options: "We’re removing the warning because people work around it with isMounted which defeats the purpose. So we might as well not warn. But the proper solution is cancellation.  I'm personally not sure what to do about it.


A logo for this library

I'd like this library to have a logo! Here's where it'd go in the README:

Motivation
I read somewhere (I can't find the post but would absolutely love to re-read) that talked about the power of branding for open source projects. Having an image in your mind is a lot more memorable than a name or a line of code. Plus, it adds some legitimacy to show that the project has gotten real attention.
Inspiration

The 🛠 emoji
The play button logo for GitHub Actions:

How you can help
If you're interested in contributing this, that'd be amazing. You can drop some potential directions in this issue and we can talk about them!

I'll give this a go

Ooh awesome @JohnPhamous! These are really cool directions. I think I'm a fan of a circular shape - the third one feel closest. Not sure about the colors though; I find them a little too bright.
Maybe something like this, reusing the gradient from the https://github.com/actions logo...

Also playing around a bit with the bottom of the wrench (the little bolt negative space). I'd prefer something a lot less visually

@JasonEtco Love the changes you made to the 3rd one. I'll take your feedback and make the changes sometime this weekend.

In all the variations, I like the circle and somehow tying back into the actions logo.  The allen wrench hole with circle and tie back to action logo is nice.

A couple of options which give a nod to the wrench.


Update storsimple-overview.md

Some typos corrected and some statements made more accurate.

Hi @vidarmsft, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;


Build cannot find build artifacts

We get this error consistently on a new repo (migrated from a DevOps repo)
Unexpected error when running action. Error Message: Cannot find path 'C:\1\_work\xxx-Services\Noble-Metal-Services\.buildartifacts\Apps' because it does not exist
I have checked our build server; the path is present up to here: C:\1_work\xxx-Services\Noble-Metal-Services\Noble-Metal-Services
Any way to know where this is going wrong?  Full log attached below.
logs_24839909455.zip

It looks like AL-Go doesn't locate your apps?

Which means that nothing is built - and then the sign action fails because it cannot find any files to sign.

It was not locating the apps, but that was due to us not having straightened out the path references in Settings and ALGOORGSETTINGS. A separate issue thread here around those topics led us to get those aligned. Now the builds appear to complete normally. Thank you.


WIP: feat: horizontal slide viewer

This request moves the slide viewer to a horizontal position at the bottom of the window. In conjunction with the tags being moved to a vertical sidebar, this redesign is intended to allow more space in the window for the active canvas.
AB#17656 AB#17657

Currently, there is a glitch on resizing the bar, in which the thumbnails start to overlap. Also, hotkeys are not updated to right/left instead of up down.


added backend to contributors page

@mdazfar2
Hii i successfly added backend to contributors page
please mark Gssoc24 Level 3

@mdazfar2 @pandeyji711 is it allowed to make PR without raising issue?
beause you are doing it

I will be removing the repo from the program , if you are not ready to follow guidelines.
I have mentioned to you earlier as well.
Dont assign level3, even if you assign leaderboard is transparent to everyone, contributor are gonna flag to us.

I noticed you been merging the PR''s of this person without creating issues.
Can I know whats really happening here?

@sanjay-kv I have added backent to the website that is 178 lines of code!!!

@sanjay-kv  for rising issue i will take care of it next time!!

@pandeyji711 bro check your other level3 PRs
just adding button
adding social icons
Adding dark theme

Adding dark theam
we have to takel  js , we have to select element manage colour , managing transitions ,
its not just colour change!!!!
please cheak the darkmode pr , and resultent feature

@mdazfar2 I will put your repo on hold until manual review finish as I see recent few PR's when I check into the Code changes its doesn't qualify for the same. which is creating lot of issue among other contributors trusting this program leaderboard.
I have requested access to repo to manage as well.

@sanjay-kv
This Pr is capable for level 3 I have added Backend 178 lines of code ande manage  apis

@sanjay-kv I am agree that
adding button
adding social icons   is not meens for level 3
But in all other I have worked A lot!!!

@sanjay-kv please accept my request and manage accordingly

have you got it?

@pandeyji711
don't worry he will not remove all
edit thoses which are needed to be

@Yashgabani845 Thank you for your understanding. I will ensure to arrising issue  first in the future.

Hi i got the invite .
Thanks for adding me, i will be making some changes in labels.
https://github.com/mdazfar2/HelpOps-Hub/pull/72/files especially like this one.
Dont think like I'm taking too much authority over here, but to run program smoothly this action has to be taken, I wont be putting your project on hold.  but I hope you would be okay with changing some of the labels in your repo.
you can give level3 if you need but if one person getting level3 for these changes for multiple PRs is making the issue in the community and counted as spamming.
I hope you understand the seriousness of this issue. I have seen your repo, its really good, keep adding additional features but be very careful on assigning level 3 labels as it has 45 points.

@Yashgabani845 Thank you for your understanding. I will ensure to arrising issue first in the future.

Im not demotivating you as well, I saw your contribution looking at your code base. its amazing, not everyone is capable of doing such things. but its just the label issues in general. keep doing great contribution. sorry about any inconvenience caused to you @pandeyji711


Support printing values other than integers

The entry function called from C expects an integer, even though there is no specific reason for it to be so. This is just the first thing that worked.
https://github.com/jaseemabid/Olifant/blob/695047f5a858683392262c1736552233c3ef08fd/runtime/olifant.c#L4
This prevents olifant from returning any other kinds of data types (like boolean) as the final result of the program.
Option 1
Return a (type, value) tuple to C. This should be the easiest way to ensure C interop, efficiency and using the best of machine capabilities, but I'm not sure how I can work out details like memory management and complex data structures that won't fit into a word.  Maybe value will be a void* and should be treated like a black box?
Option 2
Encode runtime information into the value itself like Python's PyObject or using the last 3 bits of a word aligned pointer like Ocaml. This is kind of cool but seems unnecessary in a completely statically typed language. The paper incremental approach to compiler construction talks in depth about using a similar memory model for the scheme implementation.
I'm not sure which way to go.

CC: @balajirrao @nachivpn

Why do you even need a "final result of a program" now that you have FFIs to do IO? Typically the final result would simply be an exit code, right?

See how the result is printed. https://github.com/jaseemabid/Olifant/blob/695047f5a858683392262c1736552233c3ef08fd/runtime/olifant.c#L19

Ok, I thought you had an FFI to print values from Olifant. Why not add that? Are you trying to keep it "purely" functional?

Not really. As of now, I don't have enough machinery in the language to implement a show. This whole thing could be thrown away once the lang can do that.

Check this out:
https://github.com/nachivpn/javalette/blob/master/lib/runtime.ll


Add ZSTD compression

Requires libzstd and probably GDAL configuration flags.

I was going to open an issue about this :D, I'll try to open a PR.

Looks like it is included in libtiff 4.0.10: http://libtiff.maptools.org/v4.0.10.html

@snowman2 thanks for pointing that out. We'll get ZSTD then we upgrade to GDAL 3.2.0. Early 2021.


Bump wrapper

This enables the use of org.gradle.vfs.debug.

@bot-gradle test Gradleception please.

Sorry I don't understand what you said, please type @bot-gradle help to get help.

Let's try enabling it by default and see if it cramps everyone's style too much.


CLI: publish binaries so that developers do not need to build them locally

Building/upgrading the CLI can take a long time, let's publish pre-built binaries so that users can just download and use

@allan-bailey feel free to merge this issue if there's already another issue to track this

When these land, please let me know so we can update the install docs. Thanks!

@ebmifa can we mark this as done given https://github.com/MystenLabs/sui/releases/tag/devnet-0.13.0 ?

Hey @ebmifa!  I was just looking into this because it came up during my oncall rotation.  As I understand it right now (from looking at our recent releases), we do have prebuilts but:

Only for devnet, (testnet-0.15.1 release didn't come with any).
They include debug symbols.

Are the debug symbols here to stay?  They make the binaries quite large (1.1GB for sui) which limits their utility (long build vs long download).

Makes sense, thanks for clarifying!  This makes total sense for developers running validators and full nodes (and for us).  For developers who are using the sui CLI locally, we would likely want a stripped version of sui and maybe sui-faucet for a variety of platforms (eventually) as well.

We now have releases with the Sui binary, so closing this issue. https://github.com/MystenLabs/sui/releases


[bug] ClusterStatus machines count is wrong

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
The count of connected and requested machines in the ClusterStatus are 0, but in the MachineSetStatus they are good.
output of omnictl get clusterstatus foo -o yaml:
spec:
    available: true
    machines:
        total: 40
        healthy: 40
        connected: 0
        requested: 0

and output of omnictl get machinesetstatus foo-bar -o yaml:
spec:
    phase: 3
    ready: true
    error: ""
    machines:
        total: 32
        healthy: 32
        connected: 32
        requested: 32

Expected Behavior
output of omnictl get clusterstatus foo -o yaml:
spec:
    available: true
    machines:
        total: 40
        healthy: 40
        connected: 40
        requested: 40

Steps To Reproduce

create a cluster with a machineset
run omnictl get clusterstatus CLUSTER_NAME -o yaml - see that the connected and requested machines are set to 0
run omnictl get machinesetstatus MACHINE_SET_NAME -o yaml - see that the connected and requested machines is not 0

What browsers are you seeing the problem on?
No response
Anything else?
No response

moved to https://github.com/siderolabs/omni/issues/609


Sometimes getting Unauthorized on login

Sometimes when I attempt a login it first does a refresh, which results in a Unauthorized, which then rejects my login attempt.
It seems to depend on the contents of the superlogin session stored in localStorage. When the session is too old a refresh is triggered first, where login tokens are checked on the server. When these are invalid the Unauthorized error response is given. In these cases there won't be a login request triggered.
The fact that this only happens when the session is too old made it hard to reproduce for me.
I'm assuming the logins tokens become invalid after the sessionLife setting is exceeded.
Should there even be a refresh triggered when doing a login?

I see, this is probably happening because you don't call refresh on application load.
Or do not use the checkExpired option.
If you do, that's probably because we forget to delete the session.
In any cases, it feels overkill to do a refresh on login anyways.

I do call refresh on application load, but that doesn't seem to help.
It looks like only the login function deletes the session when it fails, no other methods delete the session.
I'm having a hard time assessing the ramifications, but maybe we should delete the session when refresh fails? Or maybe specifically when the response is Unauthorized?
I'm also thinking there are more calls where refreshing the session doesn't make sense, like register and social auth. I just noticed I have the same unauthorized problem with registering for example.

I now see the session is also deleted in _onLogout, which should be called in the injected error response handler:
https://github.com/micky2be/superlogin-client/blob/master/src/index.js#L112-L117
But it looks like, for me, response.status is undefined for some reason.
(I'm not using basic auth anymore)

This library could really use some unit tests. It looks like this is also a subtle result of changes in Axios, like: https://github.com/micky2be/superlogin-client/issues/25. The status was moved, into a response object.

For the unit test I agree. Just didn't have time to focus on it.

Released in 0.4.3


Updates needed for Wurm Unlimited 1.6

Thanks for contribute.

Yeah you are right - sorry about that  - I handedited on Github

Don't worry.


Harmonize (strip away Type) *Type models

ref: https://github.com/dandi/dandischema/pull/6/files#r639128369 saying
I am somewhat at loss of when Type suffix is used or not

Here are the ones without `Type` suffix
$> git grep 'class .*(' dandischema/models.py | grep -v 'Type('
dandischema/models.py:class HandleKeyEnumEncoder(json.JSONEncoder):
dandischema/models.py:class DandiBaseModelMeta(ModelMetaclass):
dandischema/models.py:class DandiBaseModel(BaseModel, metaclass=DandiBaseModelMeta):
dandischema/models.py:class PropertyValue(DandiBaseModel):
dandischema/models.py:class Anatomy(BaseType):
dandischema/models.py:class Disorder(BaseType):
dandischema/models.py:class ContactPoint(DandiBaseModel):
dandischema/models.py:class Contributor(DandiBaseModel):
dandischema/models.py:class Organization(Contributor):
dandischema/models.py:class Person(Contributor):
dandischema/models.py:class Software(DandiBaseModel):
dandischema/models.py:class Agent(DandiBaseModel):
dandischema/models.py:class EthicsApproval(DandiBaseModel):
dandischema/models.py:class Resource(DandiBaseModel):
dandischema/models.py:class AccessRequirements(DandiBaseModel):
dandischema/models.py:class AssetsSummary(DandiBaseModel):
dandischema/models.py:class Equipment(DandiBaseModel):
dandischema/models.py:class Activity(DandiBaseModel):
dandischema/models.py:class Project(Activity):
dandischema/models.py:class Session(Activity):
dandischema/models.py:class PublishActivity(Activity):
dandischema/models.py:class Locus(DandiBaseModel):
dandischema/models.py:class Allele(DandiBaseModel):
dandischema/models.py:class GenotypeInfo(DandiBaseModel):
dandischema/models.py:class RelatedParticipant(DandiBaseModel):
dandischema/models.py:class Participant(DandiBaseModel):
dandischema/models.py:class BioSample(DandiBaseModel):
dandischema/models.py:class CommonModel(DandiBaseModel):
dandischema/models.py:class Dandiset(CommonModel):
dandischema/models.py:class BareAsset(CommonModel):
dandischema/models.py:class Asset(BareAsset):
dandischema/models.py:class Publishable(DandiBaseModel):
dandischema/models.py:class PublishedDandiset(Dandiset, Publishable):
dandischema/models.py:class PublishedAsset(Asset, Publishable):

so from looks of it, if Type is desired for subclasses of BaseType then it is only
dandischema/models.py:class Anatomy(BaseType):
dandischema/models.py:class Disorder(BaseType):

which should be tuned up.  But what is the rationale/benefit from keeping Type suffix altogether?  It seems that BaseType is just an Enum-based DandiBaseModel.  So they all could just be as happy without Type suffix and thus become consistent (I do not see a value from somehow annotating with Type that it is Enum based... what if some would be relaxed and no longer Enum based?)

the difference between the Enum types and the BaseType is that BaseType allows users to fill in the identifier and name, while Enum types limit to specific predefined types. also the predefined types were short lists, the ones related to BaseType are not, except may be SexType, which indeed we could enumerate. all the non-enum types are also more biologically related, while the enumerated ones are more archive/data related.
i don't like this distinction but made life a little easier.


feature: add zk serializer.

Ⅰ. Describe what this PR did
add zk serializer.
Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

server/src/main/resources/registry.conf need to add?

registry.c

Yes, it needs to be added here.

Codecov Report

Merging #2554 into develop will decrease coverage by 0.01%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             develop    #2554      +/-   ##
=============================================
- Coverage      51.41%   51.40%   -0.02%
+ Complexity      2807     2805       -2
=============================================
  Files            553      553
  Lines          17698    17698
  Branches        2093     2093
=============================================
- Hits            9099     9097       -2
  Misses          7742     7742
- Partials         857      859       +2

Impacted Files
Coverage Δ
Complexity Δ

...o/seata/server/coordinator/DefaultCoordinator.java
54.63% <0.00%> (-0.52%)
28.00% <0.00%> (-1.00%)

...in/java/io/seata/server/session/GlobalSession.java
83.71% <0.00%> (-0.46%)
71.00% <0.00%> (-1.00%)


When I was trying to Build the Java Archive (JAR) and native library, I got the error shown that: Analysis of target '//tensorflow/java:libtensorflow_jni' failed; build aborted.

NOTE: Only file GitHub issues for bugs and feature requests.  All other topics will be closed.
For general support from the community, see StackOverflow.
To make bugs and feature requests more easy to find and organize, we close issues that are deemed
out of scope for GitHub Issues and point people to StackOverflow.
For bugs or installation issues, please provide the following information.
The more information you provide, the more easily we will be able to offer
help and advice.
What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?
Environment info
Operating System:
Installed version of CUDA and cuDNN:
(please attach the output of ls -l /path/to/cuda/lib/libcud*):
If installed from binary pip package, provide:

A link to the pip package you installed:
The output from python -c "import tensorflow; print(tensorflow.__version__)".

If installed from source, provide

The commit hash (git rev-parse HEAD)
The output of bazel version

If possible, provide a minimal reproducible example (We usually don't have time to read hundreds of lines of your code)
What other attempted solutions have you tried?
Logs or other output that would be helpful
(If logs are large, please upload as attachment or provide link).

There isn't enough detail here to go by. Please include as much detail as you can to reproduce the problem. Filling in all the fields above of the bug template would help, as would any logs of the steps you took and the full output of the error.

I followed the step of TensorFlow for Java. I installed all the requirement and clone the whole tensorflow repository from git.  I was caught the error when I was trying to use this commend:
bazel build -c opt
//tensorflow/java:tensorflow
//tensorflow/java:libtensorflow_jni
under the tensorflow folder.
The detailed errors:
ERROR: /Users/neil/tensorflow/tensorflow/core/BUILD:1226:1: no such target '//tensorflow/tools/git:gen/spec.json': target 'gen/spec.json' not declared in package 'tensorflow/tools/git' defined by /Users/neil/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: /Users/neil/tensorflow/tensorflow/core/BUILD:1226:1: no such target '//tensorflow/tools/git:gen/head': target 'gen/head' not declared in package 'tensorflow/tools/git' defined by /Users/neil/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: /Users/neil/tensorflow/tensorflow/core/BUILD:1226:1: no such target '//tensorflow/tools/git:gen/branch_ref': target 'gen/branch_ref' not declared in package 'tensorflow/tools/git' defined by /Users/neil/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: Analysis of target '//tensorflow/java:tensorflow' failed; build aborted.
INFO: Elapsed time: 0.178s
Neils-MacBook-Pro:tensorflow neil$ bazel build -c opt \

//tensorflow/java:tensorflow
//tensorflow/java:libtensorflow_jni
ERROR: /Users/neil/tensorflow/tensorflow/core/BUILD:1226:1: no such target '//tensorflow/tools/git:gen/spec.json': target 'gen/spec.json' not declared in package 'tensorflow/tools/git' defined by /Users/neil/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: /Users/neil/tensorflow/tensorflow/core/BUILD:1226:1: no such target '//tensorflow/tools/git:gen/head': target 'gen/head' not declared in package 'tensorflow/tools/git' defined by /Users/neil/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: /Users/neil/tensorflow/tensorflow/core/BUILD:1226:1: no such target '//tensorflow/tools/git:gen/branch_ref': target 'gen/branch_ref' not declared in package 'tensorflow/tools/git' defined by /Users/neil/tensorflow/tensorflow/tools/git/BUILD and referenced by '//tensorflow/core:version_info_gen'.
ERROR: Analysis of target '//tensorflow/java:tensorflow' failed; build aborted.

Ah, I see. The problem is that the configure script needs to be run first (as per the "install from sources" instructions on www.tensorflow.org) I will update the java README to reflect that.
Please run that and let us know if that solves your problem.


Rework token deserialization and remove asynchelper

In an effort to prevent some deadlock issues we've seen in production, reworked the way that the token stores deserialize items out of the database. Instead of trying to do a blocking-asynchronous call to the database in the JsonConverters, defer that work to the store itself, where it can be done in an asynchronous context.
I fully admit that this isn't nearly as elegant and clever, and it means that the stores need to know more about the structure of what's being deserialized. But that seems a small price to pay for much greater stability, and the ability to expunge the AsyncHelper.
There's also a little test coverage for OperationalDbContext.
#101

Hi @sixten, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
We will now validate the agreement and then real humans will evaluate your PR.
TTYL, DNFBOT;


Exclude mocked js from repo language statistics

Although it's not vendor code, I think it's the easiest way to exclude it
From https://github.com/github/linguist#vendored-code

It should be a golang project

👍   Go: up Js: down


CWE - ADP mismatch on CNA provided CWE information

I'm doing some research and noticed these CVEs have conflicting CNA / ADP CWE enrichment. Nothing I see as urgent but thought you might be interested in knowing so you can continue to refine your processeses.
CVE-2024-3367
CVE-2024-21885
CVE-2024-20361
CVE-2024-27310
CVE-2024-27312
CVE-2023-22934
CVE-2023-4234
CVE-2023-5675
CVE-2021-34983

@patrickmgarrity Our analysts have been checking through these and updating as appropriate. Thank you!

In all cases, there should be no daylight between the CNA's CWE determination and the ADP's CWE determination. In many of these, the CWE section of the ADP bucket has already been removed -- we'll work on this process of removing after the vendor updates to ensure things go quicker.


slashing with offchain workers

Plumbing to allow simplified slashing utilizing data stored by the offchain_index::set API.
Related #3722

I think this can be discarded in favour of #6220


Basic: normalise android triples when loading modules

The android API level can be ignored when loading the module.  The API
level controls the NDK APIs which are available and is equivalent to the
SDK version for Darwin.  This allows us to keep the API level in the
triple which future versions of Android's toolchain does.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - 4b8b5473dfcbecd3ca952d57d4fb53669d4f6269

Build failed
Swift Test OS X Platform
Git Sha - 4b8b5473dfcbecd3ca952d57d4fb53669d4f6269

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - 4b8b5473dfcbecd3ca952d57d4fb53669d4f6269

Build failed
Swift Test OS X Platform
Git Sha - 4b8b5473dfcbecd3ca952d57d4fb53669d4f6269

Seems that the test should be limited to when the Android stdlib has been built. I don't know how that can be transformed into a lit requirement, thought.

I mean, I could do // REQUIRES: OS=android.  But, I'm trying to be clever and create the Swift.swiftmodule manually just to make sure that we build and test with different triples.  This seems to work locally, I wonder why it doesn't work here.

@swift-ci please test

Build failed
Swift Test OS X Platform
Git Sha - 35899156c734ac3f8a46970ac35aa9e3bdf584ae

Build failed
Swift Test Linux Platform
Git Sha - 35899156c734ac3f8a46970ac35aa9e3bdf584ae


In Version 2.0.5,Close/open any plugin, the editor actions of the plugin will be reset.

Here is the reproduction process

I'm not sure what's happening here. Do the settings reset after the IDE is restarted?

When opening or closing the plugin and restarting it


Add New System Hook Optional Parameters Missing

I've noticed that the optional parameters are not available in AddHookOptions.
Is there any other way to pass optional paramters?
https://docs.gitlab.com/ee/api/system_hooks.html#add-new-system-hook
opt := &gitlab.AddHookOptions{
	URL: &hookEndpoint,
}

i've tried to add to token parameter by modifying the AddHookOptions in a fork, but it seems that Gitlab doesn't like it. :(

Did you try this implementation? I've tried id adding only the Token and Gitlab seems to reject it.

See the docs, URL is required so it needs at least that. The rest is optional: https://docs.gitlab.com/ee/api/system_hooks.html#add-new-system-hook

I'm sorry, I've tested the master version and it works like a charm.
Thank you very much for the quick response.


Fix for Issue#267

Wanted to know which release was the closed issue #267 fixed in ?

Closing due to inactivity.


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-k8s-20-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 682

Questions ?
Please refer to the Backport tool documentation


How to install MVC 6 template on VS 2014 CTP

Hi,
I need a mvc 6 template for look up to new features but i don't it. After I talk about this problem with Imran [on ASP.NET forum. ]
(http://forums.asp.net/t/1992275.aspx?How+to+install+mvc+6+template+on+Visual+Studio+14+CTP)
Why "MVC 5 View Page (Razor)" writes on "add new item" screen of ASP.NET vNext project?

I think there is no further action item on this bug and there is a connect bug open on vs tooling for this.


Fix background for terminals.

Fix background for terminals. Separate settings for GUI (default 'black') and cterm backgrounds (default 'bg')
This fixes issue #7.

By changing background on Normal highlight to g:vim_{gui,cterm}background, also GUI mode works better for me.

Thanks!

Thank you!


XML not well-formed bridge Leboncoin

Describe the bug
When I try to read the XML produced by the LeBonCoin bridge I've got this error (through firefox)
XML Parsing Error: not well-formed
Location: file:///tmp/mozilla_user0/dn6siIHE
Line Number 239, Column 11:Loyer 1300€ - Dépôt de garantie 1300€
The culprit seems to be this character :   , just after "1300"
To Reproduce
Steps to reproduce the behavior:
The URL parameters were
?action=display&bridge=LeBonCoin&keywords=&region=&department=29&cities=&category=10&pricemin=&pricemax=&estate=1&roomsmin=&roomsmax=&squaremin=100&squaremax=300&mileagemin=&mileagemax=&yearmin=&yearmax=&cubiccapacitymin=&cubiccapacitymax=&fuel=&owner=&format=Mrss
Expected behavior
The browser should be able to read the XML file.
Temporary solution that I did for me :
Modify
formats/MrssFormat.php
add this function (found here https://www.ryadel.com/en/php-skip-invalid-characters-utf-8-xml-file-string/ ):
 * Removes invalid characters from a UTF-8 XML string
 *
 * @access public
 * @param string a XML string potentially containing invalid characters
 * @return string
 */
function sanitizeXML($string)
{
    if (!empty($string))
    {
        // remove EOT+NOREP+EOX|EOT+<char> sequence (FatturaPA)
        $string = preg_replace('/(\x{0004}(?:\x{201A}|\x{FFFD})(?:\x{0003}|\x{0004}).)/u', '', $string);

        $regex = '/(
            [\xC0-\xC1] # Invalid UTF-8 Bytes
            | [\xF5-\xFF] # Invalid UTF-8 Bytes
            | \xE0[\x80-\x9F] # Overlong encoding of prior code point
            | \xF0[\x80-\x8F] # Overlong encoding of prior code point
            | [\xC2-\xDF](?![\x80-\xBF]) # Invalid UTF-8 Sequence Start
            | [\xE0-\xEF](?![\x80-\xBF]{2}) # Invalid UTF-8 Sequence Start
            | [\xF0-\xF4](?![\x80-\xBF]{3}) # Invalid UTF-8 Sequence Start
            | (?<=[\x0-\x7F\xF5-\xFF])[\x80-\xBF] # Invalid UTF-8 Sequence Middle
            | (?<![\xC2-\xDF]|[\xE0-\xEF]|[\xE0-\xEF][\x80-\xBF]|[\xF0-\xF4]|[\xF0-\xF4][\x80-\xBF]|[\xF0-\xF4][\x80-\xBF]{2})[\x80-\xBF] # Overlong Sequence
            | (?<=[\xE0-\xEF])[\x80-\xBF](?![\x80-\xBF]) # Short 3 byte sequence
            | (?<=[\xF0-\xF4])[\x80-\xBF](?![\x80-\xBF]{2}) # Short 4 byte sequence
            | (?<=[\xF0-\xF4][\x80-\xBF])[\x80-\xBF](?![\x80-\xBF]) # Short 4 byte sequence (2)
        )/x';
        $string = preg_replace($regex, '', $string);

        $result = "";
        $current;
        $length = strlen($string);
        for ($i=0; $i < $length; $i++)
        {
            $current = ord($string{$i});
            if (($current == 0x9) ||
                ($current == 0xA) ||
                ($current == 0xD) ||
                (($current >= 0x20) && ($current <= 0xD7FF)) ||
                (($current >= 0xE000) && ($current <= 0xFFFD)) ||
                (($current >= 0x10000) && ($current <= 0x10FFFF)))
            {
                $result .= chr($current);
            }
            else
            {
                $ret;    // use this to strip invalid character(s)
                // $ret .= " ";    // use this to replace them with spaces
            }
        }
        $string = $result;
    }
    return $string;
}

and then, modify the function stringify, at the end, (in the same file) to have :
                // Remove invalid non-UTF8 characters
                ini_set('mbstring.substitute_character', 'none');
                $toReturn = mb_convert_encoding($toReturn, $this->getCharset(), 'UTF-8');
                $toReturn = sanitizeXML($toReturn);
                return $toReturn;

Solution: discard non-utf8. Possibly log it. They respond with a captcha now which is a bigger issue.

https://github.com/RSS-Bridge/rss-bridge/issues/1820


Start IOCs automatically

Now that this hook exists in repo2docker, we can start supervisord (and whatever processes it manages, including caporoto IOCs) automatically. I think we should put a !supervsiorctl status command at the top of every notebook, so that users can check that the IOCs are up and have some concept of what is happening, but we don't need to rely on users to start the IOCs and wait for them to start.

I like this approach!

The task here is:

[ ] Port this file from intake-bluesky-demos to here. The paths might need to be changed.
[ ] Update every notebook that has a supervisord invocation (it would be in the first or second code cell, always) and replace it with something that runs !supervisorctl -c <path to supervisor.conf> status instead. Update the text around it to say something like, "Check that the IOC that simulates hardware is running. The output of this should include mini_beamline [...] RUNNING if all is well."


Convert all ActorIsolated to LockIsolated.

Can you briefly explain the reason behind this ?

Can you briefly explain the reason behind this ?
will actorisolated be deprecated ?

Yeah, we will be deprecating ActorIsolated in swift-concurrency-extras. It really just has no advanced over LockIsolated (as far as we can tell), yet it's less ergonomic to use.
Do you have an example of where it is better to use ActorIsolated over LockIsolated?

@Alex293 For reference, here's the PR deprecating: https://github.com/pointfreeco/swift-concurrency-extras/pull/29

I don’t have any specific use case, I was just wondering why, thanks !


rules for ignoring pa11y results

This pull request adds a new feature: you can now specify a URL that points to a YAML file with ignore rules for the pa11ycrawler. The rules will look like this:
"/":
  - message: "Duplicate id attribute *"
  - message: "Check that the title element describes the document."
"/*/about":
  - message: "* contrast ratio between the text and * background *"
"/dontcare":
  - type: notice

In this example, pa11ycrawler will ignore messages about duplicate id attributes and the <title> element on the root page of the website. On all "about" pages, it will ignore messages about contrast ratios. And on the /dontcare page, it will ignore any output that is of type "notice" (but still report warnings and errors).
We will need a separate repository for this YAML file, which I've created here: https://github.com/singingwolfboy/pa11ycrawler-ignore. It currently has one rule in it, suggested by @cptvitamin. More rules can be added, and this repo can be moved to the edx organization on GitHub. To test the crawler using this ruleset, you can run:
$ scrapy crawl edx -a port=8003 -a pa11y_ignore_rules_url=https://raw.githubusercontent.com/singingwolfboy/pa11ycrawler-ignore/master/ignore.yaml

@benpatterson can you review this, please?

Generally makes sense. Literally, will we be passing in the github url target for the yaml file (e.g., https://raw.githubusercontent.com/singingwolfboy/special_repo/master/my_yaml.yml )?

That's right. The YAML file can be hosted anywhere, it's just convenient to host it from GitHub, since doing so allows us to use pull requests to review changes to the ignore rules.

Not saying we'd do this, but just to think through it (and perhaps an ignorant question w.r.t. the requests library), but could the url target also be a local file?

ok right on
👍

@benpatterson: being able to specify a local file would be a nice feature, but it doesn't support that right now. We could either use https://github.com/dashea/requests-file so that you can specify a file:// URL instead of a http:// URL, or I could add another argument called pa11y_ignore_rules_file and you could specify the filename that way. Either way, I think it makes sense to wait until we have an actual need before implementing this.
As an aside, I really wish I could use the Scrapy settings system to specify this information, since it's the sort of thing that seems unlikely to change from one run to the next. Unfortunately, settings can't be accessed in a spider __init__() method, which means trying to use settings for this gets crazy complicated. I figured that spider arguments were the better way to go, even if they get long.


Merge LoRA into 405B

Env: 8 A100-80G GPUs, Transformer: 4.43.3, torch: 2.4.0+cu121
When I tried to merge LoRA into LLaMA-3.1 405B with load_in_8bit=True, I got an error in BNB;
so I tried to use CPU and merge 405B with bf16, and it was still killed suddenly.

Could you please provide more information: What code did you use to merge? What is the full error message that you got for both tests you did?

Thanks, for your prompt reply!
For case 2, no error was received and it was killed.
My test code:
`
import torch
from peft import PeftModel
import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name_or_path = "llama3/Meta-Llama-3.1-405B"
lora_path = ""
output_path = ""
base_model = AutoModelForCausalLM.from_pretrained(
model_name_or_path,
load_in_8bit=False,
torch_dtype=torch.bfloat16,
device_map={"": "cpu"}
)
lora_model = PeftModel.from_pretrained(
base_model,
lora_path,
torch_dtype=torch.bfloat16,
device_map={"": "cpu"}
)
model = lora_model.merge_and_unload(progressbar=True)
model.save_pretrained(output_path)
base_tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
base_tokenizer.save_pretrained(output_path)
`

For case 1, I made a device_map allocation in advance.
Return: Error an illegal memory access was encountered at line 524 in file /mmfs1/gscratch/zlab/timdettmers/git/bitsandbytes/csrc/ops.cu
num_gpus = 8
num_layers = 126
Specify how many layers GPU 0 should handle
layers_for_gpu0 = num_layers // num_gpus // 2
remaining_layers = num_layers - layers_for_gpu0
layers_per_remaining_gpu = remaining_layers // (num_gpus - 1)
extra_layers = remaining_layers % (num_gpus - 1)
Assign layers to GPU 0
for layer in range(layers_for_gpu0):
device_map[f'model.layers.{layer}'] = 0
Distribute remaining layers across the other 7 GPUs
for i in range(1, num_gpus):
start_layer = layers_for_gpu0 + (i - 1) * layers_per_remaining_gpu + min(i - 1, extra_layers)
end_layer = layers_for_gpu0 + i * layers_per_remaining_gpu + min(i, extra_layers)
for layer in range(start_layer, end_layer):
device_map[f'model.layers.{layer}'] = i
    # Ensure LoRA modules for each layer are on the same GPU
    for lora_module in ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'mlp.gate_proj', 'mlp.up_proj', 'mlp.down_proj']:
        device_map[f'model.layers.{layer}.{lora_module}.lora_A'] = i
        device_map[f'model.layers.{layer}.{lora_module}.lora_B'] = i
        device_map[f'model.layers.{layer}.{lora_module}.lora_embedding_A'] = i
        device_map[f'model.layers.{layer}.{lora_module}.lora_embedding_B'] = i

device_map['model.embed_tokens'] = 0
device_map['model.norm'] = 0
device_map['lm_head'] = 0
device_map['model.lm_head.lora_A'] = 0
device_map['model.lm_head.lora_B'] = 0
device_map['model.lm_head.lora_embedding_A'] = 0
device_map['model.lm_head.lora_embedding_B'] = 0
model_name_or_path = "llama3/Meta-Llama-3.1-405B"
lora_path = ""
output_path = ""
base_model = AutoModelForCausalLM.from_pretrained(
model_name_or_path,
load_in_8bit=True,
torch_dtype=torch.bfloat16,
device_map=device_map
)
lora_model = PeftModel.from_pretrained(
base_model,
lora_path,
torch_dtype=torch.bfloat16,
device_map=device_map
)

Thanks for the additional information. Could you please paste the full error message (stack trace) for case 1? Also, if you monitor memory, do you observe that there might not be enough memory for the operation? Merging incurs a slight memory overhead.
Unfortunately, this is not my area of expertise and I don't have a setup to test this. Pinging @matthewdouglas and @Titus-von-Koeller in case they can help with this.

@BenjaminBossan Thanks, this is the full error message:

Unloading and merging model:   0%|          | 0/2654 [00:00<?, ?it/s]/home/oppoer/.local/lib/python3.10/site-packages/peft/tuners/lora/bnb.py:83: UserWarning: Merge lora module to 8-bit linear may get different generations due to rounding errors.
  warnings.warn(

Unloading and merging model:   0%|          | 7/2654 [00:02<12:46,  3.45it/s]
Unloading and merging model:   0%|          | 9/2654 [00:02<11:27,  3.85it/s]
Unloading and merging model:   0%|          | 11/2654 [00:02<10:29,  4.20it/s]
Unloading and merging model:   0%|          | 13/2654 [00:04<18:46,  2.34it/s]
Unloading and merging model:   1%|          | 17/2654 [00:10<39:43,  1.11it/s]
Unloading and merging model:   1%|          | 19/2654 [00:15<56:25,  1.28s/it]
Unloading and merging model:   1%|          | 20/2654 [00:19<42:35,  1.03it/s]
Error during merge: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Error positioning: peft/utils/integrations.py
im, imt, SCim, SCimt, coo_tensorim = bnb.functional.double_quant(im)


Tinkerbell: Can't deploy multiple controllers

When deploying a Tinkerbell cluster with multiple controllers, the following error is shown:
module.controllers.null_resource.copy-controller-secrets[0]: Creation complete after 3m24s [id=4321032068653746242]

Error: Invalid index: The given key does not identify an element in this collection value.

Error: Invalid index: The given key does not identify an element in this collection value.

FATA[0416] Applying cluster failed: applying platform: failed checking execution status: exit status 1  args="[]" command="lokoctl cluster apply"

Sample config file:
cluster "tinkerbell" {
  asset_dir               = "./assets"
  name                    = "demo"
  dns_zone                = "example.com"
  ssh_public_keys         = ["ssh-rsa ..."]
  controller_ip_addresses = [
    "<IP_ADDRESS>",
    "<IP_ADDRESS>",
    "<IP_ADDRESS>",
  ]

  experimental_sandbox {
    pool_path          = "/home/johannes/pool"
    flatcar_image_path = "/home/johannes/cluster/flatcar_production_qemu_image.img"
    hosts_cidr         = "<IP_ADDRESS>/24"
  }

  worker_pool "pool1" {
    ip_addresses    = [
      "<IP_ADDRESS>",
      "<IP_ADDRESS>",
      "<IP_ADDRESS>",
    ]
  }
}

component "httpbin" {
  ingress_host = "httpbin.example.com"
}

cc @invidian


Create npm script for alpha video postprocessing from Character Animator

Also check and compare ffmpeg commands:
https://css-tricks.com/overlaying-video-with-transparency-while-wrangling-cross-browser-support/

lgtm


TypeError: Cannot read property 'hasOwnProperty' of null

Garza, Brent Anthony<EMAIL_ADDRESS>1:58 AM (17 minutes ago)
to me
When I copy and paste the following website (https://gerontologist.oxfordjournals.org/content/43/suppl_2/96.full#ref-16), I receive the following error:

This seems to be a rare occurrence, but I have seen a similar issue before. This error is thrown when value being posted to TrackBrowser server is null.
I can't reproduce the issue when I copy-paste the URL in TrackBrowser, but I will keep trying to get the same issue.

Can't reproduce - will reopen this issue when same issues are reported


GitSPDX.py runs as a bash script when executed

After successful install I tried to execute GitSPDX.py:
$ ./GitSPDX.py
./GitSPDX.py: line 19: import: command not found
./GitSPDX.py: line 20: import: command not found
./GitSPDX.py: line 21: import: command not found
./GitSPDX.py: line 22: import: command not found
./GitSPDX.py: line 23: import: command not found
./GitSPDX.py: line 24: import: command not found
./GitSPDX.py: line 25: import: command not found
./GitSPDX.py: line 28: import: command not found
./GitSPDX.py: line 29: import: command not found
./GitSPDX.py: line 30: import: command not found
from: too many arguments
./GitSPDX.py: line 33: import: command not found
./GitSPDX.py: line 35: syntax error near unexpected token `('
./GitSPDX.py: line 35: `fileConfig = Config.Config()'

GitSPDX.py has the execute bit but no shebang line (like #!/usr/bin/python2) at the top of the file. This is potentially quite dangerous... on some Linux systems there is a /usr/bin/import command installed that will clobber any file specified as an argument to it if run during an X session.
At any rate, I understand the readme file says to invoke like python GitSPDX.py, but this is less user friendly (in my humble opinion) than allowing the program to be run directly.

According to the usage documentation you should run the program using python GitSPDX.py. I agree that ./GitSPDX.py would be a more friendly option but would be an enhancement at this point.

Feature has been added.


feat: cloud source repo scanner

Fixes #19

[x] Implement cwaler
[x] Add entry in the scanner.py
[x] update example configuration
[x] Add unittest
[x] Add dummy test file to compare against

The unit test will fail because the source repo hasn't been enabled in the test-project for gcp-scanner yet in google cloud.
Once the source repo is enabled, updating the test/sourcerepo file will fix the test.

@mshudrak need a review on this PR

LGTM


[STYLE CHANGE] Audio button in the  Gita storyline

Description:
[Provide a clear and concise description of the style change you are requesting.]
want to add some additional style changes in the audio button
Reason for Change:
to make it more appealing and attractive
[Explain the reasons behind the requested style change, such as usability, branding, or user experience.]
Current Styling:
[Describe the current styling that needs to be changed.]

Proposed Styling:
[Describe the desired styling or changes you would like to see.]
Screenshots or Design Mockups (optional):
[If available, include relevant screenshots or design mockups illustrating the requested style changes.]
Additional Information (optional):
[Provide any additional information that may help in understanding or implementing the style change. This can include any specific requirements or constraints.]

Priority Level (optional):

[ ] Low
[ ] Medium
[ ] High

Timeline (optional):
[Specify any preferred or required timeline for implementing the style change.]
Possible Implementation Approach (optional):
[If you have any suggestions or ideas on how the style change could be implemented, feel free to provide them here.]

Go for it!


Adding Custom Inpainting Model ID

How do I add custom Inpaiting Model ID. Specifically majicMIX realistic
Running threw windows.

Please save the downloaded model 'majicmixRealistic_v7-inpainting.safetensors' in the 'stable-diffusion-webui/models/Stable-diffusion' folder. Then, restart either webui.bat or webui.sh. Following this, it should appear in the 'Inpainting Model ID' list under the 'Inpainting webui' tab.

its showing up  as a checkpoint but not in the model id list

The Model ID of the added safetensors file will appear not under the 'Inpainting' tab, but under the 'Inpainting webui' tab.

Thanks its there!


scripts/operatorhub-deploy: Add the script

Description

Add the script operatorhub-deploy.sh for the user to be able to deploy
new Eunomia version to OperatorHub without effort. All files needed by
operator-sdk are being automatically generated or downloaded and in the
end cleaned up. The result of running this script is a PR in the
https://github.com/operator-framework/community-operators repository.
Also add a simple howto in .github/RELEASE.md.
A test PR in community-operators: https://github.com/operator-framework/community-operators/pull/1226

created only by running the script being added in this PR
a draft one, wanted to test only
all checks have passed

Fixes #251
Type of change

New feature (non-breaking change which adds functionality)

Checklist

[ ] Unit tests and e2e tests updated (not needed in this PR)
[x] Documentation updated

@seanmalloy I thinks it's now ready to be merged


#136 pictoralist patch

Resolves #163
See changelog for details

Bar charts now generate successfully for small data sets
Duplicate logging statements removed
set_timeframe now catches errors where data < 3 months

comment line 210 in main.py to override for verification of patch

Added conditional formatting of displays to keep text within graph bounds cleanly and legibly

Attached images show conditional formatting of annotations:

Annotations of data < 25% will now display above data points or bars in a suitable color

Going to change graphing order, plotting goal line first will plot it behind the bars, increasing legibility


TypeScript definition file for is.js

This definition file allows TypeScript developers to easily integrate is.js into their projects. It allows TypeScript to understand is.js, and make sure no incorrect values are submitted to the tests (for example: checking is.lowerCase on a number will be prevented).

Let's defer to DefinitelyTyped.


docs: [IAC-2021]: Document Terraform backend initialization

Description

Document the use of environment variables to initialize the terraform backend to populate the terraform init -backend-config parameters at runtime
JIRA

Screenshot / Preview

PRs must meet these requirements to be merged:

[x] Successful preview build.
[x] Code owner review.
[x] No merge conflicts.
[x] Release notes/new features docs: Feature/version released to at least one prod environment.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665dc5477c1671c8d8c02d11--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665dd1e34edce30c16f27a5e--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665dd46fa3d21d09787c64fd--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665dd49d8484c0790ce44fea--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665dd638a3d21d0bcb7c669c--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665dee60bcfaf8154ff941d4--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665ef793e793000078f578b9--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665ef81a27ec9a007922e10f--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665f125ae79300157df57bc0--harness-developer.netlify.app


added priority in MX lookup

MX was ignoring value of priority. Added the ability to parse it

Fixed in https://github.com/StephanGeorg/node-dig-dns/commit/9b25c91fca8db5427f571e3a0aee245db85fd30e


Console application's role permission can't be edited from Console Settings ->  Roles section

Describe the issue:

Go to  Console Settings ->  Roles
Create a role with few permission
Click on the crated role
Go to permissions section
Can't change the permissions
If you go to User Management -> Roles -> created role's permissions, you can edit the permissions

https://github.com/wso2/product-is/assets/25483865/8ccbc039-2de5-4aab-9608-fb12c1c11a45
Expected Behavior:
Permission for editing should be possible from the view
Console Settings ->  Roles -> selected role permission

As per an offline discussion with @thanujalk, it was decided to keep this as a limitation.
Since there's no way to have a mapping with the created role response and the resource collections.

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Why is the Extension deprecated in VS Code?

Hi. just wondering why such a great extension is deprecated?
I know VS Code can sync your settings out of the box, but with the extension it's way cooler since you have full control over your files threw github.
I guess I can still use it, at least I could install it from the commandline. Lets see ^^

这是来自QQ邮箱的假期自动回复邮件。你好，我最近正在休假中，无法亲自回复你的邮件。我将在假期结束后，尽快给你回复。


interactive editor?

could load stuff in, fuzzy match to select one of the items with prompt_toolkit, pick a field to edit, and then modify and write back
instead of editing the json file manually

is definitely possible using prompt toolkit, but I feel like its slightly outside the scope of this project.

this also supports yaml now, so easier to edit/copy around blocks now
can see my example in https://github.com/seanbreckenridge/ttally

Can actually do this pretty easily within python now, using my pyfzf_iter fork: https://github.com/seanbreckenridge/pyfzf

picking added in https://github.com/seanbreckenridge/autotui/commit/e97d9ae62e19b1f7469e905c36b543242e4ee7e8
still need to add editing attributes of items

added in 2b7344bdb1bf67440b78eebda920cb2f8975c769


Issues in ol.source.raster

Dear All Users,
I am using openlayers raster source (ol.source.Raster) and want to store pixel values in declared public array but array not accessible inside operation "operation: function (pixels, data) {......." please help in this regard.
I also tried to declared that array in lib: but not succeed.....
...please suggest

Please ask usage questions on stackoverflow instead with the tag openlayers
https://stackoverflow.com/questions/tagged/openlayers


Passing options to custom input

Hello,
I have the following code:
App.SliderField = Ember.TextField.extend
  min: null
  max: null
  didInsertElement: ->
    @$().slider(min: @get('min'), max: @get('max'), value: @get('value'))

  updateValue: (->
    @$().slider('setValue', @get('value'))
  ).observes('value')

Ember.EasyForm.Config.registerInputType('slider', App.SliderField)

Template:
{{input numberOfSomething as="slider" min=100 max=1000}}

But min and max is null in App.SliderField  instance...
I have tried to pass them as inputConfig="min:100;max:1000", but no luck...
Is there any way to pass settings to my custom input?
PS: If I add minand max to the Ember.EasyForm.Input.inputOptions everything works, but I don't think it is a good solution.

@yratanov, @sbounmy, @jcope2013 you can proxy those fields from parentView in your case:
max: Ember.computed.alias('parentView.max'),
min: Ember.computed.alias('parentView.min'),

and then pass max in input helper
{{input numberOfSomething as="slider" min=100 max=1000}}

Most of the code of the library was rewritten to be an Ember addon compatible with Ember 1.13 and 2.*.
Now it's possible to do exactly what you want. Any option passed to the input-for component is passed to you custom input.
You can get the most recent version of ember-easy-form on the branch 1-0-stable or as an ember-addon here.
Make sure to read the README and CHANGELOG in the 1-0-stable branch.


Implementation of approximate exploitability

Hi,
Is there any implementation of approximate exploitability, like the one mentioned in the paper: https://ala2020.vub.ac.be/papers/ALA2020_paper_43.pdf
It seems that in the example code, the full game tree traversal is needed to calculate exploitability.
Please let me know if I misunderstood anything.
Thanks.

Hi @lilith829, tagging @finbarrtimbers who has been working on this.
The code we have for that work (updated paper here, btw: https://arxiv.org/abs/2004.09677) uses a fair bit of internal code that is not easy to release.
However, I have started a basic one that makes the necessary modifications to IS-MCTS (to play against a fixed policy). It's currently only a prototype because it does everything using a table (even the learning part) so cannot directly be used with neural-nets in an AlphaZero-style learning loop for large games yet. I had not released it yet because there's still some finishing touches I need do, but to get to the full version would require some work. I would be inclined to finally finish if it would be useful to you.
The best option currently is to run DQN against a fixed policy. We don't have an example of that but we can make one too. For now you can look at breakthrough_dqn.py and just modify it to play against fixed policies.

Thank you @lanctot. Yes, some examples using IS-MCTS against fixed policy would be helpful. For DQN approach, Do you mean to replace random bots with fixed policy (in eval_against_random_bots)? Is the returned value equivalent to the exploitability? Thanks.

Thank you @lanctot. Yes, some examples using IS-MCTS against fixed policy would be helpful.

Ok there's a bit of work to do on this before I can release it, might be a week or two. I'll keep you posted.

For DQN approach, Do you mean to replace random bots with fixed policy (in eval_against_random_bots)? Is the returned value equivalent to the exploitability? Thanks.

Yes that is what I meant but no, it is not equivalent to exploitability. It is only a lower bound on exploitability. But at least it is something that is possible to get in larger games (and it is starting to be used in papers, e.g. see DREAM). I have an adaptation of this already that I can easily submit for the next update (Monday).

Ok there's a bit of work to do on this before I can release it, might be a week or two. I'll keep you posted.
Great. Thank you.

Yes that is what I meant but no, it is not equivalent to exploitability. It is only a lower bound on exploitability. But at least it is something that is possible to get in larger games (and it is starting to be used in papers, e.g. see DREAM). I have an adaptation of this already that I can easily submit for the next update (Monday).

Sounds great. As an exercise, I tried to run the example that runs DQN agents against random agents. For breakthrough game, the reward for both players are approached to 1 quickly. But when I switch the game to kuhn_poker without other changes, the reward doesn't seem to converge and not increase by episodes (even at 1M episodes). Do you know what could be the reason? Thanks.

Sounds great.

I wrote this today, btw, so will be available on the next update (Monday).

As an exercise, I tried to run the example that runs DQN agents against random agents. For breakthrough game, the reward for both players are approached to 1 quickly. But when I switch the game to kuhn_poker without other changes, the reward doesn't seem to converge and not increase by episodes (even at 1M episodes). My understanding is that with more training epochs, the rewards should be increasing and converge to some value, right? Do you know what could be the reason?

No, that's not true. In this class of games, standard RL  (i.e. DQN in self-play) is not necessarily doing strict policy improvement. it The optimal policy may be stochastic, and DQN is built on top of Q-learning for MDPs, and in the case of Kuhn poker the Markov assumption which no longer holds. So you should not expect to get monotically better over time. So, there's no guarantee on how the policies will change, and there can be intransitivities (meaning, there's no guarantee that the policies will strictly do better than uniform random over time either). Even if you use an algorithm that gets closer to a Nash equilibrium, there is no guarantee that it will strictly improve against uniform random, because "optimal" here means getting closer to a Nash equilibrium and not beating some specific strategy more than before.
Some good places to start:

https://hal.archives-ouvertes.fr/hal-00601941/document
https://arxiv.org/pdf/1806.02643.pdf

Thanks for the detailed explanation. I understand better about the results. But I still don't know the reason behind using DQN.

Why does playing against DQN agent get the lower bound of exploitability? Since DQN agent does not necessarily play the best response policy, the reward of playing against a DQN policy is higher than that of playing against best response policy. In that case, shouldn't that be upper bound?
For poker game, can I still use DQN for exploitability approximation? Since poker is not MDP, the DQN learned policy is unlikely to be close to best response policy over time. In that case, what's the meaning of reward when playing against DQN agent? If I play policy A against DQN policy and get higher reward than play policy B, can I conclude that policy A is stronger than policy B in this game?

Thank you.

We use DQN here as it's a really simple agent that is inexpensive to run. It is not particularly strong. If anything, it's likely to be relatively weak.

It's a lower bound because other agents could do better. Another way of wording it is that the exact best response is going to be at least as good as the approximate best response learned by DQN here. If it was an upper bound, we'd be saying the opposite: That the exact best response would be, in some cases, worse than the approximation learned here. This is clearly not possible, as the best response will be strictly better than the approximation learned here.

You totally can, but it's possible for it to be a bad lower bound. I know that ReBeL 0 used a variant of DQNBR (i.e. the approximate best response, or ABR, implementation that uses DQN as the BR) that used double DQN and they got strong results. In our experiments, DQNBR has done quite well in poker when set up appropriately. We had to add some additional features with our implementation to get it to be a strong approximation; I'm not sure how Marc's implementation does in poker.

I'll answer your question in terms of "approximate exploitability", which is equal to the average reward that DQN receives when playing against a fixed policy.
If you calculate the approximate exploitability of two policies, and A has a lower approximate exploitability than B, it's not conclusive proof that A is less exploitable than B, but it is a strong indication. So as long as you are careful about what you claim, I think you could make that statement. I would take a look at what Facebook claimed in the ReBeL paper, or what I claimed in the ABR paper 1; if that language is unclear, let me know and I'll try to reword it, as I go into depth discussing this there.
This is why we spend so much time in the ABR paper validating the ISMCTSBR agent; the strength of the agent is directly proportional to the accuracy of the approximation, so if you can prove that your agent is able to approximate the BR in a variety of settings where the BR is known, you can have confidence in how accurate the approximation is in settings where you don't know the exact BR value.
Finally, as I say in the ABR paper, there is no reason to use an approximate version of exploitability in games like Kuhn Poker, where the exact BR is tractable to compute. It is strictly better, and I would encourage you to use it. Approximate exploitability is only useful in settings like Heads Up No Limit Hold'em, where the exact BR is too expensive to compute.

I should also note that ABR is a bit of a proof by counterexample situation; if ABR manages to exploit your agent, then that's a strong indication that your agent is weak, but if ABR doesn't manage to exploit your agent, it's tough to draw any definitive conclusions.

Finbarr's answer is really great, I just wanted to touch on a specific point:

* For poker game, can I still use DQN for exploitability approximation? Since poker is not MDP

When you fix the opponents' policies and only have one agent learning, then it is an MDP (the fixed policies are part of the environment), and so it's ok to use RL to look for approximate best responses, even though it's an imperfect information game. More on why in Finbarr's paper and also I realized I forgot to mention PSRO which uses this fact very strongly in its approach: https://arxiv.org/abs/1711.00832-- see also especially Greenwald et al "Solving for Best Responses and Equilibria in Extensive-Form Games with Reinforcement Learning Methods" where they go into a lot more detail.
The problem crops up when agents are learning together. Strictly speaking, DQN in self-play still violates the Markov assumption even in Breakthrough, but it is much less of a problem in practice in perfect information games because the quality of the policy does not depend so strongly on information that you don't have (like hidden cards).

Thanks Finbarr and Marc for the explanation and clarification. I will spend some time digging into the details you guys share. I am looking forward to trying the rl_response.py.

When you fix the opponents' policies and only have one agent learning, then it is an MDP (the fixed policies are part of the environment), and so it's ok to use RL to look for approximate best responses, even though it's an imperfect information game.

I see. If I set the fixed policy to be the optimal policy in rl_response.py and play Kuhn poker, the DQN agent will eventually reach Nash equilibrium or close, right? When we use DQN to approximate the exploitability of game like HUNL, it would probably be okay, but I would just get an exploitability lower than the true exploitability, right?

there is no reason to use an approximate version of exploitability in games like Kuhn Poker, where the exact BR is tractable to compute. It is strictly better, and I would encourage you to use it. Approximate exploitability is only useful in settings like Heads Up No Limit Hold'em, where the exact BR is too expensive to compute.

Thanks for the information. Indeed, I use Kuhn poker as a toy example to try out the DQN agent strength. The initial game I used was heads-up limited poker which has 10^14 states. But it has OOM error when computing exploitability. The code first converts the callable policy to tabular policy which is too big to be put in the memory. Computing the exploitability of the game of this size should be doable as long as the tabular policy is not to be fitted into the memory entirely, but computed on-the-fly while doing tree traversal. Is it working?

Thanks Finbarr and Marc for the explanation and clarification. I

When you fix the opponents' policies and only have one agent learning, then it is an MDP (the fixed policies are part of the environment), and so
I see. If I set the fixed policy to be the optimal policy in rl_response.py and play Kuhn poker, the DQN agent will eventually reach Nash equilibrium or close, right?

No, it will converge to a(n approxinate) best response which is not necessarily Nash. To see this think of Rock, Paper, Scissors. An equilibrium mixed strategy is the 1/3, 1/3, 1/3. Rock is a best response to this, but rock is not an equilibrium strategy.

When we use DQN to approximate the exploitability of game like HUNL, it would probably be okay, but I would just get an exploitability lower than the true exploitability, right?

Yes exactly, it is only a lower bound.

Thanks for the information. Indeed, I use Kuhn poker as a toy example to try out the DQN agent strength. The initial game I used was heads-up limited poker which has 10^14 states. But it has OOM error when computing exploitability. The code first converts the callable policy to tabular policy which is too big to be put in the memory. Computing the exploitability of the game of this size should be doable as long as the tabular policy is not to be fitted into the memory entirely, but computed on-the-fly while doing tree traversal. Is it working?

All of our exploitability algorithms require the full policy to be in memory. The best bet for large games is DQN in rl_response.py

Hi @lilith829 , it is now in: https://github.com/deepmind/open_spiel/commit/9122a9bf520cfd2dcfc0ac2e8f0c63811a0b6e15

Computing the exploitability of the game of this size should be doable as long as the tabular policy is not to be fitted into the memory entirely, but computed on-the-fly while doing tree traversal. Is it working?

The tree traversal code in question (which I wrote) was optimized for speed, not memory usage, so it keeps a ton of stuff in memory. While it is technically feasible to calculate a BR in limit poker, our implementation would need drastic changes to do so. I also strongly suspect you'd have to distribute the BR calculation across multiple machines to make it happen in a reasonable amount of time, while our implementation is single machine only.
This is part of why we made ABR, as ISMCTSBR/DQNBR scale very nicely.


Unable to install on Heroku

I have followed the instructions provided on the wiki to install Huginn on Heroku.
I have run the script ./bin/setup_heroku it works up to when the app name is verified and it returns a blank app name.
I am probably doing something wrong....
Output below:
[returntrip@inferno huginn]$ ./bin/setup_heroku
Welcome<EMAIL_ADDRESS> It looks like you're logged into Heroku.
It looks like you don't have a Heroku app set up yet for this repo.
You can either exit now and run 'heroku create', or I can do it for you.
Would you like me to create a Heroku app for you now in this repo? (y/n) y
Creating hidden-hollows-4338... done, stack is cedar-14
https://hidden-hollows-1111.herokuapp.com/ | git@heroku.com:hidden-hollows-1111.git
Your app name is ''.  Is this correct? (Y/n)

Welcome @returntrip, it looks like Heroku may have changed something.
Did you use the "Deploy to Heroku" button on the README, or clone the repo first and then run ./bin/setup_heroku?  Both should work, but I want to know which one you did so I can test it.

Hi Andrew,
I did clone the repo. Thanks for helping.
On Mon, Aug 31, 2015 at 5:00 PM Andrew Cantino<EMAIL_ADDRESS>wrote:

Welcome @returntrip https://github.com/returntrip, it looks like Heroku
may have changed something.
Did you use the "Deploy to Heroku" button on the README, or clone the repo
first and then run ./bin/setup_heroku? Both should work, but I want to know
which one you did so I can test it.
—
Reply to this email directly or view it on GitHub
https://github.com/cantino/huginn/issues/1000#issuecomment-136398821.

Okay, thanks.  I'll take a look at that soon.

Hey @returntrip, that's odd, it just worked for me.  I cloned the repo, bundled, and ran bin/setup_heroku.  It gave me
Would you like me to create a Heroku app for you now in this repo? (y/n) y
Creating afternoon-retreat-6027... done, stack is cedar-14
https://afternoon-retreat-6027.herokuapp.com/ | https://git.heroku.com/afternoon-retreat-6027.git
Git remote heroku added
Your app name is 'afternoon-retreat-6027'.  Is this correct? (Y/n)

It's odd that you didn't see Git remote heroku added either.  Mind trying again?

Hi @cantino, sorry for the delay. I have tried again with no success. However, I have successfully deployed the app using the button.
It is probably me doing something wrong....
Not sure if this helps, I guess the script is looking for "heroku info" to return the app name. In my case I get this
[returntrip@inferno huginn]$ heroku info
!    No app specified.
!    Run this command from an app folder or specify which app to use with --app APP.
[returntrip@inferno huginn]$
My best guess is that I have not been able to setup the default heroku app correctly hence heroku info is returning the above instead of the app name. This in turn affects your script...
It must be either user error (me) or mabe my Fedora install (probably still my fault tho hahaha)
If I am the only one unable to run the script, it might be best to close this report. In case you want me to continue troubleshooting let me know. I will try my best.

Appreciated!  I'm sure you've found a real edge case with the script, but I don't know what it is.  If the button worked for you, that's great, you should be able to use it from now on with bin/update_heroku.

Closing since we haven't heard back.  Hope it's working for you!


Perf Improvement - StrideArrays (Julia)

I just ran a fast experiment using StrideArrays - it seems to improve the GC work and has alt least 2ms impact performance on my machine.

Someone on Julia discourse pointed out that using StrideArrays breaks the rules (because it seems the StrideArrays implementation does not check bounds by default.
This seems to be, indeed, the case - their Github repo showcases a warning about this (but the documentation does not mention anything about the subject).
Now that I know, it is only fair to add the disclaimer here - and also apologize for not digging deep enough at the time.
I'll do an update soon, keeping the legal stuff from this PR and removing/updating the stuff around StrideArrays.


Inconsistent reported active GPU on Catalina 10.15.4

Further investigation needs to be done on why 10.15.4 is not respecting active gpu.

I believe this is not only a problem on 10.15.4 Catalina.
In #54 I reported that Unity kept using the dGPU after disconnecting an external display, even though gSwitch said it switched back to iGPU only and that Unity was "hungry".
That was on Mojave, and I've just done some more tests on both Mojave and 10.15.4 Catalina with the internal display only, and gSwitch set to iGPU only.
About This Mac shows only the iGPU, but launching Unity (or an app built by Unity) shows "Yes" for "Graphics Card" in Activity Monitor, and the GPU history graph shows activity on the dGPU, even though gSwitch still says Unity is only "hungry". This behaviour is identical in Mojave and Catalina.
I don't think it's specific to Unity, that's just the only app I had at hand that I know will use the dGPU if it can.
I'm using a 2019 MBP 15" with Radeon Pro 560X dGPU.

@CodySchrank Is this fixed, or invalid?


[TikTok] Invalid URL

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2022.11.11 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and I'm willing to share it if required

Region
France
Provide a description that is worded well enough to be understood
I've had trouble downloading TikTok links saying "The read operation timed out". While searching for issues about this i came across #2396 which had a "fix" of spoofing the user agent, which i did. I'm running programmatically and changed the user agent using yt_dlp.utils.std_headers["User-Agent"] = "facebookexternalhit/1.1". This worked fine for a while, and now i'm getting a new error that I can't seem to find a fix for.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['--user-agent', 'facebookexternalhit/1.1', '-vU', 'https://vm.tiktok.com/ZTR45GpSF/']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (zip)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-56-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2020.06.20, pyxattr-0.7.2, secretstorage-3.3.1, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vm.tiktok] Extracting URL: https://vm.tiktok.com/ZTR45GpSF/
[vm.tiktok] ZTR45GpSF: Downloading webpage
[debug] [generic] Extracting URL: https://www.tiktok.com/@/video/7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG%2B60x%2FlLtJURz%2FRy5GL7fxK7hqI5ZC%2FRnX9lm6YXtN4ajMomGgA%3D&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy
[generic]<PHONE_NUMBER>794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG+60x/lLtJURz/Ry5GL7fxK7hqI5ZC/RnX9lm6YXtN4ajMomGgA=&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic]<PHONE_NUMBER>794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG+60x/lLtJURz/Ry5GL7fxK7hqI5ZC/RnX9lm6YXtN4ajMomGgA=&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy: Extracting information
[debug] Looking for embeds
ERROR: Unsupported URL: https://www.tiktok.com/@/video/7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG%2B60x%2FlLtJURz%2FRy5GL7fxK7hqI5ZC%2FRnX9lm6YXtN4ajMomGgA%3D&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy
Traceback (most recent call last):
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1485, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1561, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 674, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/generic.py", line 2721, in _real_extract
    raise UnsupportedError(url)
yt_dlp.utils.UnsupportedError: Unsupported URL: https://www.tiktok.com/@/video/7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG%2B60x%2FlLtJURz%2FRy5GL7fxK7hqI5ZC%2FRnX9lm6YXtN4ajMomGgA%3D&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy

Just in case, this is the log without the user agent change:
[debug] Command-line config: ['-vU', 'https://vm.tiktok.com/ZTR45GpSF/']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (zip)
[debug] Python 3.10.6 (CPython x86_64 64bit) - Linux-5.15.0-56-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.11.0, certifi-2020.06.20, pyxattr-0.7.2, secretstorage-3.3.1, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vm.tiktok] Extracting URL: https://vm.tiktok.com/ZTR45GpSF/
[vm.tiktok] ZTR45GpSF: Downloading webpage
[debug] [generic] Extracting URL: https://www.tiktok.com/@/video/7106798200794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG%2B60x%2FlLtJURz%2FRy5GL7fxK7hqI5ZC%2FRnX9lm6YXtN4ajMomGgA%3D&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy
[generic]<PHONE_NUMBER>794926362?_r=1&_d=secCgYIASAHKAESPgo8A2L6gE3aID26hWdYMUo7Ots2KAQqRwjDG+60x/lLtJURz/Ry5GL7fxK7hqI5ZC/RnX9lm6YXtN4ajMomGgA=&checksum=3c957febf57e591b7143c12855188c90bf89ee566ffa421f0f00fa65f8a6f57f&language=en&preview_pb=0&sec_user_id=MS4wLjABAAAAYXeo2peZSIayb4qlfmzwZA95CealXeyCT-jDYv3fvc9rw9lWjNVvzV0hz4ZZKbCP&share_app_id=1233&share_item_id=7106798200794926362&share_link_id=EA88E06F-AA6C-49C2-A526-62165636810E&source=h5_m&timestamp=1670158079&tt_from=copy&u_code=dga01g482c3bbi&user_id=6912637862613828613&utm_campaign=client_share&utm_medium=ios&utm_source=copy: Downloading webpage
ERROR: [generic] Unable to download webpage: The read operation timed out (caused by TimeoutError('The read operation timed out')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 674, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/generic.py", line 2582, in _real_extract
    full_response = self._request_webpage(url, video_id, headers={
  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 825, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 807, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3692, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.10/urllib/request.py", line 519, in open
    response = self._open(req, data)
  File "/usr/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/bin/yt-dlp/yt_dlp/utils.py", line 1527, in https_open
    return self.do_open(
  File "/usr/lib/python3.10/urllib/request.py", line 1352, in do_open
    r = h.getresponse()
  File "/usr/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.10/ssl.py", line 1274, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.10/ssl.py", line 1130, in read
    return self._sslobj.read(len, buffer)
TimeoutError: The read operation timed out

TikTok has changed the redirect target locations of the vm.tiktok.com and vt.tiktok.com shortlinks to completely omit the user_id, e.g.:
https://www.tiktok.com/@/video/7106798200794926362
instead of
https://www.tiktok.com/@user_id/video/7106798200794926362
This URL format does not match TikTokIE's _VALID_URL pattern, so the returned url_result falls back to the generic extractor. And while I can't reproduce this, some users clearly aren't able to download www.tiktok.com webpages at all without using the facebook bot user-agent, which is why OP's second example fails. The first example succeeds at downloading the webpage with the fb-bot UA, but the generic extractor doesn't know how to do anything with the data.
I'll have a PR up in a bit to fix this


Strange Results From W2A8 Model

Hi,
I executed the W2A8 ImageNet finetuing using the script by directly use n_bits_w = 2, n_bits_a = 8.
But it produces unexpected results in the W2A8 setting. Could you please advise if there are any specific hyperparameters or configurations that need adjustment in the default code to address this problem?
Here are my results.

I also have the same problem.

@ThisisBillhe Sorry to bother, but I still can not reproduce the result of W2A8 setting, is there any way to fix this?

Hi, I will look into this when I am available..working on other project now
BTW, have you successfully trained w4a8 or w4a4 model with EfficientDM? Just want to make sure you setup the project correctly.

Hi, I will look into this when I am available..working on other project now BTW, have you successfully trained w4a8 or w4a4 model with EfficientDM? Just want to make sure you setup the project correctly.

Thanks for reply so soon! Yes, I successfully trained the w4a4 model and the results look good. But for W2A8, I really dont know why the result is that strange.

what about using more steps and epochs during training, e.g., 250 ddim_steps and more epochs?

what about using more steps and epochs during training, e.g., 250 ddim_steps and more epochs?

The best result I got is by directly training 20 ddim_steps with 800 epoch, and I got FID 21 and sFID 12, still far away from the paper results. When I double the training epoch, the model training crashes at  1200 epoch. I must doing something wrong, any idea how to reproduce the paper result?