djb2 rule fails with Revil sample

Summary
Revil ransomware uses the djb2 hash algorithm, but the djb2 rule does not detect it. May be the variant because the compiler generates different code.
Examples

Sample hash: 9B11711EFED24B3C6723521A7D7EB4A52E4914DB7420E278AA36E727459D59DD (found in the Kaseya incident)
File offset: 0x6555
Disassembly code:

Pseudo-code:

Possible improvements
I tried to clone a new rule and replace shl with:
- description: hash * 33
- mnemonic: imul
- number: 33

Thanks team for great tool!

@cdong1012 want to take this?

Good suggestion, adding a basic block statement per your suggestion should better capture this.

This was added in #656! Thanks to @Still34!
hash data using djb2 (2 matches)
namespace   data-manipulation/hashing/djb2
author<EMAIL_ADDRESS><EMAIL_ADDRESS>scope       function
mbc         Data::Non-Cryptographic Hash [C0030]
references  https://twitter.com/r3c0nst/status/1392405576131436546, http://www.cse.yorku.ca/~oz/hash.html
function @ 0x2C96555
  and:
    instruction:
      and:
        mnemonic: mov @ 0x2C9655B
        number: 0x1505 @ 0x2C9655B
    or:
      instruction:
        and:
          number: 0x21 @ 0x2C96562
          or:
            mnemonic: imul @ 0x2C96562
function @ 0x2C96573
  and:
    instruction:
      and:
        mnemonic: mov @ 0x2C96579
        number: 0x1505 @ 0x2C96579
    or:
      instruction:
        and:
          number: 0x21 @ 0x2C96580
          or:
            mnemonic: imul @ 0x2C96580


Add detailes about Segment sending logic.

Focus on how to make this logic reliable and observable while also require very little code to implement.

Overall LGTM, sending data with duplication prevention in mind sounds good. any plans on how will we test this?

I guess we can look for duplicates in Amplitude once we can some data in there, but with buth Segment and Amplitude being proprietary SAAS systems, there is no much testing we can do around them.


Create a Docker Image

It would be great if there is a Dockerfile and the user can just build a Docker image and use the platform locally.

Well for using the platform locally, there is already the java tools providing many more functionality. I don't see any point of using the "online" tools locally. Moreover, we have already built an API so the user won't even need to install anything.

fully agree, my bad :sweat_smile: , should i close this then?

Done.


Temporal Table PK Change

Azure Data Studio Version:

Steps to Reproduce:
1.Create a temporal table that utilizes a non-clustered primary key
2.Deploy the table to the database.
3.Change the primary key to clustered
4.Deploy the new primary key definition to the database.
Results - dacpac script attempts to drop the existing primary key without first turning off system versioning. This obviously causes the deployment to fail. There is no work-around for this in pre or post deploy scripts.

Needs to be fixed in DacFx. @ssreerama do you know if this was fixed with the temporal table fix you recently made?
@philnoakes which version of ADS are you using?

I'm using 1.28. But the sqlpackage version i'm using is showing 14.0.4127.1

are you using sqlpackage to generate the script, not the dacpac wizard in ADS? Please use the latest version of sqlpackage if you are using sqlpackage.

@kisantia , The fix that we recently made is to support the temporal table drop, I don't think of this scenario that came under the previous fix.
@philnoakes, please update your issue status by using the latest sqlpackage, If the bug still exists, we can look into it! Thanks.

please reopen this issue if there are more details that can be provided


Migrate to Scala 3

Is it possible to use the code generation with Scala 3.x and ZIO 2.x?
I tried to do this locally and had issues with:
[error] (potatoes / update) sbt.librarymanagement.ResolveException: Error downloading com.github.ghostdogpr:caliban-tools_3:2.0.0-RC2
[error]   Not found
[error]   Not found
[error]   not found: /Users/jeffmay/.ivy2/localcom.github.ghostdogpr/caliban-tools_3/2.0.0-RC2/ivys/ivy.xml
[error]   not found: https://repo1.maven.org/maven2/com/github/ghostdogpr/caliban-tools_3/2.0.0-RC2/caliban-tools_3-2.0.0-RC2.pom

It looks like the caliban-tools project is not compiled to Scala 3, and I'm not sure how the dependency is being added (otherwise I would attempt to use the Scala 2.13 compiled version).
Is this dependent on fixing this issue? It looks like a potential fix has been sitting as a pull request since the beginning of this year. Is there any work around? Or does this PR need to be merged first?
Thanks!

Hey,
Sorry for the late reply.

Is it possible to use the code generation with Scala 3.x and ZIO 2.x?

Not yet but someone is actively working on it here: https://github.com/ghostdogpr/caliban/pull/1415
I'm currently trying to migrate this POC to ZIO2 here: https://github.com/guizmaii/poc_compile_time_caliban_client_generation/pull/155
When it'll be done, I'll migrate it to Scala3 in another PR ;)

FYI @jeffmay , it has been fixed in v2.0.1 of Caliban: https://github.com/ghostdogpr/caliban/releases/tag/v2.0.1


add changed date to index

I hope this is not a stupid pull request :)
Bit as it seems the published date remains the same after the very first publish. Published changes are reflected in the changed_date. In our case this value is far more interesting and i thought we might not be the only ones.
Thanks for your great work.

Thanks!
I can assure you it's not a stupid pull request :)
My only concern is if it's really necessary mostly because this would require all projects that use this with solr and upgrade to the new version to update their schema.xml file which makes the upgrade not so straightforward. So I'm not sure if it's worth the hassle for all users when upgrading.
Usually additions like this I do at the project level with the base index functionality.

@czpython Could you give me a hint in the correct direction how to do that at project level... I am still a learner in the django world and would highly appreciate it :)

@SachaMPS your guess is correct :)
I usually create an app called search on my project.
Then inside this app I create a base index in a file called base.py.
Inside I define my base index like so:
from aldryn_search.base import AldrynIndexBase

class BaseIndex(AldrynIndexBase):
    # following your example
    changed_date = indexes.DateTimeField(null=True)

then I can tell aldryn search to use my class as the new index base class like so (on the settings.py):
ALDRYN_SEARCH_INDEX_BASE_CLASS = 'search.base.BaseIndex'
Now that all your indexes have this field, you can add the prepare_changed_date method.
For indexes that come from third party apps like newsblog or cms page (coming from this app),
you can easily extend them to have the prepare_changed_date method.
Here's an example of how you can extend the cms page index from this app in your project.
On your search app from above, create a file called search_indexes.py.
Inside the file add the following:
from aldryn_search.search_indexes import TitleIndex as BaseTitleIndex

class TitleIndex(BaseTitleIndex):
    def prepare_changed_date(self, obj):
         return obj.page.changed_date

Last, in yout settings.py add the following:
ALDRYN_SEARCH_CMS_PAGE = False  # disabled to add custom field values
The above setting tells aldryn search to not register it's index for the cms, thus allowing you to register your own which inherits from the one in aldryn search.

@czpython Thank you very much. I already stumbled in my head around the issue that aldryn search registers its index. That it simply can be deactivated as shown didnt pop into my eyes... Got it!!! Thanks!!!

@SachaMPS my pleasure.
I'll close the pr for the reasons above.
Thank you very much for your contribution.


Training AI model

Story1: Create a framework for Text-2-Text interaction with the OpenAI
Story2: Create a framework for Speech-2-Speech interaction with the OpenAI
Story3: Summarize the interaction between bot and user
Story4: Train the model with sample data for proper interaction

Code sample is ready need to be integrated with Fast API


[WIP] Atari Actor Critic

Resolves #166
Changes:

Added a CNNActorCritic with a convolutional feature layer and separate policy and value functions within
Refactored PPO1 to work with ActorCritic instead of policy_new and value_fn
Modified get_env_properties to return input_dim instead of state_dim which will be framestack in the case of Atari envs and the state space shape everywhere else.
Atari support enabled for DQN, A2C and PPO1

Problems:

A2C and PPO1 are giving out nans for rewards every 2-3 episodes. These don't train at all as of now.

Added Atari support for PPO1, A2C and VPG. None of them seem to train well. Will track this in #185 instead after merging this. There might be an issue with VecNormalize. Will sort after #183

Ready to merge @Sharad24 @ajaysub110

Will merge this, but can you add an atari example in the documentation?
And anything else you've added.
On Sat, 18 Jul 2020, 01:39 Sampreet<EMAIL_ADDRESS>wrote:

Ready to merge @Sharad24 https://github.com/Sharad24 @ajaysub110
https://github.com/ajaysub110
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/SforAiDl/genrl/pull/175#issuecomment-660315478, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AH72FJ7HVBJCWWBVFIZZFQDR4CVZNANCNFSM4N74ZOWA
.

Will merge this, but can you add an atari example in the documentation?

And anything else you've added.

Yeah sure. Should I add it here itself?

And anything else you've added

Nothing major other than the Atari functionality. We can added docs explaining how to add new policies.

LGTM alerts?

Lol

Now? Removed the unused import, the other alert can be ignored.


php 8 and composer

Hello,
I installed php 8 and when I run composer on your application I have an error nd do accept to make an update (or look) but you app seems to work fine in php 8.0

#46

https://github.com/DarkGhostHunter/Preloader/commit/b98849c7a2d54a0e60dcf7b9aaa08ad40b74d01c


Add combinations_n

itertools already provides combinations which returns all choices of 2 items. It Seemed odd that it doesn't provide a more general form.
combinations_n returns all possible choices of n items from a given iterator.
Perhaps this implementation isn't a good fit for the library. Up to you to decide.
A few issues:

I've implemented lazy buffering. combinations_n only consumes data from the iterator when it has to. However, I don't know if the implementation is good enough, or if I placed it in the right file.
I couldn't manage to set the Clone trait correctly on the new structs (due to being new to Rust), so I removed it.

Thanks

The whole lazy buffer thing should mean we don't need to clone the iterator at all? I think Clone on the iterator can go away everywhere.

I can squish the commits into one once it's ready to merge (or you can do it).


Fix bug preventing new sections from merging into persisted phases

Fixes #1853
Changes proposed in this PR:

Fixed a bug that prevented new sections on customised template from being merged onto newly created template in UpgradeCustomizationService (this was caused by trying to save a Section with Questions. The presence validation on the Question model was preventing this)

failures in GTravis look like they were fixed in Sam's latest PR


Ticket Comment Categories has no pagenation

There is no option to see the next page in ticket comment categories. the default view is 10 items

will complete in 1.2


⚠ Minecraft has degraded performance

In 82a94ae, Minecraft (https://www.minecraft.net) experienced degraded performance:

HTTP code: 200
Response time: 1067 ms

Resolved: Minecraft performance has improved in a5bff93.


types(runtime-dom): add missing img attr loading

https://developer.mozilla.org/zh-CN/docs/Web/API/HTMLImageElement/loading

Hi, @antfu @yyx990803
Since this PR is unresponsive for a long time. I noticed that you are a recent committer for this project. May I ask do you mind applying this changes to the project?

Hello, this type of definition has any problems?

/ecosystem-ci run


During API Specification Design time, Spectral Rules are not showing up under problems in IDEs/JetRider

Hi ,
Greetings for today.
I am trying to use spectral plugin in IntelliJ IDE and also in JetRider. I added the plugin though the market place.
I have added the spectral rule set ( with fully qualified path).

and within the .spectral.yaml file, I have ruleset defined in gitlab.

main openapi.yaml has other reference yaml files under examples, headers, models, parameters, responses. and referred in openapi.yaml.
I would expect that once spectral plugin setting is done, all linting error should come under problems on API specification typing/save. but it does not come.  Is there something else needed to view the linting errors. This problem is same in Intellij and JetRider.
Thanks for your response.

From my observations....this is a problem with "Included path patterns" config param....if you put "/*openapi.yaml" a first path the .yaml file will be linted....but .json and .yml will not.
workaround: put "/*{openapi,swagger}.{json,yml,yaml} in "Included path patterns" param

I have the same problem, with the IntelliJ Plugin. The settings are as follows, but there are not error showing up within the IDE itself. Everything else works fine, when you run the CLI, the error are showing properly, but unfortunately this plugins doesn't do it's job.


add in checking of our dusty specs in dusty repo

review @jsingle

Looks good if the tests pass :boom:

@paetling Isn't this going to make it impossible to ship backwards-incompatible changes to the specs? You try to change Dusty, but it fails because the old specs don't validate against new Dusty, so then you try to change the specs, but that fails because the new specs don't validate against old Dusty...


over use of "automatically"

Issue in ./help/edge/data-collection/adobe-analytics/analytics-overview.md
"The Adobe Experience Platform Edge Network automatically maps many XDM variables automatically. The complete list of automatically mapped variables is listed here."
Too many "automatically". May be rephrase to: "The Adobe Experience Platform Edge Network maps many XDM variables automatically. The complete list of these variables is listed here."

Thank you for reporting this. We will work to get the documentation updated.
Internally tracking in https://jira.corp.adobe.com/browse/PLAT-86583

This issue has been fixed. Thank you.


[Relay] Check if the attribute "name" exists before accessing it

I found that some models fail to compile with tvm when the target is: compute-library, llvm -mtriple=aarch64-linux-gnu -mattr=+v8.2a,+neon
The error is:  AttributeError: <class 'tvm.ir.expr.GlobalVar'> has no attribute name.
This patch checks if the attribute name exists before accessing it, and solves the compilation issues that I had, producing the expected result during the following inference.

Thanks @NicolaLancellotti!


updated template

Pull Request Test Coverage Report for Build<PHONE_NUMBER>2
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 57.948%

Totals

Change from base Build<PHONE_NUMBER>4:
0.0%

Covered Lines:
2993

Relevant Lines:
5165

💛  - Coveralls


Update haskell.nix to get latest ghcjs emcc fixes

This is based on the PR input-output-hk/haskell.nix#1064.
emcc is the emscriten C to JS compiler used to by ghcjs when running hsc2hs because so that constants in C code compiled with emcc will match constants in the code generated by ghcjs.
Hydra jobset

Nice. I looked at the Enum fix and it seems generally good. So if we update and review #109 first, then we can merge this.


[ENHANCEMENT] Covariate shift conformal

As described in this paper :
https://arxiv.org/abs/1904.06019

Hi, I recently implemented this paper as part of a project and would love to discuss how it could be added to MAPIE.

Hi @RudrakshTuwani , thanks for your interest in MAPIE ! Maybe the cleanest way to do this would be to

share a github repo with your implementation so that we can give you feedbacks
init a pull request on MAPIE, making a separate class of your own in a dedicated module, following the template provided by the regression module
Don't forget to write unit tests so as to maintain 100% coverage and to write a pedagogical example

Awesome, thanks for the guidance. I will keep you posted!

Any news about this @RudrakshTuwani ?

Hey @gmartinonQM, I had implemented the density ratio estimation class and a basic API for weighted conformal before life got in the way. Is it possible for us to meet virtually next week? I have some high-level questions and I feel like we can progress faster that way.

Hello @gmartinonQM, I have some good news! I have implemented the split conformal variant in my fork here: https://github.com/RudrakshTuwani/MAPIE?organization=RudrakshTuwani&organization=RudrakshTuwani
Changes in files:

mapie/regression.py - Added class MapieCovShiftRegressor.
mapie/dre.py - Added classes DensityRatioEstimator and ProbClassificationDRE.
mapie/utils.py - Added a function for calculating weighted empirical quantile.

The replication of paper results is at examples/regression/4-covariate-shift/paper_replication.ipynb. There are a couple of differences but they are mostly due to different model defaults between Python and R. In general, we see that MapieCovShiftRegressor is able to adapt to covariate shift with oracle as well as estimated density ratios.
There's still work to be done. Right now I have just limited the functionality of MapieCovShiftRegressor to cv="prefit" and method="base". I can try working on adapting the class to other scenarios. It will likely be non-trivial and I think I would need to spend some time thinking about how to do it in a way that doesn't violate exchangeability. If you feel the current limitations are not significant, I can also spend time on writing tests and documentation.
Let me know what you think! Thank you :)

Hi @RudrakshTuwani , thanks for your contribution ! Could you create a pull request of your fork toward master ?

Tested this out of curiosity. Does not work anymore unfortunately, I think some fitting parameters have to be added to the model since it checks for residuals_

Hi @RudrakshTuwani, thanks again for your great work and sorry about our silence over the past few weeks.
You may have noticed that we implemented a new type of class ConformityScore that allows the user to define "custom" conformity scores (instead of residuals). Following this update, I adapted your implementation of the covariate shift method so the method can directly be used in MapieRegressor through a CovariateShiftConformityScore class. I pushed a branch here : https://github.com/scikit-learn-contrib/MAPIE/tree/add-covshift-in-conformityscore. The branch includes the modified files and your notebook adapted to my suggested modifications (I obtain the same results).
Please let me know what you'd like us to do. If you have time in the following weeks, I would be happy to help you finishing the pull request you started by considering the suggested modifications (and adding some unit tests to make sure everything is tested and controlled). If you don't have time, I can also keep working on your PR  and add you as a reviewer.
Thank you once again !

Hey, I apologize for the radio silence on this!
What's the cleanest way to do this? Should I just clone @vtaquet's branch and add a few tests? Thanks!

Hi @RudrakshTuwani ! Yes, you can clone my branch, double-check it and add the unit tests to keep the 100% test coverage. Please let me know if you have some questions about it. Thanks again !


Performance improvement investigation

Investigation
It's required to better understand, in terms of performance, the AWS Lambda Custom Runtime with Swift.
It's required to provide solution to reduce the Initalization time and the Overhead time that you can monitor when X-Ray is enabled.

It could be helpful if you can comment this with ideas.

I believe the init stage depends on how big the lambda layer is as it has to pull it from s3, can try stripping stuff from the runtime.

@ballwood Yes, I agree, it's the reason I want to have the build of Swift on amazonlinux #5 #6. This could reduce the size of it. The second time you launch the lambda this is zero.

What is quite obscure, it is the Overhead meaning.

might be overhead from x-ray whilst it sends the profiling information?

The investigation it's closed.
Read the results here:
https://medium.com/better-programming/swift-on-aws-lambda-performs-like-node-and-python-using-swift-sprinter-d920521130f1


scss not compiling

ERROR in ./src/app.vue?vue&type=style&index=0&id=5ef48958&scoped=true&lang=scss& (./node_modules/vue-loader/lib??ref--2!./node_modules/vue-loader/lib??vue-loader-options!./src/app.vue?vue&type=style&index=0&id=5ef48958&scoped=true&lang=scss&)
Module build failed (from ./node_modules/vue-loader/lib/index.js):
TypeError: Cannot read property 'content' of undefined
    at selectBlock (C:\Users\pc\Desktop\play\essentialfootball\node_modules\vue-loader\lib\select.js:41:13)
    at Object.module.exports (C:\Users\pc\Desktop\play\essentialfootball\node_modules\vue-loader\lib\index.js:79:12)
 @ ./src/app.vue?vue&type=style&index=0&id=5ef48958&scoped=true&lang=scss& 1:0-196 1:212-215 1:217-410 1:217-410
 @ ./src/app.vue
 @ ./src/app.ts

i could not compile scss styles, but I solved by changing how the vue style loader was loaded:
      {
        test: /\.scss$/,
        use: [
          'vue-style-loader',
          'css-loader',
          'sass-loader'
        ]
      }

Yes, I just found the same issue and solution as you. Seems like that option part is not entirely correct:
test: /\.vue$/,
loader: 'vue-loader',
options: {
    loaders: {
        'scss': 'vue-style-loader!css-loader!sass-loader',
        'sass': 'vue-style-loader!css-loader!sass-loader?indentedSyntax',
    }
}```

and this should be replaced by the code found by @Ymx1ZQ (at least for the SCSS part).

Checking the official vue-loader page I found that the `loader(s)` option is now deprecated (not sure if `loader == loaders`, or a typo? (https://vue-loader.vuejs.org/migrating.html#options-deprecation)
In any case: I can't find any documentation for the `loaders` option anymore on the official page: https://vue-loader.vuejs.org/options.html.

And it seems in the current vue-loader (V15) you need to explicitly specify the loader chain for the styles (https://vue-loader.vuejs.org/migrating.html#loader-inference)


Transfer component

Feature request type
New component
Component name
MudTransfer
Is your feature request related to a problem?
No response
Describe the solution you'd like

Probably before doing this component there is another task - implement multi-select on the List component
Have you seen this feature anywhere else?
No response
Describe alternatives you've considered
No response
Pull Request

[X] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Good news. We have MudTransferList extension component now.
Ready on NuGet: https://www.nuget.org/packages/CodeBeam.MudExtensions/6.2.7
Try online: https://codebeam-mudextensions.pages.dev/mudtransferlist
https://user-images.githubusercontent.com/78308169/220634541-078285bf-14ca-4dfe-a68f-017a0b71a546.mp4
@MudBlazor/core-team if you think we do not bring this component to core library (i think its not strictly required, its a list extension) we can close this issue as solved with extension.

Also see MudDropZone which is more natural to use anyway.


[+0-all] Now that enough of SILGen has been updated that I can write …

…basic tests, add a simple initial test.
Before, I would hit errors immediately. This tests the following commits:
e003366f1f [silgen] Ensure that when we emit the return statement, we always have a plus one value afterwards.
04056ce243 [silgen] When emitting convertBridgeObjectToRef, forward ownership through the instruction rather than unconditionally adding a cleanup.
rdar://34222540

@swift-ci test and merge


Extra line break removed from **Assembly and **Version

@v-hearya,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

patterns-practices/reference/addmodule-mthd-moduleinfogroup-str-type-str.md
:white_check_mark:Succeeded

patterns-practices/reference/addmodule-mthd-moduleinfogroup-type-str.md
:white_check_mark:Succeeded

patterns-practices/reference/addmodule-mthd-str-str-initializationmode-str.md
:white_check_mark:Succeeded

patterns-practices/reference/addmodule-mthd-str-str-str-initializationmode-str.md
:white_check_mark:Succeeded

patterns-practices/reference/addmodule-mthd-str-str-str.md
:white_check_mark:Succeeded

patterns-practices/reference/addmodule-mthd-type-initializationmode-str.md
:white_check_mark:Succeeded

patterns-practices/reference/addmodule-mthd-type-str.md
:white_check_mark:Succeeded

patterns-practices/reference/allactiveregion-activeviews-property-mspp-regions.md
:white_check_mark:Succeeded

patterns-practices/reference/allactiveregion-class-mspp-regions.md
:white_check_mark:Succeeded

patterns-practices/reference/allactiveregion-constructor-mspp-regions.md
:white_check_mark:Succeeded

patterns-practices/reference/allactiveregion-deactivate-method-mspp-regions.md
:white_check_mark:Succeeded

patterns-practices/reference/assemblyresolver-class-mspp-modularity.md
:white_check_mark:Succeeded

patterns-practices/reference/assemblyresolver-constructor-mspp-modularity.md
:white_check_mark:Succeeded

patterns-practices/reference/assemblyresolver-dispose-method-boolean-mspp-modularity.md
:white_check_mark:Succeeded

patterns-practices/reference/assemblyresolver-dispose-method-mspp-modularity.md
:white_check_mark:Succeeded

patterns-practices/reference/assemblyresolver-loadassemblyfrom-method-mspp-modularity.md
:white_check_mark:Succeeded

patterns-practices/reference/autopopulateregionbehavior-addviewintoregion-method-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/autopopulateregionbehavior-behaviorkey-field-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/autopopulateregionbehavior-class-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/autopopulateregionbehavior-constructor-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/autopopulateregionbehavior-createviewstoautopopulate-method-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/autopopulateregionbehavior-onattach-method-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/autopopulateregionbehavior-onviewregistered-method-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/backgroundeventsubscription-tpayload-class-mspp-pubsubevents.md
:white_check_mark:Succeeded

patterns-practices/reference/backgroundeventsubscription-tpayload-constructor-mspp-pubsubevents.md
:white_check_mark:Succeeded

patterns-practices/reference/backgroundeventsubscription-tpayload-invokeaction-method-mspp-pubsubevents.md
:white_check_mark:Succeeded

patterns-practices/reference/bindablebase-class-mspp-mvvm.md
:white_check_mark:Succeeded

patterns-practices/reference/bindablebase-constructor-mspp-mvvm.md
:white_check_mark:Succeeded

patterns-practices/reference/bindablebase-onpropertychanged-method-string-mspp-mvvm.md
:white_check_mark:Succeeded

patterns-practices/reference/bindablebase-onpropertychanged-t-method-expression-func-t-mspp-mvvm.md
:white_check_mark:Succeeded

patterns-practices/reference/bindablebase-propertychanged-event-mspp-mvvm.md
:white_check_mark:Succeeded

patterns-practices/reference/bindablebase-setproperty-t-method-mspp-mvvm.md
:white_check_mark:Succeeded

patterns-practices/reference/bindregioncontexttodependencyobjectbehavior-attach-method-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/bindregioncontexttodependencyobjectbehavior-behaviorkey-field-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/bindregioncontexttodependencyobjectbehavior-class-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/bindregioncontexttodependencyobjectbehavior-constructor-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/bindregioncontexttodependencyobjectbehavior-region-property-mspp-regions-behaviors.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-class-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-configuredefaultregionbehaviors-method-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-configuremodulecatalog-method-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-configureregionadaptermappings-method-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-configureservicelocator-method-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-constructor-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-createlogger-method-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-createmodulecatalog-method-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-createshell-method-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-initializemodules-method-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-initializeshell-method-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-logger-property-mspp.md
:white_check_mark:Succeeded

patterns-practices/reference/bootstrapper-modulecatalog-property-mspp.md
:white_check_mark:Succeeded

This comment lists only the first 50 files in the pull request.
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


⚠ Bofrost.gr has degraded performance

In 12279c0, Bofrost.gr (https://www.bofrost.gr) experienced degraded performance:

HTTP code: 200
Response time: 3983 ms

Resolved: Bofrost.gr performance has improved in 89bf6e5.


Remove duplicate common folders

The common folder was copied to https://github.com/dotnet/docs/tree/master/samples/snippets/common and also to each individual language. I think we should only keep one copy of it.

I agree.

@mairaw  Unless you have something more pressing for me to work on, I'd like to take on this issue. Since I've already deleted a few of these in other PRs, I familiar with the issue and how to proceed while making sure that the \common files are truly dupes.


[airgap rke2] custom rke2 cluster SUC system-agent-upgrader containers stuck in re-deploying/terminating state didn't match pod anti-affinity rules.

Rancher Server Setup

Rancher version: v2.6.3-rc4
Installation option (Docker install/Helm Chart): helm

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc):rke1

Proxy/Cert Details: valid letsencrypt certs

Information about the Cluster

Kubernetes version: 1.21.7+rke2r1
Cluster Type (Local/Downstream): downstream

If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider):custom

Describe the bug

custom cluster (single node all roles) SUC deployment is continuously deploying and terminating containers with image rancher/system-agent:v0.1.1-rc5-suc

To Reproduce

deploy an airgap HA setup
deploy a downstream rke2 cluster

Result

pods/containers are continuously restarting for system-agent-upgrader

Expected Result
Expecting there to be no restarts/reinstalling pods or workloads.

@slickwarren some questions:

does this only fail for RKE1 local cluster?
does it matter which version of rke2 the imported cluster is running?
does this same scenario also fail when not air-gapped?

@dweomer Not able to reproduce this in a non-airgapped scenario in Digital Ocean.

Deployed an RKE1 cluster (single-node)
Installed cert-manager and rancher (v2.6.3-rc4)
Created a custom cluster in Rancher (RKE2 v1.21.7-rke2r2)
Ran the registration command on the other Digital Ocean droplet.
Cluster comes up active. I see the system-upgrade-controller deployment installed on the downstream cluster briefly, but it is removed rather quickly.

log snippet shared with me by @thedadams ~ 12/10: log.txt
i noticed that the plan is constantly changing, flapping between two states. attached are the decoded values from the objectset.rio.cattle.io/applied annotation on the job(s) representing applied state:

1.txt
2.txt (did see some noise on stderr about decoding some info. think it was from base64 -d but possibly it was jq in the pipeline complaining)

the diff for these is:
--- 1.json	2021-12-09 14:13:57.707992369 -0700
+++ 2.json	2021-12-09 14:13:52.827980985 -0700
@@ -7,6 +7,7 @@
       "objectset.rio.cattle.io/owner-namespace": "cattle-system",
       "upgrade.cattle.io/ttl-seconds-after-finished": "900"
     },
+    "creationTimestamp": null,
     "labels": {
       "objectset.rio.cattle.io/hash": "5578292ae46c6979afa82eb77388959b0ddb938a",
       "plan.upgrade.cattle.io/system-agent-upgrader": "8a0b20eef98cf5f73ebdb2e10aff4508e1c86b2290af1feb6248a85f",

which represents a changing value for .spec.metadata.creationTimestamp which shouldn't cause the flapping.

@dweomer getting back to your questions:

This was tested in airgap with an rke1 local cluster
1.22 was not working for rke2 with this RC, we will ask for a new rancher RC today to verify this
This is not breaking for a non-airgap setup with the same version(s)

https://github.com/rancher/system-upgrade-controller/releases/tag/v0.8.1

@dweomer Thanks. I tested that version and it works for the setup I tested.

@cmurphy Can you update SUC images in system-charts and charts to v0.8.1?

Root cause
The root cause here was that SUC was continually deleting and recreating the upgrade job.
What was fixed, or what changes have occurred
Default values are added to some fields, so when Apply was used with these default values missing, Wrangler thought that the job was changing and needed to be deleted and recreated.
Areas or cases that should be tested
I was only able to reproduce this issue in air-gapped environment. So, the thing that should be tested here is the scenario listed in the description.
What areas could experience regressions?
There should be no regressions as long as the test case in the description passes.
Are the repro steps accurate/minimal?
Yes.

Available to test in v2.6.3-rc7

blocked from fully validating by https://github.com/rancher/rancher/issues/35485#issuecomment-997071182

Available to test in rancher/rancher v2.6.3-rc8

tested on v2.6.3-rc8 in an HA Airgap setup:

deploy rke2 downstream cluster (1.21.7)

SUC image is deployed and working properly -- pass

upgrade rke2 downstream cluster -- pass


Question: Does the user name and password work with MFA accounts?

Just watched the video, our environment is locked down with OKTA, would this still work? Also will it work in general with Microsoft MFA enabled?
Thanks.

Actually sorry, you can achieve this by using the native OAuth2 flow. It's a little more configuration
https://learning.getpostman.com/docs/postman/sending_api_requests/authorization/#oauth-20
You have to use the endpoints from your application in portal.azure.com to do this. Once you have the access token you can add it to your environment variables. This uses a UI to sign the user in.
The other way is to go into Graph Explorer, sign in, use developer console and type tokenPlease()

Thanks @jthake-msft. I wasn't going to be using this on the production environment, it was for our test environment which mimic's our production environment setup. Unfortunately I have no control of turning off MFA / OKTA in the test environment either. I'll check out the OAuth2 flow. Again thank you.

Re-opening as I'm going to add a stream lined flow for the OAuth2 Postman functionality.

@pmatthews05 I have written steps now on how to do this. Little bit more work to get going. Please give me any feedback here,
https://github.com/microsoftgraph/microsoftgraph-postman-collections/blob/master/README.md#setting-up-on-behalf-of-delegated-access

Hi @jthake-msft ,
Sorry it's taken me a couple of days to get round to this. I have tested your README.md and it is all correct apart from the callback URL should be "https://app.getpostman.com/oauth2/callback" not "https://www.getpostman.com/oauth2/callback" if you have followed Day 9 correctly.

Thanks for the tip. i'll change that.


add language-list

Adding language-list, which is a list of programming languages being developed on Github.

Thanks Thomas! I have modified your PR a bit and merged it in 0c7fd5827ec3372c687d9d56d228decb58fa8b51.

No worries @jnv!


An idea for increasing speed

Hi albertobsd!
There is an idea how to speed up the search for a given range in Keyhunt.
I had a hypothesis
Example: time ./keyhunt -m bsgs -t 6 -f tests/in.txt -r 49dccfd96dc5df56487436f5a1b18c4f5d340000000000000000000000000000:49dccfd96dc5df56487436f5a1b18c4f5d34ffffffffffffffffffffffffffff
In a sequential search, you will often find 4,5,6,7,8 ... of the same characters in succession.
I studied almost all private keys on which there were coins, none of which has 4 or more symbols in succession. There are exceptions when you manually create an address from a private key.
As in the English language, not words in which 4 identical letters go in succession.
An example of identical symbols:
49dccfd96dc5df56487436f5a1b18c4f5d34f65ddb48cb5aaaaa000000000000
49dccfd96dc5df56487436f5a1b18c4f5d34f65ddb48cb5e0000077770000000
49dccfd96dc5df56487436f5a1b18c4f5d34f65ddb48cb5e00000ffffff00000
If we assume that here the last a is equal to the month of the search
49dccfd96dc5df56487436f5a1b18c4f5d34f65ddb48cb5aaa**[a]**a000000000000
Then it turns out this month wasted.
If you remove all the same symbols, the speed will increase symmetrically.
49dccfd96dc5df56487436f5a1b18c4f5d34f65ddb48cb5eeee0000000000000 all keys after eeee are invalid.
(eeee0000000000000->eeeefffffffffffff)
Other examples:
FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFE BAAE DCE6 AF48 A03B BFD2 5E8C D036 4141 all keys after FFFF are invalid.
7777 FFFF FFFF FFFF FFFF FFFF FFFF  FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF all keys after 7777 are invalid.
1234 1111 FFFF FFFF FFFF FFFF FFFF  FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF all keys after 1111 are invalid.
1234 5111 1FFF FFFF FFFF FFFF FFFF  FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF FFFF all keys after 1111 are invalid.
What I suggest:
Search the private keys variable for identical symbols and replace them with +1.
Example: 49dccfd96dc5df56487436f5a1b18c4f5d34f65ddb48cb5aaaaa000000000000 change aaa to aab total search speed × 2.
49dccfd96dc5df56487436f5a1b18c4f5d34f65ddb48cb5aaaba000000000001
I know that an additional search for identical characters (prefixes) may reduce the speed by ~ 5-10%
The total travel time will be less by -30-50%.
I tried to implement this in LostCoins there is multithreading, GPU, I didn't have enough C++ knowledge.
You can output the range 1- ffffffff to a file and see how many times 3,4,5,6,7... characters were dropped in a row.
If you increase the range, the same ones will drop out symmetrically more often + intermediate ones. BitCoinCore randomizes private keys, there are almost no identical symbols in a row in random.
As far as I understand, the keyhunt algorithm creates bloom filters from hexadecimal data hashes for a private key. If we remove all incorrect values from the filter in which there are 4 or more identical symbols.
First, the filter will weigh less memory.
Secondly, the speed will increase.
Thank you for reading, I tried to write clearly.
It is interesting to hear your opinion

https://github.com/albertobsd/keyhunt/issues/115

Because of these identical 0-F characters, I prefer random search.
There are two options for how this can be done:

Scalar multiplication of vectors is difficult to implement, only masters can do it.

When creating a bloom filter, skip (not write to memory) all hashes with 4 or more identical symbols.

We will wait for an answer.

I have some work for this is not implemented yet, im testing it

When creating a bloom filter, skip (not write to memory) all hashes with 4 or more identical symbols.

The correct bloom filter creation is critical for the correct work of the BSGS algorimth i CAN'T skip any element in the bloom filter, the bloom filter will remain intact, for more information see https://andrea.corbellini.name/2015/06/08/elliptic-curve-cryptography-breaking-security-and-a-comparison-with-rsa/
As you said Direct scalar multiplication is in average 128 times slower than Point addition.
my idea for this approach with BSGS is the next: lets to suppose that BSGS can test some 48 bits in one single cycle  the range is 120bits example:
800000000000000000000000000000
XXXXXXXXXXXXXXXXXXYYYYYYYYYYYY

So the YY.. is the part that BSGS scan at good speed and the XX... part is the only part that can be accelerated or skipping values until match with some restriction.
Im using some deterministic approach for this only using values that match with some percentage of bits in 1 , so if you only want test values between [ 35%, %60 ] with bits in 1 instead of have<PHONE_NUMBER>00000000000000000000 as base range we are going to have<PHONE_NUMBER>0107f7ff000000000000 where the las zeros are going to be checked with BSGS from<PHONE_NUMBER>00  to ffffffffffff
Some list of the values to be use as BASE RANGE for BSGS is the next:
New range:<PHONE_NUMBER>0107f7ff000000000000
New range:<PHONE_NUMBER>0107fbff000000000000
New range:<PHONE_NUMBER>0107fdff000000000000
New range:<PHONE_NUMBER>0107feff000000000000
New range:<PHONE_NUMBER>0107ff7f000000000000
New range:<PHONE_NUMBER>0107ffbf000000000000
New range:<PHONE_NUMBER>0107ffdf000000000000
New range:<PHONE_NUMBER>0107ffef000000000000
New range:<PHONE_NUMBER>010bf7ff000000000000
New range:<PHONE_NUMBER>010bfbff000000000000

They may look some biased but is because the set of restrictions:

Average bits in 1 between 35%, %60
No two same byte value together example 80 01 01, the 01 is together to another 01 01, this value is omitted.
No three same  nibbles value together example 80001234, there are three 0 next to other 0 so that kind of values are omitted.

Yes those values can be selected at request, the good thing is that you can set the minimum  and maximum average values of bits  in 1 so if you don't found anything with  [ 35%, %60 ] you can change it to  [ 60%, %75 ] or any other value that you want and no one value is going to be repeated  un less you selected values like [50% , 60%] or something else overlapped range.
What you thing guys about this approach ?
Regards!

I have no knowledge of C++ and English, just a little logic.

i CAN'T skip any element in the bloom filter

If you cannot remove the excess from the filter, you can do something similar at the output, skip and not transform further.
Example:
S = start private key + out from bloom filter
string str[16] = "000, 111, 222, 333, 444, 555, 666, 777, 888, 999, aaa, bbb, ccc, ddd, eee, fff";
S.find (str, pos = 1)
if (pos == 1) {
string zero = "quick skiped..";
}else{
    Continuation of the transformation of the private key into the same, public key...
   for... while...
}

Primitively, it will skip 100 times faster than converting unnecessary values with the same characters to public.
XXXXXXXXXXXXXXXXXXYYYYYYYYYYYY
I agree xxx is more important to skip. Since x can be equal to the week (month, year) of the empty search
Initially, I thought to make a replacement, but this is not logical, it is better to skip

I don't understand the dotted algorithm.
Why make a big bloom filter?
For example, instead of a 64-bit filter, 256 GB
You can make 4 x 100 MB each with 16 bits 0000-FFFF and multiply them together. Will the speed be slow?

For puzzles
New range:<PHONE_NUMBER>0107ffbf000000000000
For skip...
8010101010101010100000000000000
80101010101010101a0000000000000
80101010101010101e0000000000000
80101010101010102a0000000000000
80101010101010102f00000000000000

Why make a big bloom filter?

Because is the way that the BSGS algorithm works, there is no other way, i've thinking in that for more than one year and there is no other way, if you now some other way please let me know.
Please READ: https://andrea.corbellini.name/2015/06/08/elliptic-curve-cryptography-breaking-security-and-a-comparison-with-rsa/ if you don't know English please use some web page to translate it to your language.

You can make 4 x 100 MB each with 16 bits 0000-FFFF and multiply them together. Will the speed be slow?

No, BSGS doesn't work in that way you need to have all the b Points in the Cache (Bloomfilter)

Thanks for the clarification, it is difficult to read the formulas for elliptic curves, I will study a little.
There is an idea to use video card memory instead of RAM.
For example, the 3090 has 24GB DDR6 of very fast memory, as well as very high bandwidth. It would be used :)))
I read that you don’t want to do what you don’t want to use. Let's hope that the programmer kanhavishva will show interest and implement bgbs on the GPU. I am learning from its sources.
Thank you and all the masters for your hard work and the fastest program.

I read that you don’t want to do what you don’t want to use.

I don't code for GPU because I don't have GPU, so why I going to code something if i will never use it? Also i need  access a GPU to test the code for GPU....  I don't have a desktop computer, I only have one old laptop from 5 years ago, i can't put a GPU there.

Let's hope fortune smiles at you or a donate has arrived for a new pc.
So far, only thanks if I find a puzzle from me 3090 :)

Speed question:
BGBS vs Rotor-Cuda
Let's take max BGBS rate
PC 1 Public key 256GB Speed ~ 1 Ekeys/s (1,000,000,000,000,000,000 private keys/s)
Servers 10TB Speed ~10 Zkeys/s
(10,000,000,000,000,000,000,000 private keys/s)
Take a Rotor-Cuda search at (ripmd160) GPU.
There are 30,000,000 addresses in the database.
GPU Speed (RTX 3090 1000Mk/s) ~ 1,000,000,000 private keys/s
1 address (ripmd160) has ~79228162514264337593543950336 private keys.
30,000,000 addresses =<PHONE_NUMBER>427930127806318510080000000 private keys.
Theoretically this equals a base with that many addresses.
We calculate the speed:
2376844875427930127806318510080000000 * 1,000,000,000 =
2,376,844,875,427,930,127,806 Ykeys/s
(2,376,844,875,427,930,127,806,318,510,080,000,000,000,000,000 private keys/s)
If you take the base of all coins (302,000,000 addresses) all.bin, the speed will be even higher ...
BGBS is only good for finding one puzzle 32-160 bit. This is the fastest solution to date.
If you are looking for a 160-bit puzzle, then it is better to do it with the address base. Since theoretically in 160 bits there can be all privat keys from all addresses from the base.
It would be cool if the BGBS algorithm searched the address base (ripmd160) with a speed exceeding the GPU.
Do you think I counted correctly?
I would like to hear your opinion on this.

Hi, yes rmd hashes has more probability to be found that is true.
Somo of your calculations are wrong.

It would be cool if the BGBS algorithm searched the address base (ripmd160) with a speed exceeding the GPU.

Yes, but it is not possible we cannot do any math operarion with rmd hashes.

At the expense of BGBS understood.
Tell me what is the error in the calculations?

It's clear about BGBS.
I agree on the account of more chances to find.
When searching by addresses (rimpd160), the address base hypothetically increases, which makes it more likely to find.
I am tormented by another question
1 address (ripmd160) has ~<PHONE_NUMBER>4264337593543950336 private keys
If you find another private key, it will have a different public key.
If the blockchain already has a public key and the one found does not match.
Does the system not authorize?
If the blockchain does not have a public key (address no out transactions), I suppose there will be no problems.
I suppose you do not know the answer, you must first find another private key and check it not in theory but in practice

It is called P2PKH ( Pay to Public Key Hash) if the hash match the it will works, doesn't matter if the publickey doesn't match

Examined your random code.
Tested it, it is correct for jigsaw puzzles.
Rand(min, max); randomite between ranges, does not go beyond the range.
Did you do it yourself?
I was looking for such a void for my project, I did not find it, everywhere only 0-F.
I made several options myself differently:
First option:
b1.Sub(&Start);
Int b2.Rand(&b1);
Int key.Set(&Start);
key.Add(&b2)

Option 2:
Int b1.Set(&End);
b1.Sub(&Start);
int bb = b1.GetBitLength();
Int b2.Rand(bb);
Int key.Set (&Start);
key.Add (&b2)

Third option:
Int b1.Set(&End);
b1.Sub(&Start);
int bb = b1.GetBitLength();
int br = 1 + rand ()% bb;
Int b2.Rand(bb);
Int key.Set(&Start);
key.Add(&b2)

I wanted to ask you how does random work in keyhunt?
How long is it updated?

Do I understand correctly that hash values are added to the bloom filter in this way?
000000000001
000000000002
...
00000000001a
...
000000ff77ff
...
0fffffffffff
(1000000000000)
or no zeros?
1
2
1a

The only way to speed up your puzzle search is to use GPU (s).
The GPU has many cores, very fast memory and high bandwidth.
From calculations, the expected speed is 1 public key in BSBG x1000 mode (RTX3090 x2000).
albertobsd I suggest you implement BGBS in Rotor-Cuda on GPU.
I know that you have a laptop without a GPU, so you have a smart head.
For work and tests, I will give you access through TeamViewer to my laptop 24/7.
I have Windows 10 x64, 2070, 64GB. 256m2, 1TB (if you need usb 6TB)
Make BGBS in Rotor-Cuda under Windows, it can then be compiled using makefile under linux.
The sources for keyhunt and Rotor-Cuda are almost the same.
Or you can implement your keyhunt on a GPU.
As you wish.
Implementing BGBS on a GPU is not an easy task.
It is advisable for you to join your efforts with the kanhavishva master.
He has already implemented almost all keyhunt modes in KeyHunt-Cuda (except BGBS) on GPU.

While I was thinking, but somehow already a month ago I implemented it :)))
BSGS-Cuda
Look at the source code, can it work for keyhunt (Rotor-Cuda)?
I'll test it in the evening, accomplish your goal of speed


[B-03566] sync single project based on project name

Disable sync on startup.
sync() also calls startProjectFileListeners() which happens on start.
(This behavior will no longer happen but may still need to be performed on start.)
startProjectFileListener() is now called for the specific project on sync(id) for a specific project.
The path /sync is removed and the path /sync/{id} is now expected.
Logs include project id or project name.

Coverage decreased (-0.5%) to 26.706% when pulling c0e43d307d2a22c025cc682f0217239b0ada641f on dame-sprint-oct-b03566-sync_service into 685bcb16545d46dbcf0cf5856e47824326790ebf on dame-sprint-oct-staging.


Error 'unexpected error creating API'

I am trying to add this integration for the first time.
I get the error 'Unexpected error adding the API'
I can log in on Polestar web site and app using the same credentials.
How can I best troubleshoot this?

maybe take a look at the logs and see what log you can see

Traceback (most recent call last):
File "/config/custom_components/polestar_api/config_flow.py", line 47, in _create_device
await device.init()
File "/config/custom_components/polestar_api/polestar_api.py", line 51, in init
'data': result['data']['getConsumerCarsV2'][0], 'timestamp': datetime.now()}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

i think duplicate error. https://github.com/leeyuentuen/polestar_api/issues/15
you don't have car in that account.
can you check if you have the full access on that account?

When I log in using my phone number, it work. 😊
But using my e-mail it does not work.
(On the web page and in the app both works)

When I log in using my phone number, it works. 😊
But using my e-mail does not work.
(On the web page and in the app both works)

When using version <1.2.0 (with the VIN setup form field) - it was working fine. Updating to the latest version doesn't break the app, but a clean install of 1.2.0 - gives the error below.

When using version <1.2.0 (with the VIN setup form field) - it was working fine. Updating to the latest version doesn't break the app, but a clean install of 1.2.0 - gives the error below.

@Borales  thanks for report, should be fix on next public code:
https://github.com/leeyuentuen/polestar_api/releases/tag/1.2.1

@leeyuentuen checked 1.2.1 - that solved the problem for me, thanks!

i'll close this ticket, for issue, open a new one with logs for better follow up the problem.


TrackUri encoding when setting is incorrect

The code below needs some work, can someone point out how it should be encoded?
From src/helpers/xmlhelper.ts
  static EncodeTrackUri(trackUri: string): string {
    if (trackUri.startsWith('http')) return encodeURI(trackUri);
    if (trackUri.startsWith('x-sonos-hta')) return trackUri;

    // Part below needs some work.
    const index = trackUri.indexOf(':') + 1;
    return trackUri.substr(0, index) + this.EncodeXml(trackUri.substr(index)).replace(/:/g, '%3a');
  }

From what I can tell, SoCo doesn't encode anything. It only changes "http:" or anything before the first colon to "x-rincon-mp3radio". I'm not sure I agree with that either. The EncodeXml seems reasonable, to ensure invalid xml is never sent. But it might be better to leave the other specific use case encodings (http, x-sonos-hta, ":") up to the caller.
I'm sure these cases ☝are in here for a specific historical reason (maybe spotify?) but they don't seem universally correct. For instance, the "http" one is just not correct anymore. I verified that Sonos doesn't know what to do with an http uri.
https://github.com/SoCo/SoCo/blob/527876acc234daba10525574f835238de1ad64a1/soco/core.py#L580


added key to assisted factory

added key to assisted factory
:recycle: Current situation & Problem
the key was missing in the factory, resulting in the same view model being returned for all types
:gear: Release Notes
added key
:pencil: Code of Conduct & Contributing Guidelines
By submitting creating this pull request, you agree to follow our Code of Conduct and Contributing Guidelines:

[x] I agree to follow the Code of Conduct and Contributing Guidelines.

@PSchmiedmayer we need your help with merging here. we can't reach the codecov in the PR because the UI tests don't count towards it


Does not work with Chrome 124

When I am trying to use this driver with the latest docker image selenium/standalone-chrome:124.0-chromedriver-124.0-20240505, the Selenium session never starts.

The problem is not with the Chrome 124 version, but with the Selenium 4 version used in the mentioned Docker image.
This driver is:

fully compatible with Selenium 2;
partially compatible with Selenium 3 (window switching and frame switching doesn't work);
not compatible with Selenium 4.

There is https://github.com/minkphp/webdriver-classic-driver driver, which works on Selenium 4. It's still in testing and hasn't been released, but you can try that.
If you don't care about the used Google Chrome version, then try using previous Docker images where Selenium 3 was used (if windows and frames support isn't an issue for you).


WFS examples don't work

Describe the bug
The WFS examples in the "examples" section do not work.
To Reproduce
Steps to reproduce the behavior:

Go to examples / WFS
Copy the code
Paste into index.html
open index.html in browser, or run npm start and go to localhost:1234
marvel at the error messages

Expected behavior
should give the result visible on the examples page

It would be nice if you could copy those error messages here.
CORS?


Next 13.4 App & Pages Routes co-usage

What version of this package are you using?
"next": "13.4.1",
"next-translate": "^2.0.5",

What operating system, Node.js, and npm version?
CSB: https://codesandbox.io/p/sandbox/funny-heisenberg-d5cwo6
What happened?
As per Next 13+, you can use legacy pages and new app router at the same time, allowing incremental adaptation. When next-translate is used on such project, with appDir: true in the next config, only app router pages are translated.
MRR: https://codesandbox.io/p/sandbox/funny-heisenberg-d5cwo6
See index page live in app dir, and second page live in the pages dir. i18n implementation is same for both, but only app router page picks up the translation.
This also confirmed by logger:
next-translate - compiled page: / - locale: en - namespaces: common - used loader: force-dynamic (server page)

But no logger record when you visit legacy router page.
App router page: https://d5cwo6-3000.csb.app/
Pages router page: https://d5cwo6-3000.csb.app/legacy-router
What did you expect to happen?
Both pages could be translated and have access to all configured namespaces.
Are you willing to submit a pull request to fix this bug?
Not sure of the cause.

I'm having this same issue.  I need this as soon as possible so I'm willing to give it a try on contributing to the solution. Note this would be my first time contributing to open source, so I need guidance.
From what I've been looking in this project, the solution could be to inject next-translate as a page by page basis in next-translate-plugin rather than a general isNext13App for all pages.
I'm going to explore a little more, but if @aralroca read this and is willing to put his own perspective I'd be appreciated.

It seems better to me that it is in the next-translate-plugin. I close this one and have it reported there better, thanks.


az resource create --location 'canadacentral' fails.

This is autogenerated. Please review and update as needed.
While running
az resource create --location 'canadacentral' --name 'Unique-Resource-Namebgw-g' --properties '{}' --resource-group 'UniqueResourceGroupName' --resource-type 'Microsoft.ClassicCompute/domainNames'
I get the following error:
Unable to create domain name 'Unique-Resource-Namebgw-g': 'The location constraint is not valid'.
Funny thing is that the creation of the Resource Group worked just fine:
az group create --location 'canadacentral' --name 'UniqueResourceGroupName'
>>
{
  "id": "/subscriptions/2b662f63-8485-47ac-aa45-92ba0b0e5b79/resourceGroups/UniqueResourceGroupName",
  "location": "canadacentral",
  "managedBy": null,
  "name": "UniqueResourceGroupName",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null,
  "type": "Microsoft.Resources/resourceGroups"
}

Command Name
az resource create
Errors:
Unable to create domain name 'Unique-Resource-Namebgw-g': 'The location constraint is not valid'.

To Reproduce:
Just by running the command above will trigger the error. Not sure what is going on but it works just fine for other regions like az resource create --location 'westus' --name 'Unique-Resource-Namebgw-g' --properties '{}' --resource-group 'UniqueResourceGroupName' --resource-type 'Microsoft.ClassicCompute/domainNames'

Put any pre-requisite steps here...
az resource create --location {} --name {} --properties {} --resource-group {} --resource-type {}

Expected Behavior
Environment Summary
Windows-10-10.0.18362-SP0
Python 3.6.6

azure-cli 2.1.0

Extensions:
interactive 0.4.3

Additional Context

Thanks for your feedback

Thank you for the rapid response! Anything that I can do as work around in the meantime?

add to S167.

@happyiphone Hi, I cannot reproduce this issue. Do you still have this problem?  If so, please give me some of your log information:

You can view the Activity log of the Resource Group on the portal after the resource creation fails. After clicking the failed operation name, there is a JSON information corresponding to the failure reason, then tell me what is the error message and correlationId of the failure?
If there is no obvious clue in step 1, Could you add --debug to the original command? And then please show me the information returned by debug.

It has been fixed in one of the latest releases (S166 I think). Thanks for checking!


Sous Chefs Adoption

Description
Describe what this change achieves
Issues Resolved
List any existing issues this PR resolves
Check List

[ ] All tests pass. See TESTING.md for details.
[ ] New functionality includes testing.
[ ] New functionality has been documented in the README if applicable.

@jeffbyrnes @bmhughes @Xorima Do we do a major release for Sous Chef adoption?

Minor normally


yaml file for MPT-1/3/7/13B with GBS=512 at https://www.mosaicml.com/blog/amd-mi250

Hi, at https://www.mosaicml.com/blog/amd-mi250, MPT-1B, MPT-3B, MPT-7B and MPT-13B with GBS=512  are used, also copy the figure below.

There are two changes in the yaml files mentioned in the webpage:

model.attn_config.attn_impl=flash
model.loss_fn=torch_crossentropy

and to support global batch size as 512, we also need to change for MPT-7B and MPT-13B:
3) change global_train_batch_size from 1024 to 512
Besides the above three changes, is there anything else we need to change? thanks.
For example, I suspect if we need to change the micro batch size (4, 8, 16, 32, 64) for a better performance, the current values of mbs are:
MPT-1B:  device_train_microbatch_size: 4
MPT-3B:  device_train_microbatch_size: 8
MPT-7B:   device_train_microbatch_size: 8
MPT-13B: device_train_microbatch_size: 8
Is there pipeline parallel, tensor parallel, or anything else? thanks.
It would be better if you can share the complete yaml files, thanks.

hi, I did some try on A100-40GB with 8 cards in a system, with the 3 changes mentioned above:

model.attn_config.attn_impl=flash
model.loss_fn=torch_crossentropy
global_train_batch_size=512

I saw Train throughput/device/flops_per_sec is 130--150 TFLOPS for MPT-1b/3b/7b, a big gap with 165-176 in the webpage table, and for MPT-13b, there's even OOM (out of memory).
So, could you share the exact yaml + code  (or a docker image with everything ready) to reproduce the same perf data in the webpage at https://www.mosaicml.com/blog/amd-mi250? thanks.

btw, we are using NVIDIA-SMI 525.105.17   Driver Version: 525.105.17,  which is released at 2023.3.30.

i tried the possible combinations of MBS and other params such as attn_impl, and got similar perf for MPT-1B and MPT-13B (within ~3% diff) but still get 10%+ perf gap for MPT-3B and MPT-7B, comparing with data in the webpage. anyone can help? thanks.

Hello @guoyejun , if you are interested primarily in the NVIDIA benchmarking data, I would primarily refer you to our benchmarking page here: https://github.com/mosaicml/llm-foundry/tree/main/scripts/train/benchmarking
You can find details about the exact docker container used, as well as the MBS settings.

Hello @guoyejun , if you are interested primarily in the NVIDIA benchmarking data, I would primarily refer you to our benchmarking page here: https://github.com/mosaicml/llm-foundry/tree/main/scripts/train/benchmarking
You can find details about the exact docker container used, as well as the MBS settings.

thanks @hanlint , but looks that the data at https://github.com/mosaicml/llm-foundry/tree/main/scripts/train/benchmarking does not align with the data at https://www.mosaicml.com/blog/amd-mi250.

according to https://www.mosaicml.com/blog/amd-mi250,  we get the parameters that seq length is 2048, global batch size is 512 and GPU number is 8.

we filter the seq length, gbs and gpu# for the A100 80GB at the table from https://github.com/mosaicml/llm-foundry/tree/main/scripts/train/benchmarking, see below, only MPT-7B is there, and I don't understand how the result is aligned with the above table.

we filter the seq length, gbs and gpu# for the A100 40GB at the table from https://github.com/mosaicml/llm-foundry/tree/main/scripts/train/benchmarking, see below, MPT-1B and MPT-13B are there, and I don't understand how the result is aligned with the first table.

Hi @guoyejun, the settings/results in the train/benchmarking/README.md are a few months old and need to be refreshed. The stack is even faster now, that is why there is a discrepancy between the blog (new) and the README (old).
In the blog post, we tune for the best performance available independently on each system, so for AMD MI250 we used:

model.attn_config.attn_impl=flash
model.loss_fn=torch_crossentropy

And on A100 we used these settings:

model.attn_config.attn_impl=triton
model.loss_fn=fused_crossentropy

Here are the settings we used for every run in the blog post:

Model
GPU
Act Ckpt
Microbatch Size
Global Train Batch Size
TFLOP/s

MPT-1B
A100-80GB
False
16
512
185

MPT-1B
A100-40GB
False
8
512
171

MPT-1B
MI250
False
12
512
133

MPT-3B
A100-80GB
False
8
512
189

MPT-3B
A100-40GB
False
4
512
165

MPT-3B
MI250
False
8
512
140

MPT-7B
A100-80GB
False
4
512
193

MPT-7B
A100-40GB
False
2
512
176

MPT-7B
MI250
False
4
512
144

MPT-13B
A100-80GB
True
16
512
171

MPT-13B
A100-40GB
True
4
512
160

MPT-13B
MI250
True
16
512
122

Hope this helps! I will close the issue for now as I think this is all the details required to reproduce, but feel free to reopen / ping me if you see any issues.


[WIP] Verify send guard in Run

Contains the current progress of verifying the send guard in the run method.
Still missing:

proof of absIO_val(dp, msgs[i].Buffers[i], ingressID, msgs[i].N) == absIO_val(dp, tmpBuf, ingressID, msgs[i].N)
remove dummy functions (e.g. absIO_valDummy())
rewrite SplitRange_Bytes in Run
remove old code

Can you please merge master here when #237 and #246 are merged in master? Otherwise, the CI will not run on this branch

@mlimbeck can you remind me what are the lemmas that need to be proven to complete the checks for the guard? Maybe, it would make sense to merge the IO spec branch here and to merge this PR in that branch soon to avoid a proliferation of merge conflicts


🐛 [Bug] Password reset no error message

🐛 Bugreport

No error message is displayed on password reset page when the call fails or is successful. Have toasts implemented here

Done


Where can I find packages changelogs?

Hello,
First, thanks a lot for these great hooks :heart_eyes:
I am using some of them in a project. I am upgrading dependencies, and some hooks have new versions. I wanted to check what's new, but I couldn't find any changelogs. Do you generate some? If yes, where can I find them?
Thanks.

Any plans on adding them?
It'd be a lot easier to upgrade if we knew what to look out for

No. Most haven't changed in years. They're simple packages.


Include types during model equality check

Currently, BaseModel.__eq__ only considers the dict representation of the two models. This leads to incorrect (imo) equality for models of different types but the same fields. For example (not contrived, ~real code I wrote):
from pydantic import BaseModel

class Type(BaseModel):
    description: str = ""
    class Config:
        frozen = True  # Only relevant for hash comparison below

class Int8(Type):
    pass

class Int16(Type):
    pass

# then...

>>> i8, i16 = Int8(), Int16()
>>> i8 == i16
True
>>> hash(i8) == hash(i16)
False

Change Summary
This PR adds the model classes into the equality check to distinguish on type+content. Note that this new behavior more closely matches __hash__ (for frozen models), which mixes the class + the values (not sure why the keys aren't included :shrug:)
@samuelcolvin suggests a very similar change in https://github.com/samuelcolvin/pydantic/issues/1494#issuecomment-630144720. I avoided most of the "breaking change" described there - we're still converting the models to dicts. However, the class check may still be a breaking change for some esoteric uses, including 1 (fixed) test: test_generic_recursive_models, which was arguably incorrect in the first place (comparing an instance with the TypeVar against one without).
Related issue number
Tangentially: #1494
Checklist

[x] Unit tests for the changes exist
[x] Tests pass on CI and coverage remains at 100%
[x] Documentation reflects the changes where applicable
[x] changes/<pull request or issue id>-<github username>.md file added describing change
(see changes/README.md for details)
[x] My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

If this extra __eq__ check makes sense, please review when you have a minute @PrettyWood! 🙇

Hmm, this works fine for simple models, but doesn't work on nested ones as they are just turned into and compared as dicts.
Any suggestions on how that should be handled?

@JacobHayes I agree the current behaviour of __eq__ is not always desired.
Personally I overwrote it on one of my project where I rely on it with
    def __eq__(self, other: Any) -> bool:
        return self.__class__ == other.__class__ and tuple(self._iter()) == tuple(other._iter())

This is more consistent with dataclass (and pydantic.dataclass!) behaviour.
But this change needs some work because it breaks some stuff with generics and custom types.
And most importantly it will probably break some code for endusers that "don't care" about types and just want to compare the content.
Quite a lot of people want pydantic to be stricter but at the same time the "looseness" of it is one of its strength 🤷

Thanks for the snippet, I'll use your simpler one! I was about to suggest a page like the old flask snippets, but looks like they decided otherwise and deleted it. 😁
As you note on the breaks, do you think it best to just close this PR (fine with me)? I've seen some other talk about having a strict flag (and doing so here actually seems easy enough), but maybe that also falls into too much complexity for a relatively simple override.

I honestly think it would make sense for v2.
The current behaviour with dict comparison is even simpler to overwrite and this "loose" way of checking equality could be done by people who actually rely on it.

Ran into this issue in my use case, and wanted to ping it. As mentioned above, I believe the Python standard is to include class equality in an equality check, and I think the opposite behaviour (only considering data content) is confusing and non-intuitive. The comparison to dataclass is a good example of this.
I also wanted to add another argument for implementing this in v2:
It has been a long-contented issue whether what pydantic does should be described using the term "parse" or "validate" (see https://github.com/pydantic/pydantic/issues/578). I see in the v2 blog post that the issue will finally be settled, and "validate" is the winner. I am not very happy with this conclusion (and it also seems @samuelcolvin was not fully satisfied with this conclusion either, according to the smiley), but that is another issue and the ship has sailed for that one, I believe. (Granted, I did find pydantic myself by googling "python validation library" or similar, so there are at least practical arguments for this.)
I believe the main arguments for using the term "parse" can be most easily grasped by reading the influential blog post "Parse, don't validate" by Alexis King (which I btw discovered from a comment in the above-mentioned issue: https://github.com/pydantic/pydantic/issues/578#issuecomment-626575276). So the main point of this concept (also termed "type-driven development") is to parse data once according to the required model and then let this added information (that the data now adheres to the model) be stored/remembered by the data type. In that way, you would not need to validate the data ever again in your code base.  [Btw: adding a section on this in the pydantic docs would be awesome and a way to not fully cave in to the "validate" crowd! :) ]
Consequently, if one wants to use pydantic as the vessel for "type-driven design", the model itself is important for downstream logic, not only the parsed data. Hence, I think it would be natural for eq to consider also include the data types in the checks.
The main counter-argument is, I assume, that lots of existing code (and especially tests) will start failing if this is added, so if this is to be changed, v2 at least seems the proper place to do it.
I believe the best solution would be to add a config flag to enable/disable this behaviour (as mentioned above: https://github.com/pydantic/pydantic/pull/3066#issuecomment-894636366). However, I don't think this should be the same as the strict flag, as one would probably not set strict to True if following the "type-driven design" concepts. One could argue that checking types in equality tests should be the default if strict=False, however this seems counter-intuitive on a semantic level, as the equality check itself is actually more strict in this case. So a include_types_in_equality (or better-worded version) is probably the best solution, IMO. It also follows the zen of Python ("explicit is better than implicit").
I would also argue for include_types_in_equality being set to True as default for v2, as strict is by default set to False (I believe).

Revisiting this in v2; I think there are a few pieces of state beyond __dict__ that we may want to check for equality (beyond the type, which I agree should match). In particular, you can see essentially the full object state information here:
https://github.com/pydantic/pydantic/blob/83b3c49e99ceb4599d9286a3d793cea44ac36d4b/pydantic/main.py#L245-L251
This includes __dict__, but also __fields_set__ and __private_attrs__.
Currently, I am thinking we should have __eq__ look for matches on __dict__ and __private_attrs__, but not __fields_set__. Would anyone object to that? @samuelcolvin let me know if I should create a separate issue for this.
An additional concern: If we do matching on type, there are some nuisances with equality checks for generics with different parametrizations. In particular, note the following behavior:
T = TypeVar("T")
S = TypeVar("S")
class MyModel(BaseModel, Generic[T]):
    x: T
assert MyModel[T] is MyModel
assert MyModel[S] is not MyModel  # a subclass is created because S != T

Therefore, you could end up with:
MyModel[S](x=1) != MyModel[T](x=1)  # due to type inequality

I don't think this inequality is meaningful, but I also am not super keen to go down the rabbit hole of ensuring we get exact type equality if you reparametrize a type with a single typevar (I expect this would wreak some havoc on the current approach to resolving recursive generic types, and I really don't want to revisit that without a better reason than this edge case.)
I am open to opinions about how to resolve the generic types equality issue (or to just leave it unresolved).
(Note that currently, MyModel[int, str] != MyModel[int, T][str], but I think the caching implementation can be improved to fix this, so I'm less concerned about this as a source of equality misbehavior. I think it will be harder to resolve with generics that are re-parametrized with different typevars though.)


Lock versions of dependencies

Occasionally there are issues with latest versions of flask-base dependencies (gevent seems to be the one that causes problems most often).
Recent release 20.5.1 was not installing at all for a while, now it installs but causes pkg_resources.DistributionNotFound: The 'setuptools' distribution was not found and is required by zope.interface when running server (in docker).
We should lock the versions of our dependencies to avoid situations when releasing new versions of them breaks our sites, causes CI fails or prevents us from releasing.

Related GH workflow failing: https://github.com/canonical-web-and-design/vanilla-framework/pull/3096/checks?check_run_id=712141954


set default['solrcloud']['enable_ssl'] to false

IMHO, it is better to disable ssl by default

attribute disabled and updated README.


some-modules-and-internal-refactor

Related to #35

I think the best way moving forward would be to create individual PRs for each separate component

I did not do a good job at making clean commits. Separating this now would take a lot of time. I suggest preparing everything so that we both are happy and then merging this at once.

I did not do a good job at making clean commits. Separating this now would take a lot of time. I suggest preparing everything so that we both are happy and then merging this at once.

I'd rather wait for #34 to be merged then. Still, I might cherry-pick some changes from this onto the repo for now, will see.

Hey there, sorry for leaving this open for so long. Unfortunately due to the sheer amount of changes there is honestly no real way I can merge this all at once. Sorry for leaving this hanging for so long, I guess I let myself believe I'd get to it eventually, but at this point I think it's fair to say that I'm not really capable of taking this up in this state on my own.
So even though it hurts letting all these contributions go, for the sake of my sanity I'll be closing the PR. I do very much encourage you to submit these individually if you're interested; if not, I have added all of your changes into my personal nixvim ideas list so that I can add them individually in the future, in a more manageable way.

I have changed even more in the meantime (mostly API changes, clean-ups, and some new modules), so merging my changes to your project would likely not be possible, anyway.
I, also, think contributing them individually does not make sense anymore.
In case you or anyone is interested, you can find my fork via my profile. (Don't want to advertise for it here)


STOPS_BEFORE on multiple tokens retains the first token

The following query returns "What did the fish say when" instead of the expected "What did the fish say".
argmax(max_len=80)
   """A list of good dad jokes. A indicates the punchline
   Q: How does a penguin build its house?
   A: Igloos it together.
   Q: Which knight invented King Arthur's Round Table?
   A: Sir Cumference.
   Q:[JOKE]"""
from
   "openai/text-davinci-003"
where
   STOPS_BEFORE(JOKE, "when it hit")

This seems to be a bug with the OpenAI API and its "stop" parameter.
As a fix, we can disable LMQL's use of this parameter. Then the query works as intended:
argmax(max_len=80, chatty_openai=True, openai_nonstop=True)
   """A list of good dad jokes. A indicates the punchline
   Q: How does a penguin build its house?
   A: Igloos it together.
   Q: Which knight invented King Arthur's Round Table?
   A: Sir Cumference.
   Q:[JOKE]"""
from
   "openai/text-davinci-003"
where
   STOPS_BEFORE(JOKE, "when it hit")

I am not sure if that is the case, since when I run the query with the openai API the text parameter does not contain "when". However, the logprobs do still contain "when" as a token, and maybe LMQL uses that to to append it to the prompt? Below the raw output I get from OpenAI:
{'id': 'cmpl-7CwuSOAaCztaDmCAkalwTiSfghXuu',
 'object': 'text_completion',
 'created':<PHONE_NUMBER>,
 'model': 'text-davinci-003',
 'choices': [{'text': ' What did the fish say ',
   'index': 0,
   'logprobs': {'tokens': [' What', ' did', ' the', ' fish', ' say', ' when'],
    'token_logprobs': [-0.24742532,
     -0.21822312,
     -0.15329956,
     -0.6100834,
     -0.00090957596,
     -0.00039830978],
    'top_logprobs': [{' What': -0.24742532},
     {' did': -0.21822312},
     {' the': -0.15329956},
     {' fish': -0.6100834},
     {' say': -0.00090957596},
     {' when': -0.00039830978}],
    'text_offset': [214, 219, 223, 227, 232, 236]},
   'finish_reason': 'stop'}],
 'usage': {'prompt_tokens': 71, 'completion_tokens': 6, 'total_tokens': 77}}

Good observation. So then we can actually fix it, we just need to make sure to not consume "tokens" beyond what the "text" return value contains. I suspect, this can even include the case, where the last token that is produced is detokenized and then truncated. E.g. "tokens" may include ",\n" as token, but "text" already ends before "\n", leading to a ","-terminated string.
In these cases, we need to retokenize "text", to make sure a split (sub)token is replaced by its truncated version.

You are right, the following also produces the mistake:
argmax
   """A list of good dad jokes. A indicates the punchline
   Q: How does a penguin build its house?
   A: Igloos it together.
   Q: Which knight invented King Arthur's Round Table?
   A: Sir Cumference.
   Q:[JOKE]"""
from
   "openai/text-davinci-003"
where
   len(JOKE) < 120 and
   STOPS_BEFORE(JOKE, "id ")

Note the space at the end of "id ", this is necessary for the bug to appear, since " " is part of the token and if we just use "id", LMQL works as expected.


Update circe-core, circe-generic, ... to 0.14.7

About this PR
📦 Updates

io.circe:circe-core
io.circe:circe-generic
io.circe:circe-parser

from 0.12.3 to 0.14.7
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.circe" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "io.circe" }
}]

labels: library-update, early-semver-major, semver-spec-minor, version-scheme:early-semver, commit-count:1

Superseded by #328.


Data node Crash Loop

The data node is constantly restarting in the initialization phase after working well for few days.
kubectl logs es-data-0 -f

[2018-07-09T07:53:35,420][INFO ][o.e.n.Node               ] [es-data-0] initializing ...
[2018-07-09T07:53:35,572][INFO ][o.e.e.NodeEnvironment    ] [es-data-0] using [1] data paths, mounts [[/data (<ip-addr>:vol_016257cd4643b18711209d769f93e979)]], net usable_space [1.9gb], net total_space [2.9gb], types [fuse.glusterfs]
[2018-07-09T07:53:35,573][INFO ][o.e.e.NodeEnvironment    ] [es-data-0] heap size [2.9gb], compressed ordinary object pointers [true]

After this it crashes and starts a new init once again.
This the the yaml that I am using. I've set up requests and limits accordingly (the java heap is smaller that the pod limits, as people suggested), but with no avail.
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: es-data
  labels:
    component: elasticsearch
    role: data
spec:
  serviceName: elasticsearch-data
  replicas: 2
  template:
    metadata:
      labels:
        component: elasticsearch
        role: data
    spec:
      initContainers:
      - name: init-sysctl
        image: busybox:1.27.2
        command:
        - sysctl
        - -w
        - vm.max_map_count=262144
        securityContext:
          privileged: true
      containers:
      - name: es-data
        image: quay.io/pires/docker-elasticsearch-kubernetes:6.3.0
        env:
        - name: NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: CLUSTER_NAME
          value: myesdb
        - name: NODE_MASTER
          value: "false"
        - name: NODE_INGEST
          value: "false"
        - name: HTTP_ENABLE
          value: "true"
        - name: ES_JAVA_OPTS
          value: -Xms3g -Xmx3g
        - name: PROCESSORS
          valueFrom:
            resourceFieldRef:
              resource: limits.cpu
        resources:
          requests:
            memory: 3Gi
          limits:
            cpu: 2
            memory: 4Gi
        ports:
        - containerPort: 9200
          name: http
        - containerPort: 9300
          name: transport
        livenessProbe:
          tcpSocket:
            port: transport
          initialDelaySeconds: 20
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /_cluster/health
            port: http
          initialDelaySeconds: 20
          timeoutSeconds: 5
        volumeMounts:
        - name: storage
          mountPath: /data
  volumeClaimTemplates:
  - metadata:
      name: storage
    spec:
      storageClassName: gluster-heketi-external
      accessModes: [ ReadWriteOnce ]
      resources:
        requests:
          storage: 3Gi

Any idea or suggestion?

Does kubectl describe po/es-data-0 shows any kind of timeouts / issues during startup observed by kubernetes?
Eventually the pod gets killed because it does not startup fast enough to answer to the livenessProbe.

I've changed the ES_JAVA_OPTS to be 50% of the limits (and different variations of those parameters), but still the same crash loop.
Also, I don't see anything strange in the events log, besides the liveliness probe failure when trying to reach _cluster/health endpoint, but that is because the init process has not came to the point to start up the service successfully.
kubectl describe pod es-data-0
Name:           es-data-0
Namespace:      default
Node:           kubernetes-node1/<ip-addr>
Start Time:     Mon, 09 Jul 2018 10:30:02 +0200
Labels:         component=elasticsearch
                controller-revision-hash=es-data-776697d896
                role=data
                statefulset.kubernetes.io/pod-name=es-data-0
Annotations:    <none>
Status:         Running
IP:             <IP_ADDRESS>
Controlled By:  StatefulSet/es-data
Init Containers:
  init-sysctl:
    Container ID:  docker://320600efc4f4e2450933de60300b04b62fc442b422f55db0636c42ace9750115
    Image:         busybox:1.27.2
    Image ID:      docker-pullable://busybox@sha256:bbc3a03235220b170ba48a157dd097dd1379299370e1ed99ce976df0355d24f0
    Port:          <none>
    Host Port:     <none>
    Command:
      sysctl
      -w
      vm.max_map_count=262144
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Mon, 09 Jul 2018 10:30:03 +0200
      Finished:     Mon, 09 Jul 2018 10:30:03 +0200
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-875d5 (ro)
Containers:
  es-data:
    Container ID:   docker://0ab42eec948bfb61180ff55d9c431e9cbc7afb719fe47a2b76c716e6f0a726cc
    Image:          quay.io/pires/docker-elasticsearch-kubernetes:6.3.0
    Image ID:       docker-pullable://quay.io/pires/docker-elasticsearch-kubernetes@sha256:dcd3e9db3d2c6b9a448d135aebcacac30a4cca655d42efaa115aa57405cd22f3
    Ports:          9200/TCP, 9300/TCP
    Host Ports:     0/TCP, 0/TCP
    State:          Running
      Started:      Mon, 09 Jul 2018 10:30:49 +0200
    Last State:     Terminated
      Reason:       Error
      Exit Code:    143
      Started:      Mon, 09 Jul 2018 10:30:05 +0200
      Finished:     Mon, 09 Jul 2018 10:30:49 +0200
    Ready:          False
    Restart Count:  1
    Limits:
      cpu:     2
      memory:  4Gi
    Requests:
      cpu:      2
      memory:   2Gi
    Liveness:   tcp-socket :transport delay=20s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:http/_cluster/health delay=20s timeout=5s period=10s #success=1 #failure=3
    Environment:
      NAMESPACE:     default (v1:metadata.namespace)
      NODE_NAME:     es-data-0 (v1:metadata.name)
      CLUSTER_NAME:  myesdb
      NODE_MASTER:   false
      NODE_INGEST:   false
      HTTP_ENABLE:   true
      ES_JAVA_OPTS:  -Xms2g -Xmx2g
      PROCESSORS:    2 (limits.cpu)
    Mounts:
      /data from storage (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-875d5 (ro)
Conditions:
  Type           Status
  Initialized    True
  Ready          False
  PodScheduled   True
Volumes:
  storage:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  storage-es-data-0
    ReadOnly:   false
  default-token-875d5:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-875d5
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                 Age                From                       Message
  ----     ------                 ----               ----                       -------
  Normal   Scheduled              1m                 default-scheduler          Successfully assigned es-data-0 to kubernetes-node1
  Normal   SuccessfulMountVolume  1m                 kubelet, kubernetes-node1  MountVolume.SetUp succeeded for volume "default-token-875d5"
  Normal   SuccessfulMountVolume  1m                 kubelet, kubernetes-node1  MountVolume.SetUp succeeded for volume "pvc-306d82b3-805e-11e8-ac5c-7625bd182864"
  Normal   Pulled                 59s                kubelet, kubernetes-node1  Container image "busybox:1.27.2" already present on machine
  Normal   Created                59s                kubelet, kubernetes-node1  Created container
  Normal   Started                59s                kubelet, kubernetes-node1  Started container
  Warning  Unhealthy              19s (x2 over 29s)  kubelet, kubernetes-node1  Readiness probe failed: Get http://<IP_ADDRESS>:9200/_cluster/health: dial tcp <IP_ADDRESS>:9200: getsockopt: connection refused
  Warning  Unhealthy              14s (x3 over 34s)  kubelet, kubernetes-node1  Liveness probe failed: dial tcp <IP_ADDRESS>:9300: getsockopt: connection refused
  Normal   Pulled                 13s (x2 over 57s)  kubelet, kubernetes-node1  Container image "quay.io/pires/docker-elasticsearch-kubernetes:6.3.0" already present on machine
  Normal   Created                13s (x2 over 57s)  kubelet, kubernetes-node1  Created container
  Normal   Started                13s (x2 over 57s)  kubelet, kubernetes-node1  Started container
  Normal   Killing                13s                kubelet, kubernetes-node1  Killing container with id docker://es-data:Container failed liveness probe.. Container will be killed and recreated.

But you can see that, like I already assumed, the liveness is failing and therefore killing the container :-/
  Warning  Unhealthy              14s (x3 over 34s)  kubelet, kubernetes-node1  Liveness probe failed: dial tcp <IP_ADDRESS>:9300: getsockopt: connection refused
  Normal   Killing                13s                kubelet, kubernetes-node1  Killing container with id docker://es-data:Container failed liveness probe.. Container will be killed and recreated.

Could you try to increase the initialDelaySeconds like 2 minutes or so to see if it then will come up?

Hey, that worked. :)
That was a trivial overlook from my side. But, as the cluster gets in some data to process and startup the services, the readiness/liveliness need to be adjusted probably.
Thank you @mat1010 ! I will close the issue now.

@ivanovaleksandar  it appears your ElasticSearch data node is running with GlusterFS persistent storage. Do you run into issue related to CorruptIndexException errors?


Need some examples

I'm so excited exploring this project, I want to move to comtrya, are there examples that I can refer?

The best place for now is probably my dotfiles
https://github.com/rawkode/rawkode
there’s a dotfiles directory.
We’re working on a more sophisticated test suite which will contain examples of every action too, but that’s a work in progress.

https://github.com/topics/comtrya can find some examples from github.

I am going to close this issue since we have an examples direction with some samples. If there are missing examples, please feel free to open an issue requesting specific examples.


Documentation

[ ] TOC in the readme
[ ] link to the file in the src dir
[ ] create a nice logo
[ ] contributors list
[x] add them as owners

@tomekwi @hemanth I added you guys as owners feel free to push as you like or create a PR if you want feedback. But don't feel obligated:smile:

:tongue: Sure!

Wow, thanks :D

We should state somewhere in the readme that the whole thing is build out of oneliner functions.

I close this since only logo is left and  that will come.

:+1: