Duplicated code (seen with constant time comparison)

Hi,
BouncyCastle recently fixed a vulnerability in OpenBSDBCrypt. The culprit was faulty constant time comparison. The details of the error aren't very interesting. BouncyCastle is a big project, and sometimes our attention just slips. This error however hints at a deeper problem: code duplication.
OpenBSDBCrypt re-implemented constant time comparison inline instead of calling a common utility function. It still does. Constant time comparison is a bit trickier than it lets on. I submit that it should be implemented in one place, and used everywhere. Not only will it make the code base a little tighter and smaller, it will eliminate an entire class of bugs.
While we're at it, it might be a good idea to check whether other common functions are duplicated. Some of them may have to be inline for performance reasons, but those who don't should probably be factored out as well.

I agree about adding the utility function, although we want to leave it as is for one more release. One thing, would you point out where else the comparison is being done? Thanks.

I agree about adding the utility function

Thanks. :+1:

(mind you, that could have easily been wrong instead...)

That has crossed my mind. I believe that had it been a common function from the beginning, it would have received more attention, and any bug would have been uncovered sooner. Here, special attention will be needed when moving this function to a common utility, to make sure no further bug is introduced.
(While we're at it, you may want to question the use of the == equality operator. It may be fine in Java, but I've heard C compilers tends to generate variable time for those — which is why I personally use XOR instead.)

we want to leave it as is for one more release.

No rush. I see you have a lot on your plate, I understand this issue is not high priority.

would you point out where else the comparison is being done?

Well… I don't know BouncyCastle's code base, so I haven't checked beyond OpenBSDBcrypt. I just acted on a strong suspicion that other primitives & constructions may be affected. Sorry.
You most probably know the following, but in case someone else wants to pick it up: my first reflex would be to check anything that has "check" or "verify" in its name, or any function that returns a boolean. Those functions are most likely found in the following:

Authenticated encryption.
MAC (Such as GHASH, Poly1305, HMAC…)
Hashes, maybe
Password hashes
Signature verification

Also, consider using MessageDigest.isEqual (https://github.com/AdoptOpenJDK/openjdk-jdk11u/blob/712f3de16fd2c4e31d60bd105aac864e2929abaf/jdk/src/java.base/share/classes/java/security/MessageDigest.java#L455).

Seconded, with some reservation about the preliminary checks:

Don't test whether the two buffers are one and the same. It's redundant with the main loop, and in practice always returns false.
If one of the buffers is null, that's a programming error. Throw an exception.
If the buffers have different lengths, that's a programming error. Throw an exception.

Also, OpenBSDBCrypt compares strings. What @dbaxa proposes compares byte arrays. Some adjustment will be needed. Either we decide that comparing strings is not a good idea, and we replace them all with proper byte arrays… or we modify the function so it compares strings. I'm personally not competent enough to have an opinion.

On a related note, Spring security converts strings to byte arrays like so - https://github.com/spring-projects/spring-security/blob/master/crypto/src/main/java/org/springframework/security/crypto/bcrypt/BCrypt.java#L758

By way of further information we have some constant time functions in our Arrays class already, not all JVMs actually have the MessageDigest API (probably a lot less true now than 20 years ago) and calling something like Arrays.constantTimeAreEqual() doesn't leave any doubt in the mind of a reader. OpenBSDBCrypt appears to be the only place a String comparison is done like this at the moment, but at this stage I'm leaning towards adding a Strings.constantTimeAreEqual().

It may be fine in Java, but I've heard C compilers tends to generate variable time for those — which is why I personally use XOR instead.)

It is guaranteed in java. As in, both a.charAt(i) == b.charAt(i) is as constant as anything you're going to write, as is flag &= condition - that is as constant as flag = flag & condition is going to be. I don't see how you can use XOR for that last operation; you'd be toggling the flag, very much not what you wanted. Theoretically you can translate each of the 60 comparisons to a bit on a long, and then test if the end result is 0 to indicate no differences - now you still don't really need XOR, you might as well use bitwise OR - same result. I don't think this would make the code and more constant-time than the much simpler flag &= condition used now.

I don't see how you can use XOR for that last operation

I don't. XOR replaces equality:
isDifferent |= a[i] ^ b[i];

Then I use an arithmetic trick to implement the isDifferent == 0 test. Here's my code in C.
Of course, those precautions aren't needed in languages that guarantee a == b is constant time.

Actually I have a feeling the guarantee breaks down for longs... I'd have to check though. I've added Strings.constantTimeAreEqual() so there's something in place if the need to do this, by us or others, comes up again.


Preliminary models support and models example 11

This adds support for models without animations or materials textures & an example to showcase that.
One thing I hoped to be able to do but which didn't seem to work was shadowing *claylib-background* in a let so that with-drawing would clear the background accordingly. Having to call clear-background again is redundant. Any idea why shadowing doesn't work?

We've opened enough cans of worms I don't think it makes sense to merge this. If there are standalone pieces here you're welcome to resubmit a smaller PR.


flutter logs includes W/ActivityManager lines but they're often not interesting

I don't want to see stuff like:
W/ActivityManager: Permission Denial: Accessing service ComponentInfo{com.google.android.music/com.google.android.music.dial.DialMediaRouteProviderService} from pid=2906, uid=10085 that is not exported from uid 10065

Just hit this as a new Flutter user. It's very confusing when you see these errors the first time ever you run a Flutter app, as it leads you to think you have done something wrong, or the system is not working.

Agreed. Thanks for the feedback here. Patches welcome! :)


Use a per-install name for securityconfig secret

Description
Give the securityconfig secret an autogenerated unique name to
facilitate installing the chart multiple times in the same namespace.
This helps with the common case of sharing the securityconfig between
multiple instantiations of this chart to construct an Opensearch
cluster.
Issues Resolved
Closes: #40
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

The problem with this is the previous implementation allowed someone to pass in their own template/secret.yaml in their child chart.

Could you explain how this was supposed to work previously?
The volume bindings on the statefulset have conditionals like this, which is identical to the conditional on the creation of the securityconfig secret object:
{{- if and .Values.securityConfig.config.securityConfigSecret .Values.securityConfig.config.data }}

This means that they wouldn't have been able to supply an external secret name and have the statefulset use it with this mechanism.
I think you can still use an external secret by specifying extraVolumes + extraVolumeMounts?

@smlx I'm going to say this wasn't ported very well from the original. Those conditions are a bug, but I think your change still introduces more restrictions.
It'd be more logical to just remove the verbose (and unnecessary) conditions you've outlined.

Ok, so what if we changed it to have this kind of logic:

if you define anything under data, the chart will automatically create a secret and mount it (this is what the PR does now)
if you define securityConfigSecret, the chart will assume this secret is created externally and try to mount it.
it is an error to define both data and securityConfigSecret

Does that satisfy all the use cases? I'll obviously add some comments explaining how this works.

@smlx Perfect middle ground. Agreed.

Ok, so what if we changed it to have this kind of logic:

if you define anything under data, the chart will automatically create a secret and mount it (this is what the PR does now)
if you define securityConfigSecret, the chart will assume this secret is created externally and try to mount it.
it is an error to define both data and securityConfigSecret

Does that satisfy all the use cases? I'll obviously add some comments explaining how this works.

@smlx Can we add this explanation in the README for OpenSearch chart. It will be great to avoid any confusions for users.

Any progress on this PR @smlx ?

I've rebased and bumped the version.

Ok, so what if we changed it to have this kind of logic:

if you define anything under data, the chart will automatically create a secret and mount it (this is what the PR does now)
if you define securityConfigSecret, the chart will assume this secret is created externally and try to mount it.
it is an error to define both data and securityConfigSecret

Does that satisfy all the use cases? I'll obviously add some comments explaining how this works.

@smlx Can we add this explanation in the README for OpenSearch chart. It will be great to avoid any confusions for users.

@smlx Can we add it in the README as well?

@TheAlgo ok, added a line to the README.

@peterzhuamazon Yes to 1.1.0 as this introduces a new non-breaking feature and therefore mandates a minor version number increment.

OK, changed version to 1.1.0.

@smlx I think that's the best way to address this.


feat: pseudo-code meritocratic votes

Opening a PR to illustrate what I've been thinking about.
I've been working with the Open Zeppelin Governor suite of code. This usually works so that holders of either ERC721s or ERC20s can partake in governance and vote on proposals. I want to implement meritocratic voting so that if someone has certain attributes they have a higher vote than the next person. I'm imagining the best way to do this might be to expand upon the standard ERC721Votes class in Open Zeppelin :
For ERC721Votes  _getVotingUnits returns 1, or 0 - as in the user either owns an NFT token or not.
I've been thinking it would be more useful for a meritocratic score to instead returns a number between 0 and 1, that could be 0.6 for example, representing their voting score.
This voting score is determined by applying different multipliers to a starting score (let's say 0.2) depending on if the person is more or less qualified to vote on the proposal.
To give an example use case for governance in an Open Source software funding allocation organization, we may want to give a higher vote to those people who have been attested to being great open source developers. This could take the form of an attestation in the Ethereum Attestation Station, or it could be an Attribute that is saved in the storage of the NFT itself.
Perhaps a multiplier attribute called isBuilder for example. Perhaps this multiplies their voting score by 20%. We should allow for any number of multiplier attributes, each of which increases the multiplier of the token holder's vote. I guess this should be a soul bound NFT.
Anyway I've generated some semi-pseudo code below, curious to hear what you think....

I've thought about this for a while. How about an additive approach to determine the voting score @positonic?
The way I would design this is to pick a base like 1,000 to represent the fraction with 3 decimals precision (or more, if we wanted, it doesn't really matter). Everybody would start with 0 voting power and add voting power based on different properties.
Are they a builder? -> + 200 points (20%)
Do they have an attestation that they're an expert in a topic? -> + 300 points (30%)
Did they put money in the pool? -> + X points (capped at 500 to not run over the total of 1000 points)
You'd have to make sure that you can't have attributes adding up to more than 100%. This should be enforced by the smart contracts and the interface to create the campaign.
Why I think this is better than doing multiplication:

starting from 0 is more intuitive than starting from an arbitrary amount of 0.2
Designing the attributes so that they exactly make up 100% if they're all added together is straightforward

What do you think?


removed tools folder

as per user feedback #9782 , after reading this article, I found Tools folder must be removed from   C:\ , so I removed  C:\Tools

No this is not correct. The path is arbitrary.


Cars not saved on the map [AGAIN]

Hey.
When you buy a car in cider two it disapear from the map and it doesent appear on the inventory. I saw that there was an old thread concerning this issue. If you have solution please post them.
Thanks

Woops, I didn't mean to unassign @VortixDev. I just wanted to set this as expired.


Create Entity and History Push to a Haystack server

@ci-richard-mcelhinney As discussed, we have a proposal to extend the Haystack client capabilities to include pushing history data to a Haystack server. It supports pushing to pre-defined entities or automatic creation of point entities if they don't exist. If support is detected via ops op, it will use a create operation, which is a WideSky extension to the Haystack REST API standard.
Below is a proposed update to the docs which adds to the existing section 7 "Using NHaystack as a client"

7.1. Exporting history to another Project Haystack server
The NHaystack client can be used to push history stored on the Niagara station
to an upstream Project Haystack-compatible server.  This feature has been
successfully tested on both NHaystack server and on WideSky, but should
work with any Project Haystack server.
The NHaystack History Export Manager can be found by right-clicking on
Histories beneath your Project Haystack server instance and selecting Views â†’
NHaystack History Export Manager.  From here, clicking Discover will present
you with a list of all the histories on the station.
After selecting the histories of interest, click Add, and you'll be presented
with a dialogue box listing the histories and the options to be set.
From this form, each history must be associated with the Ref of the point
to which it will write its historical data.  There are three ways this can be
done.
7.1.1 Manual association of points via the Add form
The lowest-common denominator method here is to use a standard Project
Haystack client to get a list of valid points from your server, then manually
copying the id of each point into the Id field on the Add form.
This should work for any Project Haystack server.  The point is assumed to
carry a his tag, be of the correct kind for the type of history being
exported, and either carries a tz tag matching that of the history, or is on
a server that supports time-zone conversion.
For more detail on tagging assumptions, see section 7.1.2 below.
7.1.2 Automatic look-up using axStation and axHistoryId.
Depending on the Project Haystack server, it may be easier to retrieve a list
of histories from the station and tag the points so the station can find them.
Such a list can be obtained from WorkPlace via the following procedure:

Double click on your station's "History" space (it'll take you to a chart
view)
Press CTRL+L to bring up the ORD dialogue
At the end of the text field (after history:), add
bql:select id, recordType, timeZone from /
From the File menu, select Export.
Choose "Text to CSV" and "Save to File"
Enter the path where you want the file and click OK.

Filter out of that list the histories you want to upload.  To associate a
history in this list to a point in your asset model, tag the point with the
following two tags:

axStation (Str): should be the name of your Niagara station (as seen in
brackets in the WorkPlace navigation tree, e.g. mystation).
axHistoryId (Str): should be the ID of the history taken from your BQL
query dump.

The point will also need to have kind and tz set up properly:

Histories of type history:NumericTrendRecord should be tagged with
kind=="Number".  Ideally these should also be tagged unit (Str) with an
appropriate Project Haystack unit.
Histories of type history:BooleanTrendRecord should be tagged with
kind=="Bool"
Histories of type history:EnumTrendRecord should be tagged with
kind=="Str" and ideally should also be tagged enum (Str) with a
comma-separated list of valid enum values.
Histories of other types should be tagged with kind=="Str".

7.1.3 Automatic creation of points
If the server supports the createRec call, then points will be automatically
created on the Project Haystack server.  The station will issue a HTTP POST
instruction to the server's createRec endpoint with a single-row grid
containing the following columns:

name (Str): This will be the name of the control point, or if that's not known,
the name of the history.
dis (Str): This will be the Display Name of the control point (or
history), or if those are blank/null/not known, the name of the new point.
point (Marker)
his (Marker)
kind (Str): The point kind; one of Str, Bool or Number, depending on
the type of history being exported.
tz (Str): The timezone of the history (this is set in the Add dialogue,
default is automatically determined from the history database).
axStation (Str): The name of the Niagara station exporting the history
axHistoryId (Str): The ID of the history being exported
axSlotPath (Str): If known, the slot path of the control point (may be
omitted if not known).
unit (Str): If known and history is a NumericTrendRecord, the
measurement unit for the control point
enum (Str): If known and history is a EnumTrendRecord, the list of valid
enumeration values for the control point

It is assumed that siteRef and equipRef are optional and can be filled in
later via other CRUD ops by the end user.  The axStation and axHistoryId
should be used to obtain listings of the orphaned point entities so they can
be associated with the correct site and equip.
7.2 Fine tuning payload sizes
The number of records exported at a time can be tuned by configuring the
Upload Size.  This is a count of the number of rows used in each hisWrite
request.  It defaults to 10000, which depending on the server may work, or may
result in a "Request Entity Too Large" error message from the server.
In such cases, the driver will automatically divide this value by two and try
again, so if the server experiences difficulty processing the set number of
records, you'll see smaller payloads (5000, 2500, 1250, etc) attempted.  (It
will not go any smaller than 1 record.)
You can manually set this value to any payload size suits your requirements.
7.3 Setting an alternate start point
By default, exports begin at the very first record in the history.  This can
be configured via the "Upload From Time" field.  This specifies the start
point for the very next export task.
It can be used to "skip over" invalid data that was captured during the
commissioning process, as well as to re-upload data in the event that data is
lost or corrupted on the upstream Project Haystack server.

Thanks @patrickc77, I'll take a look and let you know if we have any further suggestions.

How was the review going? Would you like me to submit some code? It would still need a rebase with the latest branch.

@patrickc77 I have added the documentation you provided to the Wiki so we can collaborate on this more easily.  Please let me know if you need to be added as a contributor to the project so you can edit where necessary.  I've modified some of the tag names to make them a little more generic as well.  I will review the docs further today.
We probably need to document the REST API calls for the extra Project Haystack Ops somewhere, would you like to do it on this Wiki as well?

I'm happy to be added as a contributor. We didn't add create, update & delete capability to the nHaystack server, they are a WideSky extension at the moment, but I'm happy to document them. I'm not sure where in the Wiki they would go.

Couple of comments:

In 7.1.2 would the 'unit' tag also be required? I believe there may be some sanity checks here in place that checks this however im unsure if this is within Haystack or Skyspark.
This may relate to the payload size but how should the sync of data occur? Such as, if a new point is created would the sync occur from the 'beginning' of the history on the server? Furthermore, if the client rec has a 'hisEnd' should the data sync from that time period onwards?

Any rules for required tags will be specific to the implementation of a Haystack server at the minute.  I'm pretty sure there aren't any rules documented for required tags.  I mean, they might be documented but their enforcement will be down to the specific implementation.

@callum-rosel

unit tag: If a specific Haystack implementation requires it, that's fine. We needed to document the minimum requirements for nHaystack client.
Data sync: Is there something in section 7.3 that's not covered?

@ci-richard-mcelhinney There are slight differences between the original comment and the version in the wiki shall we treat the wiki as design reference?

@patrickc77 agree, let’s edit and work on the wiki version, I think it tracks changes as well.

@patrickc77 I'm closing this issue as your contributions have been merged into the n4 branch and will be included in the upcoming release.


Added support for phony dims

This fixes #41
If num_scales == 0, then we detected dimensions but could not access names. This will add phony_dim_x as dimension names.
Tested on 'az://modis-006/VNP14A1/08/04/2020001/VNP14A1.A2020001.h08v04.001.2020003132203.h5'

We could also let _get_array_dims run as-is and if the returned dims is empty, then add the phony dimensions. Thoughts?

This is fine, but I would appreciate a comment in that block, linking to the issue and saying that this is what netCDF4 does.

I added a comment block, let me know if there's anything else you'd like added


How do I run this?

Instructions state to just git clone..? Now what?  ;-)

@js1972 : Read me got updated.

Read the README.md file.  It is a text file usually found in every repository with information to help you get started.
This free course has content about Unity - the engine that "drives" the self-driving car sim.  Pun not intended.

Thanks @deepmehtait I see the readme has just been updated - perfect.


KAFKA-14412: Decouple RocksDB access from CF

To support future use-cases that use different strategies for accessing RocksDB, we need to de-couple the RocksDB access strategy from the Column Family access strategy.
To do this, we now have two separate accessors:

DBAccessor: dictates how we access RocksDB. Currently only one strategy is supported: DirectDBAccessor, which access RocksDB directly, via the RocksDB class for all operations. In the future, a BatchedDBAccessor will be added, which enables transactions via WriteBatch.
ColumnFamilyAccessor: maps StateStore operations to operations on one or more column families. This is a rename of the old RocksDBDBAccessor.

@cadonna @mjsax @ableegoldman @lucasbru @wcarlson5 @bbejeck @vvcephei @guozhangwang
This is part of KIP-892, and has been broken out into a separate PR to reduce the review burden on the main KIP-892 implementation, since it can be merged independently.
 The existing test suite should ensure no regressions.

@nicktelford Seems like all build jobs timed out. Could you take a look?

@lucasbru I think this is just the CI causing trouble again. The build passes locally. Is there a way to rerun the build without pushing more commits?

@nicktelford I'll rerun it. That being said, the last jobs on trunk have all finished within 3-5 hours, so this must be caused by either infrastructure or the code in this PR.

@lucasbru OK, my bad. It turns out I did a minor refactoring after I ran the test suite yesterday that was so insignificant I didn't think I needed to run the tests again... Turns out I was wrong :see_no_evil:
I've fixed the bug now and the tests pass locally for real now.


which time upgrade global Lodash version to v3.1.0?

There are breaking changes between 2.4.1 and 3.1.0, so existing users may be screwed. You can upgrade at your own risk.

Just wanted to note that I tried it out and it is not working.
First error I am getting:
project/node_modules/sails/lib/app/private/after.js:50
    _emit.apply(emitter, args);
          ^
TypeError: undefined is not a function

If anyone wants to digg in, would probably not be too hard to patch.

I'm not clear on what the question is here.  Sails hasn't upgraded its version of Lodash...

Sorry, no question just thought if anyone was coming to look for the exact same issue (which someone probably will) they would at least have an immediate reference of it not working.

@Globegitter what did you try out?  Upgrading the Lodash inside of Sails core?  Why oh why :crying_cat_face: ?

If you install Lodash 3.x as a dependency of your own project, Sails will use that version rather than its own, which results in the error @Globegitter reported.
I am not sure how to work around that. I would like to use some of Lodash 3.x's features in my own code without breaking Sails.

Sails is uber awesome. However, running into the same issue. Would really like to use sortByOrder()  offered in 3.1. :(

@blah238 @mattymil Does this occur even if you use the global Sails?  Or only when Sails is installed directly in the node_modules of your project?

Only in project node_modules.

On May 11, 2015, at 12:38 PM, sgress454<EMAIL_ADDRESS>wrote:
@blah238 https://github.com/blah238 @mattymil https://github.com/mattymil Does this occur even if you use the global Sails? Or only when Sails is installed directly in the node_modules of your project?
—
Reply to this email directly or view it on GitHub https://github.com/balderdashy/sails/issues/2626#issuecomment-100973730.

Just to circle back on this, if you want to use a higher version of Lodash in your local project, I think it should work fine--just make sure you disable the global Lodash and run npm update afterwards so that your local Sails install downloads its own copy of Lodash 2.4.1.

Got it.. thanks
From:  sgress454
Reply-To:  balderdashy/sails
Date:  Wednesday, June 3, 2015 at 7:35 PM
To:  balderdashy/sails
Cc<EMAIL_ADDRESS>Subject:  Re: [sails] which time upgrade global Lodash version to v3.1.0? (#2626)
Just to circle back on this, if you want to use a higher version of Lodash in your local project, I think it should work fine--just make sure you disable the global Lodash and run npm update afterwards so that your local Sails install downloads its own copy of Lodash 2.4.1.
—
Reply to this email directly or view it on GitHub.

The question was useful indeed, just ran into the exact same issue. Thanks!

experiencing the same issue.


Tweak riddle arguments

Aligns riddle arguments with experiences found in other command line tooling. @<file> would have been ideal for response file handling but that conflicts with the PowerShell splatting operator.
Some prior art on short/long arg syntax:

https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html
https://docs.rs/clap/latest/clap/

Some prior art on @:

https://learn.microsoft.com/en-us/windows/win32/midl/the-response-file-command
https://natemcmaster.github.io/CommandLineUtils/docs/response-file-parsing.html?tabs=using-attributes

Hm, unless I'm missing it there doesn't seem to be any precedent for -@? That seems like the worst of both worlds, no? It's sigil soup and non-standard.

That's fair, thoughts on a replacement? -etc doesn't seem right given it's not et cetera (and other things).

Dropped the short forms and restored --etc. Kept the long forms (with double hyphens).

I admit I don't otherwise have any strong preferences, so long as it's not too weird. Consistency would be great but unfortunately Windows command line tools are all over the place. Even relatively recent first party, developer focused tools can't quite seem to agree on a convention (e.g. vswhere and winget).

Yes I've not been overwhelmed by the consistency in command line tools either, so I've always just tried to find the simplest and most intuitive approach for my tools.
By the looks of it, this (final) commit only changes from a single - to a -- and otherwise leaves things as they were. I don't love it, but I recognize it is at least a little more in line with the Rust family of command line tools, so I think this is probably fine.


Territory field not ISO 3166-1 alpha3 code

https://github.com/openregister/registry-data/blob/master/data/beta/field/territory.yaml#L6
says that the codes are ISO 3166-1 alpha3 which does not seem to be true:
https://en.wikipedia.org/wiki/ISO_3166-1_alpha-3

Also by having a mixture of ISO and non-ISO codes in this register, we have no way to differentiate between them. This feels to me like the description cannot be valid whilst this is the situation.


BaseView -> findViewFile: Check existence of generated view path as well

Q
A

Is bugfix?
✔

New feature?
❌

Breaks BC?
❌

Fixed issues
#14285

PHPDoc of this method explicitly mentions that file may not exist, so it does not look like right place to add this check.
https://github.com/yiisoft/yii2/blob/2808e8d814bbfe5d46c4563b2f3aa476e709dd8d/framework/base/View.php#L166


Histogram refactoring

refactor the config API to make building configured Histograms cleaner
refactor internals to remove 'Histogram' prefix on internal structs

This attempts to address #18


Public Key not Paired with Vehicle despite successfully completing https://www.tesla.com/_ak/

I am trying to access my vehicle as follows:

Created a key using tesla-keygen and run a local proxy
Stored the key on the server as ./well-known/appsepcific/com.tesla.3p.public-key.pem (is this correct?)
Successfully authenticated using https://www.tesla.com/_ak/ & allowed authorization via the iphone tesla app

Now the key appears on the Vehicle's touchscreen. Using the Vehicle endpoint works well. However, the Vehicle Command endpoint returns the following error:
{
  "response": null,
  "error": "vehicle rejected request: your public key has not been paired with the vehicle",
  "error_description": ""
}

Am I missing something?

Could you run the proxy server with -verbose and paste the output it produces when you send the command? Please redact your VIN.

This is the response I'm getting:
2024-02-01T21:45:51Z [debug] Creating proxy
2024-02-01T21:45:51Z [info ] Listening on localhost:4443
2024-02-01T21:45:54Z [info ] Received POST request for /api/1/vehicles/<VIN>/command/door_unlock
2024-02-01T21:45:54Z [debug] Executing door_unlock on <VIN>
2024-02-01T21:45:54Z [info ] Starting dispatcher service...
2024-02-01T21:45:54Z [info ] Requesting session info from DOMAIN_INFOTAINMENT
2024-02-01T21:45:54Z [debug] Sending request to https://fleet-api.prd.eu.vn.cloud.tesla.com/api/1/vehicles/<VIN>/signed_command: {"routable_message":"MgIIAzoSEhBA4WfLyzx1IBWOQDy9kI7vmgMQf910byErRuv4oPUfTjZ79HJDCkEE4CKbd4lIo2Kho+RNPa5ym1ox4xbrkLBZbCUQ7wqgUvHej3LMi13sg0FdmyuzjUW15U4bxLZM9Oengbuj6Reu8Q=="}
2024-02-01T21:45:54Z [info ] Requesting session info from DOMAIN_VEHICLE_SECURITY
2024-02-01T21:45:54Z [debug] Sending request to https://fleet-api.prd.eu.vn.cloud.tesla.com/api/1/vehicles/<VIN>/signed_command: {"routable_message":"MgIIAjoSEhBo6W0jQsfJmyyvyfmTZqC+mgMQ061rj+wDMErr/NMW06zDhnJDCkEE4CKbd4lIo2Kho+RNPa5ym1ox4xbrkLBZbCUQ7wqgUvHej3LMi13sg0FdmyuzjUW15U4bxLZM9Oengbuj6Reu8Q=="}
2024-02-01T21:45:54Z [debug] Server returned 200: OK: {"response":"MhISEEDhZ8vLPHUgFY5APL2Qju86AggDYgQIAhADkgMQf910byErRuv4oPUfTjZ79A=="}
2024-02-01T21:45:54Z [error] Returning error Internal Server Error
2024-02-01T21:45:54Z [debug] [d3ad6b8fec03304aebfcd316d3acc386] Retrying transmission after error: Post "https://fleet-api.prd.eu.vn.cloud.tesla.com/api/1/vehicles/<VIN>/signed_command": context canceled

Thanks for any advice :)

Update:
After running tesla-control -ble add-key-request public_key.pem owner cloud_key and tapping my physical card in the center console of the vehicle, the key is paired successfully. I can issue vehicle commands via BLE and via the Proxy.
However, enrolling vehicles manually with a BLE-enabled notebook isn't feasible for large fleets. Isn't https://www.tesla.com/_ak/<domain> supposed to pair the key exactly to avoid the need to physically tap the card with a BLE-enabled notebook nearby?

@fktc
Do you know what commands are avalible over ble?

@vagh141 : I don't know what commands are available over BLE. I used BLE as a workaround to pair the key with the vehicle, and am now using the http proxy & the vehicles-command endpoint to communicate with the vehicle.
The question remains why https://www.tesla.com/_ak/<domain> didn't correctly pair the key, while BLE did?

I tested my python implementation of the registration process ( here ) and found the app not reacting on https://www.tesla.com/_ak/domain at all on Android and on iOS.
Then I tested again this afternoon with the same process, and suddenly it worked.
Is anyone aware of a change in the app or the server?

@fktc are you able to reproduce the issue? If so, could you provide the vehicle's firmware version and determine if a touchscreen reboot resolves the problem?

I changed the public key on the server without re-registering it via POST /api/1/partner_accounts. It might be worthwhile being more specific in the documentation about the exact order of things, or to prevent https://www.tesla.com/_ak/ from returning success messages if the actual key on the server isn't matching the registered key.
In any case, thanks for your help! I'm closing the issue now.

@sethterashima I would really appreciate, if the key registration process returned a specific error in this case? This seems hard to catch otherwise.

@fktc https://github.com/teslamotors/vehicle-command/issues/174 i have created. could you please tell me which private key i should use to start the proxy?
In the documentation with tesla-key-gen i generated public key and registered in the app as well as the vehicle by issuing https://www.tesla.com/_ak/
But i am still not able to issue the vehicle command and getitng key is not paired in the vehicle. I suspect i am not starting the proxy with right private key. Please help


”Măgura”, ”Tămășasa”, ”Turmaș” are listed in the drop-down menu

Description:
When user writes ”Măgura”, ”Tămășasa”, ”Turmaș” the locations appear in the drop-down menu next to the comune and the county which is a part according to the law.
Precondition:
The website is up an running.

Step 1
Write in the search bar "Măgura"

Expected results
The location appears in the drop-down menu as "Sat Măgura, HUNEDOARA (Mărtinești)".

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Write in the search bar "Tămășasa"

Expected results
The location appears in the drop-down menu as "Sat Tămășasa, HUNEDOARA (Mărtinești)".

Step 4
Press "x" button.

Expected results
The location was deleted from the search bar.


pubsub: hanging in alpine

I'm trying to use pubsub. I have a very basic subscriber - full code here. I have a very basic alpine dockerfile - full code here.
When I run this dockerfile, require 'google/cloud/pubsub' hangs forever.
What gives?
edit: For the record, it works fine outside the docker image on my mac.
edit2: Only happens when I require google/cloud/pubsub. Other requires work fine.
edit3: Happens with and without compiling gems from source (BUNDLE_FORCE_RUBY_PLATFORM=1).

Possibly related to https://github.com/GoogleCloudPlatform/google-cloud-ruby/issues/1351

Hi @jadekler, I'm looking into this, but what you are describing is certainly perplexing. Have you tried replacing your code with only the require to see if it also hangs?
When you say it works on other requires, what requires do you mean? Does it work with Google Cloud Datastore?

How are you running your Docker container? Are you running it locally on your mac? If so, are you running Docker for Mac or Docker Toolbox? Or are you running this on a server? If so, where?

Wayyyy later I get this error:
#<Thread:0x00005599632d12f0@/usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:270 run> terminated with exception (report_on_exception is true):
/usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:242:in `rescue in background_run': 16:Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. (Google::Cloud::UnauthenticatedError)
	from /usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:193:in `background_run'
	from /usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:271:in `block in start_streaming!'
#<Thread:0x00005599632c3808@/usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:270 run> terminated with exception (report_on_exception is true):
/usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:242:in `rescue in background_run': 16:Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. (Google::Cloud::UnauthenticatedError)
	from /usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:193:in `background_run'
	from /usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:271:in `block in start_streaming!'
#<Thread:0x00005599632d2b50@/usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:270 run> terminated with exception (report_on_exception is true):
/usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:242:in `rescue in background_run': 16:Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. (Google::Cloud::UnauthenticatedError)
	from /usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:193:in `background_run'
	from /usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:271:in `block in start_streaming!'
#<Thread:0x00005599632d0738@/usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:270 run> terminated with exception (report_on_exception is true):
/usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:242:in `rescue in background_run': 16:Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. (Google::Cloud::UnauthenticatedError)
	from /usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:193:in `background_run'
	from /usr/local/bundle/gems/google-cloud-pubsub-0.30.1/lib/google/cloud/pubsub/subscriber/stream.rb:271:in `block in start_streaming!'

I've also tried installing ca-certificates to no avail.

@blowmage

Removing everything except the require, no more problem. So, must be further down despite my logging (logs not working as I'd expect for some reason?). Looking into this more..
I've tried require json and it works fine
This is running locally on docker for mac. I'm using docker run foo; no bells and whistles

So, another update: I've individually remove line by line and there doesn't seem to be any issue with any individual line. When I place the loop [1] back in, that's when I start having issues - my logs apart from the very first one before the require don't work, I never receive any messages, etc. I'm wondering whether the loop is the problem. What's the idiomatic way to "wait for messages forever"?
1:
loop do
  sleep 300
end

We usually encourage calling sleep with no arguments.

Right on. I put that in, same problem. @blowmage any other ideas?

This fails on other OSes too; just tried jessie.

New error:
#<Thread:0x00005603797b3658@/usr/local/bundle/gems/grpc-1.10.0/src/ruby/lib/grpc/generic/bidi_call.rb:74 run> terminated with exception (report_on_exception is true):
/usr/local/bundle/gems/grpc-1.10.0/src/ruby/lib/grpc/generic/bidi_call.rb:158:in `run_batch': call#run_batch failed somehow (GRPC::Core::CallError)
	from /usr/local/bundle/gems/grpc-1.10.0/src/ruby/lib/grpc/generic/bidi_call.rb:158:in `write_loop'
	from /usr/local/bundle/gems/grpc-1.10.0/src/ruby/lib/grpc/generic/bidi_call.rb:75:in `block in run_on_client'

This fails on other distros too; just tried jessie.

To be clear, this is failing for you on debian inside a docker container? Not in a normal VM, right?

Can you make a couple changes and let us know what output you get. (If its too big for a comment can you put it in a gist?)
Set the following environment variables: GRPC_TRACE=all and GRPC_VERBOSITY=debug
When you create the subscriber object, can you pass the argument streams: 1? This will reduce the number of messages a bit and make the output easier to follow.

Hey @blowmage. I probably won't have time to today, and I'm about to go on vacation unfortunately. I'll check in on this issue periodically, and @jonparrott may also. The code above should be enough to repro, though - let me know if you're not able to.

And, yessir, a container running on a mac.

Unfortunately, I have yet to successfully reproduce the behavior you are seeing. I have posted my repro files here. The only substantive change I made was to remove the custom SSL certificate and creating the SSL_CERT_FILE environment variable. From my tests so far I don't think a custom SSL cert is needed when running on Alpine Linux, and ultimately I think might be a candidate to explain the behavior described in this issue.
In the directory with my repro files (with all the necessary values added) I build the container which will download and compile the grpc and google-protobuf gems:
$ docker build -t gcps .
Sending build context to Docker daemon  10.24kB
Step 1/9 : FROM ruby:2.5.0-alpine3.7
 ---> 8ea9c590ec75
Step 2/9 : RUN apk update
 ---> Using cache
 ---> 42ff52238df6
Step 3/9 : RUN apk add gcc musl-dev cmake alpine-sdk linux-headers
 ---> Using cache
 ---> d46e1c694df3
Step 4/9 : ENV GOOGLE_CLOUD_PROJECT <project-id>
 ---> Using cache
 ---> 4462f6b89a7e
Step 5/9 : ENV GOOGLE_CLOUD_CREDENTIALS /rb-app/creds.json
 ---> Using cache
 ---> 02978ddc1c1e
Step 6/9 : ADD . /rb-app
 ---> dce64b894f48
Step 7/9 : WORKDIR /rb-app
Removing intermediate container 9cbd0046d810
 ---> 319271cdb5fb
Step 8/9 : RUN BUNDLE_FORCE_RUBY_PLATFORM=1 bundle install
 ---> Running in 34cf2c644335
Fetching gem metadata from https://rubygems.org/........
Fetching public_suffix 3.0.2
Installing public_suffix 3.0.2
Fetching addressable 2.5.2
Installing addressable 2.5.2
Using bundler 1.16.1
Fetching concurrent-ruby 1.0.5
Installing concurrent-ruby 1.0.5
Fetching multipart-post 2.0.0
Installing multipart-post 2.0.0
Fetching faraday 0.14.0
Installing faraday 0.14.0
Fetching google-cloud-env 1.0.1
Installing google-cloud-env 1.0.1
Fetching google-cloud-core 1.2.0
Installing google-cloud-core 1.2.0
Fetching google-protobuf <IP_ADDRESS>
Installing google-protobuf <IP_ADDRESS> with native extensions
Fetching googleapis-common-protos-types 1.0.1
Installing googleapis-common-protos-types 1.0.1
Fetching jwt 2.1.0
Installing jwt 2.1.0
Fetching little-plugger 1.1.4
Installing little-plugger 1.1.4
Fetching multi_json 1.13.1
Installing multi_json 1.13.1
Fetching logging 2.2.2
Installing logging 2.2.2
Fetching memoist 0.16.0
Installing memoist 0.16.0
Fetching os 0.9.6
Installing os 0.9.6
Fetching signet 0.8.1
Installing signet 0.8.1
Fetching googleauth 0.6.2
Installing googleauth 0.6.2
Fetching grpc 1.10.0
Installing grpc 1.10.0 with native extensions
Fetching googleapis-common-protos 1.3.7
Installing googleapis-common-protos 1.3.7
Fetching rly 0.2.3
Installing rly 0.2.3
Fetching google-gax 1.0.1
Installing google-gax 1.0.1
Fetching grpc-google-iam-v1 0.6.9
Installing grpc-google-iam-v1 0.6.9
Fetching google-cloud-pubsub 0.30.1
Installing google-cloud-pubsub 0.30.1
Bundle complete! 1 Gemfile dependency, 24 gems now installed.
Bundled gems are installed into `/usr/local/bundle`
Removing intermediate container 34cf2c644335
 ---> 5ac2e03f31a6
Step 9/9 : CMD ruby /rb-app/app.rb
 ---> Running in 6c40529ccda0
Removing intermediate container 6c40529ccda0
 ---> 94cfc1fc33b1
Successfully built 94cfc1fc33b1
Successfully tagged gcps:latest

Then I run the container:
$ docker run -itP gcps
req
Ruby starting up
Creating new
Sub exists/creates
Sub listen
Ruby receiving..

In a separate local IRB session running on my development machine, outside of docker, I publish a topic and see it printed in the running container's output:
Ruby received<PHONE_NUMBER>7317 hello docker!

I let the process run for the next ~30 minutes, occasionally publishing new topics and seeing the messages printed in the container's output:
Ruby received<PHONE_NUMBER>0861 hello again docker!
Ruby received<PHONE_NUMBER>9010 hello docker, thrice!

After ~30 minutes I see the following printed in the container's output:
#<Thread:0x0000561f9919d598@/usr/local/bundle/gems/grpc-1.10.0/src/ruby/lib/grpc/generic/bidi_call.rb:74 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
	2: from /usr/local/bundle/gems/grpc-1.10.0/src/ruby/lib/grpc/generic/bidi_call.rb:75:in `block in run_on_client'
	1: from /usr/local/bundle/gems/grpc-1.10.0/src/ruby/lib/grpc/generic/bidi_call.rb:158:in `write_loop'
/usr/local/bundle/gems/grpc-1.10.0/src/ruby/lib/grpc/generic/bidi_call.rb:158:in `run_batch': call#run_batch failed somehow (GRPC::Core::CallError)

This is due to an internal error within GRPC. The streaming connection that the Pub/Sub Subscriber uses may fail. GRPC is a little too chatty and has printed the preceding message to stdout. However, the Pub/Sub Subscriber is resilient to these types of failures and has silently recovered and recreated the streaming connection. I publish another message and see it show up in the container's output:
Ruby received<PHONE_NUMBER>4006 hello docker, quad!

So far I haven't been able to get a container running Google Cloud Pub/Sub code on Alpine Linux to fail in any unexpected ways. But I haven't yet fully completed my tests specifying a custom SSL cert. So if a custom SSL cert is necessary for this issue then there may still be a problem somewhere.
I'll post another reply once I've completed the next set of tests.

I should add that I am also using Docker for Mac, Version 17.12.0-ce-mac55 (23011).

AH. -it. It was working fine, but for some reason the STDIN must have been getting unattached. Man. Thank you, you rock.

Yes, great work @blowmage!


[Feature] CSV Format Check

Plugin Name: CSV Format Check
Plugin Description: Validates the format of a CSV file against the expected structure.
Here is the Contributing Guide for Plugin

Can I work on this issue?

can i work on this one issue?

Hi @siddharthsambharia-portkey , I would like to work on this issue. Can you please assign this to me?

So @siddharthsambharia-portkey we have to make the expected json structure generic or else you want like user will give a structure and validate against it ?

@Atharva1723 you can do either or both


Implement Authn lockout/throttle implementation

Right now there isn't anything that detects and responds to repeated failed authentication attempts. What do you think about implementing a lockout policy or throttling system?

no objections in principle. what criteria for lockout do you propose?


Removed debug output

While using the Exasol Go driver I realized that there is a log message which gets printed out to stdout every time when I close the Exasol connection, which is not desirable.
I removed that line in the close function to prevent this extra log output.

@sabitor Thanks a lot for your pull request!
I will edit it to follow our processes and create a release.

@sabitor : your changes are now released: https://github.com/exasol/exasol-driver-go/releases/tag/v1.0.3
Thank you for your contribution!


IEEE-calculator

Allows addition, subtraction and multiplication of numbers written in the IEEE 754-format.
Division still needs to be added, calculation with denormalized values might be a bit off sometimes, still working on that.
TODOs:

Enhance code documentation👀
Add division
Add functionality of different IEEE-functions (for now just F(2, 11, -14, 15, false))
Add functionality of different bit-precisions (for now just 16-bit)

Awesome code! I left a few comments here and there.
If you want, I can also address some of the comments and push the fixes. For example, I'd offer to take care of the entire IEEEFunctionInput part, since it's mostly boring hooking-up stuff. And also involves a bit of knowing how to get Vue to do what you want.

Sure, I'd be glad if you could do that.
I also pushed some fixes to src/math/binary-number.ts, mainly calculation with denormalized values (in IEEENumber.normalizeMantissa).

@peter-fuchs Okay, I pushed some tweaks, hopefully that's helpful.


Adjust provision comment, only generate landing page images if necessary

Follow-up to #27048.

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Ahh, good catch regarding the CORPORATE_ENABLED check.


Question about how to pretrain swin-T on nuimages?

Could you please share the code of how to pretrain swin-T on nuimages? Thanks!
Best

We followed the M2BEV (https://arxiv.org/abs/2204.05088) to pre-train Swin-T on nuImages. You can refer to their papers for more details.


Update azure-pipelines.yml

List of changes
now that we have one main branch and it was renamed, no need for release branch
Types of changes
What types of changes are you proposing/introducing to the .NET client?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] New feature (non-breaking change which adds value to the project)
[ ] Breaking change (fix or feature that would cause existing functionality not to work as expected)

Documentation

[ ] Have you proposed a file change/ PR with appium to update documentation?

This can be done by navigating to the documentation section on http://appium.io selecting the appropriate command/endpoint and clicking the 'Edit this doc' link to update the C# example
Integration tests

[ ] Have you provided integration tests to pass against the beta version of appium? (for Bugfix or New feature)

Details
Please provide more details about changes if it is necessary. If there are new features you can provide code samples showing how they work and possible use cases. Also, you can create gists with pasted C# code samples or put them here using markdown.
About markdown please read Mastering markdown and Writing on GitHub

@laolubenson, I added back the release/* branch in case we will want to use it in the future


Revert "Revert "Remove GH action for deployment""

Reverts statsig-io/docs#2113

🚀 Deployed on https://67002f0165050f94a60a972c--cozy-fox-0defec.netlify.app


missing bower.json file with "main" key and dist directory with build package

could You add to repository bower.json file and dist directory with latetst build package?
bower.json
 ...
 "main": "./dist/angular-chartist.js"
 ...

Somethink like https://github.com/AngularClass/angular-websocket/blob/master/bower.json

What version are you using @rudzikdawid ?

0.4.2
should i downgrade to 0.3.13  or other version?

Those are very old versions. If you want to have bower support you should install v3.3.13. Bower support has been dropped as of version 4.x

Hey @paradox41, any particular reason for dropping bower support?

@alfmatos Sure. The reason is that NPM has become the de-facto package manager for the frontend. Bower (along with Component, JSPM, etc) never really got enough mind share. Most of the packages I use in my day-to-day work use NPM exclusively, FWIW.
Also, you can still get the 4.x line by installing from git. I just will no longer maintain a bower.json

@paradox41 much appreciated. Perhaps for legacy sake adding this to the documentation?
bower install 'https://github.com/paradox41/angular-chartist.js.git#v4.2.0' --save

That sounds good. I can do that

@paradox41 I would, but at this point I'm not sure if that will solve anything, since the bower entry for angular-chartist.js is picking up the latest version regardless if bower.jsonexists.
This affects gulp builds using wiredep (which is what I'm stuck with), as wiredep only picks up  packages that have a bower.json.
I've added bower.jsonand dist using webpack on a fork, but it seems to hit a wall. For some reason the it fails to load the chartist lib.
undefined is not an object (evaluating '_chartist2.default[this.chartType]')
Any thoughts?

Do you have Chartist loaded correctly?

I've added chartist as well throught: bower install chartist --save
Wiredep picks up both:
<script src="../bower_components/chartist/dist/chartist.min.js"></script>
<script src="../bower_components/angular-chartist.js/dist/angular-chartist.js"></script>

I'm loading angular-chartist as a regular dependency:
angular
    .module('myapp', [
        ...,
        'angular-chartist']

Can you add a debugger; in here somewhere? just check to make sure all of these guys are defined?

Right on. _chartist is undefined, as a result of global.chartist being undefined.

So does chartist expose itself as globally available?

From the chartist code, chartist 0.9.8 registers itself as Chartist. So the problem was with the variable naming. Using global.Chartist made it work.
Should this be fixed in your codebase, or something to take up with chartist upstream?

Looks like a bug in my code. I will take a look today.

Released as v4.2.1

@paradox41 Thanks a lot for the wonderful work and support.


Dynamic port binding is not supported

I'm not sure if this is the correct place to open the issue or if it's server-related but here goes:
Upon attempting to open a c# file the Server won't start and I get the following error:
 stderr: ['System.InvalidOperationException: Dynamic port binding is not supported when binding to ...pers.Start (System.Func`1[TResult] action) [0x0001c] in <3dbcd2994b404a60945cf46de9b5765d>:0 ', '']
stderr: ['']
calling :OmniSharpStartServer returns the same Error
I can edit the file just fine, it's just omnisharp not working as expected.
I did first download the roselyn http server by hand, then tried to let omnishard handle that by unsetting the server path variable but the result is identical.
Neovim, version is 0.3.1, Plugins are:

fzf
fzf.vim
omnisharp.vim
supertab
vim-fugitive
vim-dispatch
neomake
vim-multiple-cursors
auto-pairs
vim-gitgutter
vim speeddating
vim-rooter

OS is manjaro linux on kernel 4.14
Any help is greatly appreciated!
Edit: turns out setting omnisharp to use a specific port will solve the problem, prevents me from working on multiple instances however. I'll take this as a temporary solution.

I'm not sure if this is related but a few people have had trouble with the latest OmniSharp-roslyn server. Could you try with the previous version, by installing it from the vim command line like this:
:OmniSharpInstall 'v1.32.1'

@PhilTaken have you done any more experimenting with this? Do you have the latest version of OmniSharp-vim?
The only references I can find to that error message seem to have to do with Kestrel being passed 0 as the port, which indicates that something has gone wrong somewhere. Do you get anything interesting showing up in the python logs? :OmniSharpOpenPythonLog

Closed due to lack of information


Allow for index arrays in ShermanMorrison

This one solves #319 by allowing EcorrKernelNoise to work with index arrays, rather than slice objects. The function that is changed now is quant2ind, which now by default returns an list of index arrays.
The fastshermanmorrison package with Cython code is also compatible with this change, in case anyone is using that. Just upgrade it with pip
Thanks to Bjorn Larsen for providing a first outline of this modification.
TODO:

[x] Modify quant2ind to return index arrays (and slices)
[x] Fix failing tests in test_white_signals
[x] Change _setup_sparse and _get_ndiag_sparse in EcorrKernelNoise to allow for index arrays
[x] Update fastshermanmorrison-pulsar for full compliance
[x] Added unit tests that shuffle TOAs for EcorrKernelNoise

Depends on #352 for codev report

@AaronDJohnson, unit tests for the extra lines are done. As commented on the code, the remaining two lines that aren't covered have never been covered and IMHO shouldn't even exist. I vote to leave those like this


docs needed: how to build custom actions in an ejected runtime

Is your feature request related to a problem? Please describe.
Our documentation only covers how to turn on the included sample action, and does not cover the process of developing the C# class that implements the action or how these work.
Describe the solution you'd like
We need to have a clear document about how to build the custom action class and how it works with the schema loading system.

These docs from the SDK are relevant:
https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-dialogs-declarative?view=azure-bot-service-4.0&tabs=csharp#creating-the-schema-file

@luhan2017 can you provide some instructions to create a class for the custom action? Thanks!
@hibrenda please if you have any suggestions on this?

Thanks for the great work on this, @zxyanliu!!

The updated docs can be found here:
https://github.com/MicrosoftDocs/composer-docs-pr/blob/master/composer-docs/how-to-add-custom-action.md

The content is live.
https://docs.microsoft.com/en-us/composer/how-to-add-custom-action


Last publish failed due to jsdoc error

from https://travis-ci.org/github/amzn/sketch-constructor/jobs/688221385
> node ./scripts/generateDocs.js
/home/travis/build/amzn/sketch-constructor/node_modules/jsdoc-api/lib/jsdoc-command.js:114
      throw err
      ^
JSDOC_ERROR: [
    at ExplainSync.verifyOutput (/home/travis/build/amzn/sketch-constructor/node_modules/jsdoc-api/lib/jsdoc-command.js:112:19)
    at ExplainSync._runJsdoc (/home/travis/build/amzn/sketch-constructor/node_modules/jsdoc-api/lib/explain-sync.js:31:32)
    at ExplainSync.getOutput (/home/travis/build/amzn/sketch-constructor/node_modules/jsdoc-api/lib/explain-sync.js:12:21)
    at ExplainSync.execute (/home/travis/build/amzn/sketch-constructor/node_modules/jsdoc-api/lib/jsdoc-command.js:49:24)
    at Object.explainSync (/home/travis/build/amzn/sketch-constructor/node_modules/jsdoc-api/index.js:29:18)
    at JsdocToMarkdown.getJsdocDataSync (/home/travis/build/amzn/sketch-constructor/node_modules/jsdoc-to-markdown/index.js:131:21)
    at JsdocToMarkdown.getTemplateDataSync (/home/travis/build/amzn/sketch-constructor/node_modules/jsdoc-to-markdown/index.js:101:28)
    at JsdocToMarkdown.renderSync (/home/travis/build/amzn/sketch-constructor/node_modules/jsdoc-to-markdown/index.js:72:23)
    at Object.<anonymous> (/home/travis/build/amzn/sketch-constructor/scripts/generateDocs.js:6:25)
    at Module._compile (internal/modules/cjs/loader.js:1133:30)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>docs:build: `node ./scripts/generateDocs.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>docs:build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR!  Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>preversion script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/travis/.npm/_logs/2020-05-18T04_11_23_162Z-debug.log
[4:11:23 AM] [semantic-release] › ✖  Failed step "prepare" of plugin "@semantic-release/npm"

I have 0 idea how to fix this

Created issues in jsdoc-api: https://github.com/jsdoc2md/jsdoc-api/issues/21

Also fix is on the way


Add missing entry for flex to fixed-requirements.txt and regenerate requirements.txt

Changes in https://github.com/StackStorm/st2/pull/4510 didn't add / correctly update flex in requirements.txt and re-generate final requirements.txt file so requirements file was out of date.

@blag Just a heads up - https://github.com/StackStorm/st2/blob/master/CONTRIBUTING.rst#managing-python-dependencies
In short, fixed-requirements.txt is our dependency lock file and only file where we specify actual dependency versions. requirements.txt is auto generated and shouldn't be updated manually.

I some how missed change in #4552.
Closing this as duplicate and will continue there.


pex can resolve incompatible wheels with the 'u'/UCS4 abi flag

if presented with a find-links repo with deps with a 'u' flag set in the abi field (indicating UCS4/wide unicode), pex can erroneously resolve these wheels in a UCS2 interpreter producing a pex that will break at runtime with unicode errors.
given there's a general, reasonable expectation of buildtime interpreter compatibility vs runtime interpreter compatibility the pex resolver should check for unicode compatibility during resolves and limit its scope to only compatible wheels at the abi level.
repro:
[illuminati unicode_wheels]$ ls -1 test
cffi-1.7.0-cp27-cp27mu-macosx_10_10_x86_64.whl
pycparser-2.14-py2-none-any.whl
[illuminati unicode_wheels]$ pex
Python 2.7.10 (default, Dec 16 2015, 14:09:45)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import sysconfig; sysconfig.get_config_var('Py_UNICODE_SIZE')
2
>>>
[illuminati unicode_wheels]$ pex --disable-cache --no-build --no-pypi -f ./test -vvvvvvvvv cffi==1.7.0 -o /tmp/throwaway.pex
pex: crawling link i=0 link=Link('file:///private/tmp/unicode_wheels/test') follow_links=False
pex: Building pex :: Resolving distributions :: Fetching file:///private/tmp/unicode_wheels/test/cffi-1.7.0-cp27-cp27mu-macosx_10_10_x86pex: Building pex :: Resolving distributions :: Fetching file:///private/tmp/unicode_wheels/test/cffi-1.7.0-cp27-cp27mu-macosx_10_10_x86pex: Building pex :: Resolving distributions :: Translating /private/var/folders/3t/xkwqrkld4xxgklk2s4n41jb80000gn/T/tmpZVta6g/cffi-1.7.pex: Building pex :: Resolving distributions :: Fetching file:///private/tmp/unicode_wheels/test/pycparser-2.14-py2-none-any.whl        pex: Building pex :: Resolving distributions :: Fetching file:///private/tmp/unicode_wheels/test/pycparser-2.14-py2-none-any.whl :: Fetcpex: Building pex :: Resolving distributions :: Translating /private/var/folders/3t/xkwqrkld4xxgklk2s4n41jb80000gn/T/tmpeJR2pL/pycparser  pycparser 2.14ny.whl into distribution
  cffi 1.7.0
pex: Building pex: 49.4ms
pex:   Resolving interpreter: 1.6ms
pex:     Setting up interpreter /Users/kwilson/Python/CPython-2.7.11/bin/python2.7: 1.6ms
pex:   Resolving distributions: 7.5ms
pex:     Fetching file:///private/tmp/unicode_wheels/test/cffi-1.7.0-cp27-cp27mu-macosx_10_10_x86_64.whl: 1.0ms
pex:       Fetching file:///private/tmp/unicode_wheels/test/cffi-1.7.0-cp27-cp27mu-macosx_10_10_x86_64.whl: 0.5ms
pex:     Translating /private/var/folders/3t/xkwqrkld4xxgklk2s4n41jb80000gn/T/tmpZVta6g/cffi-1.7.0-cp27-cp27mu-macosx_10_10_x86_64.whl into distribution: 1.1ms
pex:     Fetching file:///private/tmp/unicode_wheels/test/pycparser-2.14-py2-none-any.whl: 1.1ms
pex:       Fetching file:///private/tmp/unicode_wheels/test/pycparser-2.14-py2-none-any.whl: 0.7ms
pex:     Translating /private/var/folders/3t/xkwqrkld4xxgklk2s4n41jb80000gn/T/tmpeJR2pL/pycparser-2.14-py2-none-any.whl into distribution: 1.3ms
Saving PEX file to /tmp/throwaway.pex

for organizations with control over the contents of their find-links repos, the suggested workaround would be to eliminate all incompatible wheels from the resolve path - but we should fix this generally in pex.

+1

this should now be mostly fixed as of #480.


link to docs in readme is giving 404 error

📓 Documentation Update
What part of documentation was unclear or wrong?
A link to docs in readme is giving 404 error. It should probably link to: https://airctic.com/0.7.0/

Very good catch!! Thanks for reporting. It's fixed now =)


Support Cryptodome

Hey,
I'm trying to use tinytuya through SeriousPython. Everything works well, except that it relies on Cryptodome, which doesn't seem to be packaged well with SeriousPython. I'm getting the following error :

Not found '_raw_ecb_d.cp312-win_amd64.pyd', Not found '_raw_ecb_d.pyd'

Is there any solution ?

What https://pypi.org/ project is that?


GitHub Authentication

Overview
I implemented a feature to log in to Crowi using GitHub account.
Features

Log in to Crowi using GitHub account
Login restrictions by GitHub Organization

Changed

Callback url when registering with a Google Account

before: /register
after: /register/google
It just behaves like a login callback

Screens

Thank you for the great PR.
I'll check it.

@lightnet328 Can you merge current master to resolve conflicts?


Inserted vanilla JS alternative function.

Got an exception when trying to load the script bc I'm not using JS. Put in a try/catch so that doesn't happen anymore.

This just saved me a lot of pain. Thanks! Shame this looks dead and the author no longer cares about it being non-functional without jQuery

Hey Adam! Thanks for the great feedback!
I always like to keep jQuery out of it where I can.
Hope this gets pulled into the project sometime soon ☺


remove custom logger

Pull Request Test Coverage Report for Build 356

15 of 18 (83.33%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.2%) to 87.287%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/js/core/Board.ts
13
16
81.25%

Totals

Change from base Build 354:
0.2%

Covered Lines:
397

Relevant Lines:
447

💛  - Coveralls


Anarchyape Hadoop

Hello David,
I am new to github. I am doing a project regarding failure injection on hadoop cluster. i was trying to use anarchyape  but i am getting few errors while executing the commands can you please give me detailed info about the exact method to inject failures onto the cluster. I am also new to the linux world so please dont mind asking you all these.
i executed the text but there was an error "connection resumed error on eth0 "
please share with  me if you have a detailed documentation regarding the implementation of anarchyape on hadoop.
thanks in advance
Regards,
NAVEEN

Hi Jack,
I am doing some simulation to inject some failure to Hadoop Cluster.
I could locally inject the failure, however, I could not inject them remotely.
I installed pdshand I copied the ape folder to the host where the failure will be executed.
Could you please explain me why the following command is not working properly (although I got the msg "Running Remote Command Succeeded" )
sudo java -jar ape.jar -R "hostX"  -S 100 5
Here is the output of the command
[-R, HOSTX, -S, 100, 5]
[ option: R remote [ARG...] :: Run commands remotely ]
[ option: S network-slow [ARG...] :: Delay all network packet delivery by a specified amount of time (in milliseconds) for a period specified in seconds ]
Mode is remote
List of Hosts:
HOSTX
Command is network-slow
Command Argument: 100
Command Argument: 5
pdsh
-Rssh
-w
HOSTX
'/usr/local/bin/ape
-L
-S
100
5'
echo "pdsh -Rssh -w HOSTX '/usr/local/bin/ape -L -S 100 5' " > 546c01f0-8a9c-459c-8ccc-d764b5434816.sh && chmod +x 546c01f0-8a9c-459c-8ccc-d764b5434816.sh && ./546c01f0-8a9c-459c-8ccc-d764b5434816.sh && rm 546c01f0-8a9c-459c-8ccc-d764b5434816.sh
Running Remote Command Succeeded
Thanks,
Mbarka

i am also facing issue, kindly help me, how can i run my commands, below are the specification of mine
1- Hadoop version 2.7.3
2-single node cluster
3-Ubuntu LTS
a little help will be highly appreciated,

@jdintruff @david78k  kindly help me out, i am using this tool but can't get any help???


[chart/redis-ha][BUG] PodDisruptionBudget applies to test pods

Describe the bug
The PodDisruptionBudget selector matches the labels not only on the StatefulSet but also on the two test containers:

https://github.com/DandyDeveloper/charts/blob/master/charts/redis-ha/templates/tests/test-redis-ha-configmap.yaml#L6
https://github.com/DandyDeveloper/charts/blob/master/charts/redis-ha/templates/tests/test-redis-ha-pod.yaml#L6

To Reproduce
Steps to reproduce the behavior:

Deploy redis-ha with these values:

redis-ha:
  haproxy:
    enabled: true
  labels:
    dns: "true"
  metrics:
    enabled: true
  podDisruptionBudget:
    # maxUnavailable: 1
    minAvailable: 2
  replicas: 3

Check the pod labels:

$ kubectl get pods --show-labels | grep redis-ha
samproxy-redis-ha-configmap-test             0/1     Completed   0          1h    app=redis-ha,foo/app=samproxy,release=samproxy
samproxy-redis-ha-server-0                   2/2     Running     0          1h    app=redis-ha,foo/app=samproxy,release=samproxy,samproxy-redis-ha=replica
samproxy-redis-ha-server-1                   2/2     Running     0          1h    app=redis-ha,foo/app=samproxy,release=samproxy,samproxy-redis-ha=replica
samproxy-redis-ha-server-2                   2/2     Running     0          12d   app=redis-ha,foo/app=samproxy,release=samproxy,samproxy-redis-ha=replica
samproxy-redis-ha-service-test               0/1     Completed   0          1h    app=redis-ha,foo/app=samproxy,release=samproxy

Check the PodDisruptionBudget:

$ kubectl describe pdb samproxy-redis-ha-pdb
Name:           samproxy-redis-ha-pdb
Namespace:      opentelemetry
Min available:  2
Selector:       app=redis-ha,foo/app=samproxy,release=samproxy
Status:
    Allowed disruptions:  1
    Current:              3
    Desired:              2
    Total:                5
Events:                   <none>

Expected behavior
The total pods for the budget should be 3, not 5.
Additional context
We use kapp to deploy helm templates. I have a possibly related feature request for custom labels: #40

@leonsp-ai Thank you for the PR. Just waiting for chart bump!


Seperate mount volume for main container and sidecar container

Reproduce step:

enable logging
create a workload, add volume, select read log file from path, input log path, and save.
create a sidecar container, add volume, UI will show a not path volume for the sidecar, which is the volume in step2, expect show nothing, if user want to add a flexvolume log for the sidecar container, he could add a volume for it, not share the volume with the first one, because the log will mess up, if user input same path log for the main container and the sidecar.

@loganhz did you assign to @aiwantaozi to test it?

Yes, I have pinged her on slack.

@aiwantaozi to validate

Verify pass.
Test step:

create a workload add log dir volume
add a sidecar container to that workload and also add a log dir volume
add pvc volume to that workload
above volume are create correct in API.
/close


Wall Messages

Suggest a new feature where it's possible for the users of abapGit to define "Wall Messages", on the repository overview screen, and for each repo. As a starting point just simple text or HTML?
This text can help to explain the policies of using abapGit, who is responsible for updating abapGit in the system, who uses it etc.

I can see that being useful. Things that you don't need (to read) every day should be hidden by default or have a user option to click and confirm it.
For repos, I would rather see us go for markdown and the option to display the readme on the repo view.

markdown: then we need to ship a markdown renderer, which could be 1000s of lines, just for this smaller feature

Regarding readme #1102

this is not a readme, its a per system text, which can be set by users, for internal purposes

Yeah, I like the idea. Just linked the issue because Marc mentioned displaying the readme.

I would be happy with a short text provided by the user exit:
This is a production instance of abapGit, feel free to fix bugs but do not attempt to experiment with the source code. If you want to develop abapGit, use your private system from ABAP Trial in Docker.

Visit "https://our.corporate.url/for/more/details".

Or on repo level
This repository is updated/pulled by<EMAIL_ADDRESS>please contact me if changes are required, bugs, comments, questions etc.

@jfilak something transportable, eg. a user exit would work best for you right?

Yep, sure, a new method in the existing user exit would be enough. I would already do that, if it was really annoying me. Right now, every once in a while, I have to send an angry email to someone who thought it's a great idea to do this and that with abapGit in our development systems.

@jfilak see #4653


Implicit internal processing option of the code status

<EMAIL_ADDRESS>option makes client to resolve status failures from the server and return them in error value. It looks transparent if the method call happens "next to" client initialization. However, when calling the method in a separate function around the client, it is not clear whether the status code should be resolved, or whether it is already in error. The only workaround I see is to pass the resolving flag to such functions and carry it everywhere with you. It is proposed to strengthen the resolution of statuses at the type level in order to avoid double work or unexpected results.

So the issue is client's modality wrt this option? Because the other way to see it is that this option can be changed on an already existing client, but it looks like this can't happen (PrmInit is copied).

Can be solved with raw/usable client split, but it works as of now as well, just needs a little more care from the SDK user. So not a top priority.

So basically the issue may be dropped in favor of #406?

Will be solved by #406.


experimental transform types

added --experimental-strip-types support in required 22.6.0 node version
client cannot use ts files so js files kept there
server files moved to only use ts

diabled js emit for serve and build

use ESM syntax for built and serve file by adding "type": "module" in package.json
use built in Record Type
update TS to 5.5 which was a big update to improve inference

I personally don't really understand what exact specific problem this is solving except "let's just using this shiny new thing"

I agree. it does feel nice to use the "shiny new thing". but my reasoning is that I think it makes more sense to avoid the build step and have less files needed to run this.
it can also avoid having the issue of accidentally forgetting the build step since you are running the actual ts file directly.

updated to support a newer flag that has less limitations

yes. this is only a node feature. so browsers (the client) cannot use ts files yet. but the server can do it.
so yes. we aren't completely avoiding the build step. it just makes it a shorter step and results in fewer files

So I don't see any point in introducing this then. Especially since once everything is rewritten in C3 we gonna completely remove TypeScript and just be left with a small amoutn of JavaScript.


IfNoneMatch on GetObject fails when ETag is enclosed in quotes

Hi there,
AFAICS the noneMatch (and match) check in com.adobe.testing.s3mock.service.ObjectService#verifyObjectMatching always fails if the supplied ETag is enclosed in quotes (which is a requirement if I understand https://github.com/adobe/S3Mock/issues/801 correctly), i.e., the GetObject operation never returns NOT_MODIFIED.
Thanks,
Thilo
P.S.: Tested w/ version 2.6.1

thanks for the report. It seems that our etag handling is still inconsistent.
There also does not seem to be a unit or integration test for the "if (not) modified since" use-case...

This turned out to be more complex than I thought - will have to try out a few more things locally and against a real S3 object...
Read the RFCs
https://www.rfc-editor.org/rfc/rfc2616#section-14.19
https://www.rfc-editor.org/rfc/rfc7232
which seem to be partially implemented by the S3 backend.
Noticed several challenges for this fix:

There is only one integration test for the ETag behaviour, and it makes the wrong assumptions: GetPutDeleteObjectV1IT#shouldCreateAndRespectEtag
There are no unit-tests for ETag functionality
Seems like AWS SDK v1 and v2 handle ETags differently.
For example, when doing a PutObject in v1, the enclosing quotes are stripped out when retrieving the ETag value from the response. If the value should be used in a subsequent request against S3, you need to add the quotes again.
When doing the same in v2, the PutObject response ETag still contains the enclosing quotes, so you can immediately pass that value to a subsequent request.
v1 requests can pass a list of ETags for "match" / "none-match", while in v2 it's only a single value.
We will have to support both.
There are a few methods defined in the API that (for example HeadObject) that support "match" / "none-match", which are not correctly implemented in S3Mock.
It is not possible to add "match"/"none-match" to PutObject requests although according to the RFCs it should to prevent the "lost update" problem...
Weird, do they expect the client to make a HeadObject call first?
S3Mock would need to support the "*" use-case as well, where this would match any Etag for an existing object, for both the "match" and "none-match" use-cases.

And there are probably several other things I forgot to write down here...

@ginkel just released 2.6.3 fixing ETag handling yet again - hopefully for good!

@afranken Works like a charm, thank you very much! :-)


Add shim cgroup support for v2 runtimes

This also has a change where task opts are passed to shims via STDIN for early inspection.

Codecov Report

Merging #3292 into master will decrease coverage by 0.02%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #3292      +/-   ##
==========================================
- Coverage    44.4%   44.38%   -0.03%
==========================================
  Files         113      113
  Lines       12231    12237       +6
==========================================
  Hits         5431     5431
- Misses       5966     5972       +6
  Partials      834      834

Flag
Coverage Δ

#linux
48.3% <0%> (-0.03%)
:arrow_down:

#windows
39.64% <0%> (-0.03%)
:arrow_down:

Impacted Files
Coverage Δ

runtime/v2/shim/util.go
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cafda1c...fe6a2b0. Read the comment docs.

LGTM

LGTM
The behavior that TaskOptions completely overwrites RuntimeOptions makes it a bit hard to understand and use though.

The behavior that TaskOptions completely overwrites RuntimeOptions makes it a bit hard to understand and use though.

Is the interaction between containerd and the shim binary documented anywhere?

It has always overwritten these options.  Task specific runtime opts take precedence over runtime configured options.


Update application-gateway-web-app-powershell.md

Added missing $rg variable

@stephenweinrich : Thanks for your contribution! The author, @vhorne, has been notified to review your proposed change.

#sign-off


overlapFeatures double counts in sum operation

Calling overlapFeatures sum on a sketch collection can return doubly-counted features without an option to enforce single counting.
For example in Bermuda, three small sketches fall inside of a feature which represents 15 shipwrecks. Each sketch is assigned as containing 15 shipwrecks. The sketch collection triple counts these 15 shipwrecks and reports 45 shipwrecks.

(The small shape at the bottom is 3 sketches. NumberofRe is the parameter of number of shipwrecks.)

Solution:  Track unique id of features to ensure each feature can only be counted once in the sum.

This is probably a bug. I can't see any reason would want to double count.  For computing metrics for the collection as a whole, overlapFeatures is designed to union the sketch collection to get rid of overlap.  It doesn't seem to be doing that properly.  I also don't see that the unit tests include a test for sum

At first glance, it's looping through each of the unioned result, and is simply incrementing the count if there's any intersect.  It probably needs a way to keep track of whether a feature has already been counted.  Maybe using an unique ID property that gets added to the unioned results features first

Err, ID that gets added to the shipwreck features I guess I meant

Agreed, sounds good. I'll be adding this in project space for Bermuda since it's behind in GP releases, then can make the edit with tests in the framework.

yeah that's probably the right approach

Jsyk, the union of sketches is working properly, these sketches don't overlap which is why they aren't dissolved together.


Jordan/validation check

This updates user input to be validated in:

frontend
route
database methods

closes #98


Rubocop Shows bash Color Escape Sequences in Output Files

I just tried running the following command with rubcop 0.34.1
rubocop --format simple --out issues.txt

and got the following output in the text file
C: 39: 32: Use ESC[33m&&ESC[0m instead of ESC[33mandESC[0m.

I believe I found where this bug is occurring and can submit the fix I mention below if appropriate.  Running rspec shows a few failures/pending tests, none related to this change, and a quick test on a real file produced the correct results.
From lib/rubocop/formatter/simple_text_formatter:
class SimpleTextFormatter < BaseFormatter
  include Colorizable, PathUtil, TextUtil

...

def annotate_message
  msg.gsub(/`(.*?)`/, Rainbow('\1').yellow)
end

In the same file, the file name/path is output in yellow using a separate syntax mixed in through Colorizable:
def report_file(file, offenses)
  output.puts yellow("== #{smart_path(file)} ==")
  ...
end

Proposed fix:
# this
def annotate_message
  msg.gsub(/`(.*?)`/, Rainbow('\1').yellow)
end

# to this
def annotate_message
  msg.gsub(/`(.*?)`/, yellow('\1'))
end


[fix] Add required commons-lang dependency

Fixes #74

@aheritier I fixed by removing commons-lang requirement.
It's released in 3.1.4.


Unresolved reference BuildConfig when upgrading to 1.5.0

Hi, I've been using this library using AGP 4 and Root Coverage Plugin 1.4.0. But after I update the AGP and kotlin version to this version (see version below) I got an error that said Unresolved reference: BuildConfig. The unresolved BuildConfig causes an error when I generate jacoco report or assembleDebug the app. Fyi I saved the value of BuildConfigField at shared gradle and apply it to each module, is that affect this issue?. Also, when I downgrade to 1.4.0 the error is gone, generating a report is successful but causes [ant:jacocoReport] Execution data for class 'xxx'  does not match.
This is the version:
AGP: 7.2.0
Kotlin Gradle Plugin: 1.6.21
Jacoco: 0.8.6

This is the stackrace:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':core:compileDebugKotlin'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:145)
        at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:282)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:143)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:131)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)

@MirzaUkas Thanks for sharing this.
For some reason I'm doubting that this has something to do with this plugin directly, since it does not by itself manipulate classes. It could be JaCoCo related but I also doubt that, since the version did not change. Maybe this has something to do with the result filtering that the plugin applies? It does (by default) filter out BuildConfig from the final report. But filtering on report level should not affect actual shipped code, and this filtering has not changed in version 1.5 compared to 1.4.
By pure chance I also experienced this error in one of my own projects, about a week ago. Back then I figured it must have had something to do with the AGP upgrade to 7.2.0.
Anyhow, I need a bit more time to investigate what exactly is going on.

Did some digging, I found that I can no longer reproduce this when I remove this line:
https://github.com/NeoTech-Software/Android-Root-Coverage-Plugin/blob/0b28266b2737d10adaa3804ad9a056caeb1ef992/plugin/src/main/kotlin/org/neotech/plugin/rootcoverage/RootCoveragePlugin.kt#L179
But obviously this line is needed for code coverage to work. So while it fixes the BuildConfig problem, it breaks code coverage.
I suspect the issue is in the variant.sources.java.all part. This is a new public but unstable API to get the Java source locations. Release 1.4 of this plugin was using an older (unofficial) method to get the Java sources. For some reason calling this API results in the BuildConfig file to disappear. I updated to AGP version 7.3.0-beta01 and by doing so the problem was gone.
Conclusion:
variant.sources.java.all is broken in AGP version 7.2 and causes class files to disappear (not generated?). In AGP 7.3.0-beta01 this issue is not present.
Solution
There are 2 things possible, the quick and easy is to update to AGP version 7.3.0-beta01 but not everyone might be able to do so. The other (more difficult) solution is to create a hot-fix for this release where I change to code so it uses the old (unofficial) method to get the Java sources. However this is not as easy as one might think, since in release 1.5 I changed the whole plugin setup to use official APIs only.
TLDR: This is potentially a bug in AGP 7.2, I'm looking to workaround this.

Thanks a lot, @Rolf-Smit for digging into this issue and for your insight.
I got it, so it's still a bug in AGP 7.2 and the quick solution is to upgrade the AGP version 7.3.0-beta01. But like you already said, not everyone can do it and I can't do it because my project show this The project is using an incompatible version (AGP 7.3.0-beta01) of the Android Gradle plugin. Latest supported version is AGP 7.2.0.
So yeah I'll wait for the update for gradle 7.3 or maybe next time you find another solution.

@MirzaUkas that is strange, higher AGP versions should be allowed (I guess I will add some test cases for that check then).
Anyhow, I'm planning to hot-fix this today (by reintroducing some old and deprecated AGP code, since that seems to be working in AGP 7.2).

@MirzaUkas I just released version 1.5.1 which contains a workaround for this issue. Can you test on your side as well and confirm?

After upgrading to 1.5.1, it works like a charm! Thanks @Rolf-Smit


Release new version

Hey @cdaringe! So I went ahead and merged all the greenkeeper PRs which updated all the ampersand- dependencies that got updated to Lodash v4.
It ended up not being as straight forward as I thought, so I thought I would open this issue for visibility and to give you a chance to review and then publish/version as you see fit.
Hopefully I didn't step on your toes too much. I was just trying to use my chunk of time to get as much Lodash v4 stuff done as possible.

@lukekarrys, no sweat. no toe-stepping at all!  it's as much your's as mine as anyone-else's :)
will publish soon


Langchain stripping out emojis when using Llama2 via llama_cpp

System Info
Langchain Version: 0.0.263 (latest at time of writing)
llama-cpp-python Version: 0.1.77 (latest at time of writing)
Python Version: 3.11.4
Platform: Apple M1 Macbook 16GB
Llama2 Model: llama-2-7b-chat.ggmlv3.q4_1.bin via https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML
Who can help?
@hwchase17 @ago
Information

[X] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[X] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[X] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction
I am trying to use the Llama.cpp integration to use the Llama2 model. I am battling with how the integration handles the output of emojis, which I assume is some kind of Unicode issue. However, I don't see the exact same issue when using llama-cpp-python directly (although there does seem to be some kind of issue with how llama-cpp handles emojis as well...).
Here are the two simple tests for comparison:
Langchain Llama-Cpp Integration
from langchain.llms import LlamaCpp
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

prompt = """
[INST] <<SYS>>
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe.
<</SYS>>

Tell me a story about a cat. Use as many different emojis as possible.[/INST]
"""

llm = LlamaCpp(
    model_path="models/llama-2-7b-chat.ggmlv3.q4_1.bin",
    n_gpu_layers=1,
    n_batch=512,
    n_ctx=2048,
    f16_kv=True,
    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),
    verbose=True
)

llm(prompt=prompt)

Provides the output (truncated for the sake of brevity)

Once upon a time, there was a curious  cat named Whiskers. She lived in a cozy  house with her loving  family. One day, while lounging in the sunny  garden, she spotted a  fascinating fly buzzing by.

This output has a complete lack of emojis, but I expect they were supposed output in the locations where there are double spaces, for example, after the words curious, cozy, loving etc (I have other examples below that are not just white space)
Llama-cpp-python
from llama_cpp import Llama

prompt = """
[INST] <<SYS>>
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe.
<</SYS>>

Tell me a story about a cat. Use as many different emojis as possible.[/INST]
"""

llm = Llama(
    model_path="models/llama-2-7b-chat.ggmlv3.q4_1.bin",
    n_gpu_layers=1,
    n_ctx=2048,
    n_batch=512,
    f16_kv=True,
    verbose=True
)

llm(prompt=prompt)

Provides the output (again, truncated for brevity):

😺🐱💤 Once upon a time, there was a curious 🐱 named Fluffy 😻. She lived in a cozy 🏠 with her loving owner, Lily 🌹

Other Output Examples
I have a significantly longer and more complex prompt in our application, that I've not included here, but creates some odd output with regards to emojis.
Here is using Langchain (exact same code as above, just a different prompt):

Hey Paddy! Great job meeting your run pledge yesterday! Keep up the good work  Your consistency is on point ♂

(There is a problem pasting the text directly, so here is a screenshot to see the odd unicode char being output):

Notice the <0x200d> and male symbol, as well as the double spacing, again, where I suspect there were supposed to be emojis output.
 Met your run pledge 🏃\u200d♂

The same unicode chars are output, so I suspect that issues is coming from the underlying model and or llama-cpp and may well be a red herring in this context.
However, all other emojis are somehow being discarded as part of the output.
As a final comparison, here is another simple prompt to try:

How can I travel from London to Johannesburg. Use only emojis to explain the trip.

Langchains response is simply an empty string:

''

Llama-cpp has the following response for the same prompt:

🇬🇧🛫🌏🚂💨🕰

Expected behavior
Expected behaviour here is for langchain to not strip out emojis as part of the response from the underlying model. We can see that when using llama-cpp directly we get a (mostly) well formed response but when using the langchain wrapper, we lose said emojis.

I've been doing some testing and am able to resolve the immediate issue by setting streaming=False when declaring the llm.  Always answer as helpfully as possible, while being safe.
<</SYS>>

Show me a list of fitness related emojis.[/INST]
"""

llm = LlamaCpp(
    model_path="models/llama-2-7b-chat-ggml-model-q4_0.bin",
    n_gpu_layers=1,
    n_batch=512,
    n_ctx=2048,
    f16_kv=True,
    verbose=True,
    streaming=False
)

Gives me this output:

Of course! Here are some fitness-related emojis that you might find useful:\n\n🏋\u200d♂ Man doing push-ups\n🏋\u200d♀ Woman doing yoga\n🏃\u200d♂ Person running\n🚲 Bicycle\n🏙 Cityscape\n🏞 Nature scenery\n💪 Weightlifter\n🏆 Trophy\n🎯 Target\n📈 Bar graph\n👍 Thumbs up\n😃 Smiling face\n🤩 Excited face\n\nI hope these emojis help you in your fitness journey! Remember to always prioritize safety and consult with a medical professional before starting any new exercise program. 💪

The problem remains with the u200d♂ being output, but I think thats an upstream issue from the llama-cpp-python library. I've logged an issue there at https://github.com/abetlen/llama-cpp-python/issues/608
This leads me to believe that the original issue of the emojis being stripped entirely relates to the GenerationChunk class.
It may be unrelated, but in the comment https://github.com/ggerganov/llama.cpp/issues/2231#issuecomment-1646723003, it would seem that, "found that some multi-byte characters are split into multiple tokens which was breaking the encoding. ", which may mean the chunking happening during the streaming process is not joining up the multi-byte chars correctly and the longer unicode chars, like emojis, are not being output correctly.
So although, for my use case, the issue is resolved by setting streaming=False, the original problem remains.

Which way is the best practice for llama2 to use LlamaCpp and HuggingFacePipeline? Do these two classes support streaming and streamingstdoutcallback?


Disable click on internal textview

Is it possible to disable the click event on the Internal textview?, I'm using this view on a list and the users can't click on the area where the markdown text is added. If possible the consumer should be able to control events with the clickable modifier or having a click block that receives the text on the clicked area.

Hello @ale-groundcloud,

I'm using this view on a list and the users can't click on the area where the markdown text is added.

Are you using links and the click doesn't work?

It's the opposite, there are no links and the textview consumes them, not allowing the event to go to the view behind it

Got it!
I guess these changes will solve it.
https://github.com/jeziellago/compose-markdown/commit/8cfa5a531038f2f65237e32add93aadb19d21dc6

Try it with 0.2.8

you can check by adding the view to a clickable parent. When you click on the textview area, the event will be consumed and not sent to the parent.

@ale-groundcloud
Try the version 0.2.9. I found the problem and did solve it.
I hope it helps you.


Add logging during setup process and ensure we're reacting to start hook

Overview
We need to react to the start event for _set_up_discourse rather than the install event, because if a container is restarted it won't get an install event.
Add logging to the _set_up_discourse method so it's easier to see at what point during the process this is failing. In production (likely due to poor disk IO related to https://bugs.launchpad.net/juju/+bug/1989004, still to be confirmed) we're seeing issues where if containers are restarted and we can end up with the set up process failing and so discourse never being started.
This only seems possible if we enter the _set_up_discourse method, relations are ready and we can connect to the container, but then somewhere later on in this same method we fail to complete the setup.
This should help us track down where that's happening.
Checklist

[x] The charm style guide was applied
[x] The contributing guide was applied
[x] The changes are compliant with ISD054 - Managing Charm Complexity
[x] The documentation is generated using src-docs
[x] The documentation for charmhub is updated.
[x] The PR is tagged with appropriate label (urgent, trivial, complex)

This is probably blocked by a dependency issue that I'm trying to solve here: https://github.com/canonical/discourse-k8s-operator/pull/125

I have the same issue locally as https://github.com/canonical/discourse-k8s-operator/actions/runs/6632062737/job/18039459125#step:13:139
But not when I switch back to 'install' instead of 'start'