Initial interning implementation

This is just a rough draft so I can make sure I'm going in the right direction.
The way I designed it, you can change (at runtime) the max str len and whether the interning is enabled or disabled.
In addition, there's a disabled Cargo feature to completely wipe out the implementation at compile-time, so there's 0 overhead.
The reason I went with a disabled feature (and not an enabled feature) is that I expect the interning to be enabled by default.
The cache size is hard-coded at 1,024 elements (I just guessed at what seemed like a reasonable number), but we can make that configurable later.
I took a look at the existing LRU crates, and I decided on uluru because it seemed to be the fastest and lightest weight (based on looking at the source code).
@alexcrichton does this seem reasonable?

I think this is a good start yeah but I'm personally mostly interested in how this hooks up into the wasm-bindgen crate and CLI moreso than the cache itself. It seems like we can easily go hog wild with configuration settings and implementations of the cache but hammering out the details of how the CLI/ABIs work I think will be pretty important too

Right, I'm just taking it one step at a time. The next step is to hook it into the #[wasm_bindgen] macro.

mk makes sense!
I figure this may make the most sense on the wasm-bindgen crate itself where a feature would basically just change the implementation of IntoWasmAbi for strings?

Yeah, that's what I was thinking. Basically, have IntoWasmAbi call the intern::str function.
And then wasm-bindgen could have a disable-interning feature which disables the interning for the wasm-bindgen-cache crate.

Okay, so because of circular dependencies I can't make wasm-bindgen-cache into a separate crate, so I had to move it into wasm-bindgen.
Here are the profiling results. First, without interning:

Now with interning:

The performance went from 1256.8ms to 1108.9ms, an improvement of 147.9ms! As you can see from the charts, decoding went from 387.2ms to 67.0ms.
Unfortunately, the increase wasn't as big as I was hoping for. I need to investigate why the cache_str function is taking up a whopping 154.3ms.
By the way, I had to implement OptionIntoWasmAbi for JsValue, and I'm not sure if the implementation is correct.

Okay, so the issue is that it was just trying to cache too many useless strings, so I took a different approach.
Rather than automatically caching all strings, instead the user has to manually call an intern("foo") function, which has the signature fn intern(s: &str) -> &str
This will add the string to the cache, and so when the string is sent to JS, it will then pick it up from the cache.
By carefully choosing which strings go into the cache and which don't, I was able to get incredible performance gains.
With interning completely disabled: (1363.8ms)

With interning enabled but not used: (1485.0ms)

With all strings interned: (1403.1ms)

With specific strings interned: (915.0ms)

So in the end, by interning only the strings which are commonly used, it gained 448.8ms in performance!
And as you can see in the last chart, decoding is very fast (69.8ms), and the cache itself is also very fast (37.5ms).
In fact, because everything is so fast, the top 7 items are all DOM built-ins, which means the app is going as fast as possible, and the browser is now the bottleneck.
 Amazing.

Sorry I've been pretty busy but this all looks great to me! Some high-level thoughts I might have are:

I think we'll probably want to have this as an off-by-default feature of wasm-bindgen. Anything on-by-default is very difficult to turn off since wasm-bindgen is so heavily depended on, and this also has some code-size impact with the interning infrastructure. Having it as a switch is perfect but I think the better default will be off-by-default. I also think that having the switch being disable vs enable makes it sort of go against how Cargo features work and may lead to surprising results.

I'd ideally prefer to avoid impl OptionIntoWasmAbi for JsValue for now if we can, would it be possible to just do raw bits under the hood to avoid adding the impl just yet?

Could the main test suite be run on CI with interning explicitly both enabled and disabled?

Anything on-by-default is very difficult to turn off since wasm-bindgen is so heavily depended on

Actually, any crate in the dependency graph (including the root/application crate) can very easily turn it off, they just need to use features = ["disable-interning"]

I also think that having the switch being disable vs enable makes it sort of go against how Cargo features work and may lead to surprising results.

I don't think so. It's true that the typical way of doing things (using default-features = false) is broken and almost unusable, but I'm not doing that.
And there isn't an "enable-interning" feature, so it's not possible for there to be any conflicts. And the API is exactly the same between the interning and not interning modes, so there's no compile errors.
So could you explain what sorts of surprising results you think might happen?

would it be possible to just do raw bits under the hood to avoid adding the impl just yet?

Sure, but what bits should it use? Maybe it should just return JsValue::UNDEFINED?

(Originally I intended to have full automatic string interning on by default, but after seeing the benchmarks it's clear that manual interning is much faster, so I'm okay with making the feature off-by-default, but I still want to know why you think it would be problematic)

Ah yeah so I was halfway done writing my original comment before I realized the feature was to disable, not an on-by-default feature which enabled. So to be clear if the feature were enable-intern and it was on-by-default, I don't think that's tenable since it could effectively never be turned off.
The feature is, however, off-by-default and instead worded as disabling interning. Cargo features are meant to be an additive set, however, due to the unioning behavior to compile crates only once. This means that if a crate relies on interning for reasonable performance, another crate may actually disable interning and the original crate has no course of action. In other words, the "disable" form is removing a feature rather than adding it which goes against how shared dependencies work in Cargo today.
That, plus the benchmark data, is basically why I think I'd prefer if this was an opt-in feature rather than on-by-default.
For returning JsValue::UNDEFINED, I think that'd work yeah!

So to be clear if the feature were enable-intern and it was on-by-default, I don't think that's tenable since it could effectively never be turned off.

Right, I learned that the hard way with futures-preview, where you cannot turn off their features at all, not even with default-features = false

This means that if a crate relies on interning for reasonable performance, another crate may actually disable interning and the original crate has no course of action.

That's absolutely true, but I don't think that's a problem, in fact I think that's desirable.
I don't see any reason why a crate would want to disable interning, unless they are the root application crate.
And in that case the root should definitely have the power to disable interning (if they find that it improves the performance of their app).
So, although it's quite rare in general, I think in this very specific case a "negative feature" can be appropriate.

That, plus the benchmark data, is basically why I think I'd prefer if this was an opt-in feature rather than on-by-default.

Yeah, I've come around to that as well.

Ah ok well in any case, want to update this to opt-in and we can go ahead and merge?

@alexcrichton So, while working on this, I noticed something: the IntoWasmAbi impl for JsValue uses mem::forget, so doesn't that mean that the object will stay in the JS heap forever, and never be cleaned up?

Oh right yeah I was thinking about that when reading this but forgot to say we should update it. I think that this is always returning (effectively) &JsValue, right? If str and String report ANYREF that may mean that &str is ok (since it'll show up as REF ANYREF) but for String I think we'll need to insert the REF there ourselves.

If the string exists in the cache, then yeah it can return &JsValue (which is fine).
The problem is that if the string isn't in the cache then the into_abi method uses JsValue::from to convert the &str into a JsValue, and then calls into_abi on that.
And since JsValue leaks (at least, if I'm understanding it correctly), that's quite bad. Maybe it's best we discuss this on Discord, since it seems to have some pretty deep implications?

Ah that's a good point! I'll be around sort of only spottily this week but feel free to ping me when you're around and we can hope it aligns! I'll be around-ish 9am-5pm CST.
Otherwise though it's true we don't have any precedent for returning a "maybe owned" JsValue from any APIs. That being said it doesn't seem entirely unresonable to invent one for strings. In the anyref world it will have no cost because the apis will still take anyref at the boundary, and for now it just means a bit more JS glue. I would imagine that the high bit (or something like that) would indicate whether it was owned or not and whether free needs to be called.

Now that I can build wasm-bindgen-cli, I was able to actually test this.
So, I significantly changed the implementation (before your review). Now the implementation is almost identical to the current string impl, except if the ptr is 0, then it assumes that the len is a pointer to the JsValue.
What that means is that there is now basically no performance difference regardless of whether interning is enabled or not:
Without interning:
   interned   0.016361628484616742 ms
   normal     0.016440500294726636 ms<PHONE_NUMBER>4487818 %)

With interning:
   interned   0.006101373767490259 ms
   normal     0.017310679540981640 ms<PHONE_NUMBER>763148 %)

~183% increase in performance sounds pretty great to me. And it doesn't regress the performance when not interning (which was a big problem with the old implementation). And the memory leak has been fixed.

I rebased, and I also created a benchmark with the 1,000 most common English words. These are the results:
No interning (113,556 bytes):
   normal str            0.015406891304873488 ms

LRU Cache (139,951 bytes):
   interned str (hit)    0.001420249078550068 ms
   normal str            0.015683253470461032 ms  + 1004.2607742067371 %
   interned str (miss)   0.065211217786988060 ms  + 4491.5338916158410 %

BTreeMap (119,950 bytes):
   interned str (hit)    0.0021928430482934585 ms
   normal str            0.0148238589445778490 ms  +<PHONE_NUMBER>735763 %
   interned str (miss)   0.0159713026124058200 ms  +<PHONE_NUMBER>373748 %

HashMap (121,198 bytes):
   interned str (hit)    0.0017783651625431442 ms
   normal str            0.0157200559689065660 ms  +<PHONE_NUMBER>344038 %
   interned str (miss)   0.0159029148265826120 ms  +<PHONE_NUMBER>288807 %

The cache is full and it has a lot of similar words, so it's a reasonably accurate stress test. In practice it will be a lot faster.
The interesting things to note:

The LRU cache is actually the biggest in terms of file size, which is surprising. Maybe it's because it statically allocates the entire cache upfront?

The LRU cache is indeed slower during misses (about 4.5x slower). Not too bad for the worst-case situation, but still not good.
I tested it on lighter work-loads and it didn't have any cost during misses, so this is specific to heavy work-loads.

Meanwhile, the BTreeMap and HashMap (from the Rust stdlib) perform exceptionally well: misses are the same performance as not interning.

On the other hand, the LRU cache is fastest for hits (10x faster). The BTreeMap and HashMap are very slightly slower on hits, but are much faster on misses.

Given the results, I switched it to use HashMap. This gives a 7.8x speed boost for hits, no speed penalty for misses, infinite scalability, and smaller file size than the LRU cache.

I just added in docs for the intern function. The only thing left is adding in CI testing for the enable-interning feature, but I'm not sure how to do that.

Also, the CI failures seem really weird, I have no clue what's going on with those.

For the cached case though we'd need to check both the pointer and the length to see if they're both zero.

I don't think there will be any issue: it's guaranteed that non-undefined JsValue will never have 0 as their idx, so the length will only be 0 if it is in fact None.
I agree that we should have tests for it.

Wait, nevermind, it doesn't support receiving cached strings in the first place, they use completely different systems, so there shouldn't be any problem with OptionFromWasmAbi.

Oh I'm mostly worried about this block which handles outgoing optional slices (including strings). That's only checking {ptr} when to handle cached strings it'd also need to handle {len} being zero

Oh I'm mostly worried about this block which handles outgoing optional slices (including strings).

That code path shouldn't ever be called. When caching is enabled, it will always go through NonstandardOutgoing::CachedString for all types (&str, String, Option<&str>, and Option<String>).

Okay, I think this is ready for merging. The CI failures seem spurious, and I added in CI tests for enable-interning.

Works for me, thanks again for pushing on this!


Google maps and 'Find your code' tool create short codes which aren't decodable offline

I am developing a weather forecasting smartwatch watchface, and I'd love to be able to give my users a way to enter the location manually. I thought of plus codes, thinking that with a support of Google, I could easily direct people on obtaining a code for their desired location.
However, I found out that even though Codes should be able to be generated and decoded offline. is one of reasons Open location codes were created (see Evaluation of Location Encoding Systems), both Google maps and Find your code tool generates codes which aren't decodable offline.
I would like to add a field that would display a full location code to a Find your code tool - and preferrably an option to generate a full code into Google maps.
Alternatively, an offline place database should be added to https://github.com/google/open-location-code decoders, and there should be a documented deterministic way of generating a city from a location, so that a developer can be sure all generated location codes will be decodable with a given offline database.

Hey @tkafka,
Full plus codes (e.g. 87G7MXQ4+M5) are fully decodable offline. You can already find the full code in the sidepanel here, although Maps does not currently display the full code.

Hi @zongweil, thanks, I know, but this issue is about the fact that neither Google Maps, or open-location-code own tool doesn't generate the full plus codes. Currently there is no easily linkable way to a tool which would generate them, and currently generated short codes are undecodable without a planetary place database. I consider this a breach of a promised purpose of these codes.

Did you click my link above? The plus.codes map has the full code in the sidebar.

Alternatively, an offline place database should be added to https://github.com/google/open-location-code decoders, and there should be a documented deterministic way of generating a city from a location, so that a developer can be sure all generated location codes will be decodable with a given offline database.

I'm not sure how feasible an offline database like this would really be, but it would be interesting to discuss this in more detail. Some thoughts and back-of-the-envelope calculations:

We most often want to replace the "area code" (first four characters of a full code) with a city or location name, but not all locations within one four-digit code area will sensibly use the same city name.

This means we would need a database that assigns names to all individual six-digit code prefixes - or perhaps even some form of data that is not based on OLC itself, but probably just as big. Even on the level of six-digit code prefixes, we're still dealing with a considerable overlap between neighboring entities (for example, 8FR8H400+ is half France and half Swiss).

There are around 26 million (91820⁴) six-digit code prefixes. If we leave out "empty" ocean areas (for example, not much going on in 35000000+) or combine those areas where one name is sufficient for a whole four-digit prefix (for example, it looks as if the whole area of 72W40000+ could be named "Midway Atoll"), we might be able to reduce this number by 50% (assumption based on the relative surface area of the earth that is ocean), but probably not by 90% or more.

That means we're dealing with a set of around 3-10 million entries, each of which consists of a plus code prefix and a (globally unique) name, perhaps 20-50 bytes.

App developers might be able to ship just part of this database for any app that is "local" - but using the whole database of ~60-500MB in a mobile&offline context is likely not feasible.

@tkafka did this answer your question?

@zongweil Thanks for the link to map - it is unfortunately a bit difficult to describe to users how exactly should they get a full code without them accidentally getting a shortcode. It would help if a map could be toggled to a 'full code' mode, eg. with url parameter.

@bocops Thank you for the estimation - when I wrote this, my assumption was that you don't need to pick the closest city to the location, that just any location inside the four letter area code will do. And I imagine it could be the one with eg. the highest population at the time, to make it deterministic. So we'd have a dictionary of area code -> name of largest place in the area, and that would be at most 36^4 = cca 1.6M places.
Which is imaginable, with leaving out empty areas and some clever compression, but I admit I initially guessed this to comfortably fit 1 MB.
Wait a second - I just mapped a sample 4 letter code: https://plus.codes/8FXQ0000+ and it has area of cca 7700 km2. Earth surface is 510 million km2, and dividing that (even without considering that there is no need to be precise for arctic and antarctic regions) leads to about 66 000 areas - give it about 30 B per entry (might be quite less with name compression and location encoding), and it could fit 2 MB. Which seems completely OK to me in exchange for universally offline-decodable codes.
Yes, we might need an encoding that doesn't waste half of rectangles for north and south pole, but still - this seems doable. What do you think?

@tkafka Your estimate for the number of 4-digit codes isn't too far off. There are 9 and 18 characters used for the first two positions, respectively, and the full range of 20 characters for all subsequent positions. That leads to 9 x 18 x 20 x 20 = 64,800 code areas.
However, one big problem I still see with the approach to just pick the biggest city in each area would be one with a lack of alignment with political structures. For example, look at one of the neighboring areas of the one you gave as an example: https://plus.codes/9F2P0000+
9F2P0000+ has Prague as its biggest city, so we'd use that. However, Prague extends beyond this code area, so a location in the south of Prague would not use Prague in its shortened code, but another city instead. If the biggest city of the code area to the south is České Budějovice, we might end up with a short code referencing this city for a location that is actually in Prague.
Even worse, because České Budějovice is shifted approximately one full 4-digit code area towards the south, trying to recover a full code (or a lat/long coordinate pair), this short code for a location in Prague might then be recovered to a completely different location that is actually in or near the other city (for example 8FXP X FXF+HJ vs. 8FXP W FXF+HJ).
At the same time, the 9F2P0000+ area overlaps with Germany and Poland, so there would even be locations in those two countries with a short code referencing Prague. That might raise one or another eyebrow as well. ;)

@tkafka I understand your concern about educating users who aren't familiar with plus codes. How will users obtain the plus code to put into your smartwatch app? Are you planning to link to a map within the app? Or will they get it from another source (e.g. their phone) and then manually plug it in?
@drinckes do you think it might make sense to expose the full code in Maps for the offline use case?

I did post about this on the mailing list a while ago
https://groups.google.com/forum/?#!searchin/open-location-code/offline|sort:date/open-location-code/zvxkqlocZUA/8dCAho8-BgAJ
Agree it seems strange to talk about one of the drivers for OpenLocationCodes was be fully functional offline, but then its highly encouraged to create codes that don't work without access to a gazetteer*. Worse the specs dont mandate any particular gazetteer, so to support decoding ones created by Google Maps, would need the Google Specific gazetteer database (for example).
I think would be interesting to create such a database, would be willing to help. geonames.org or maybe osmnames.org could be used as a starting point.

at the same time there seems to be very little practical code around, demonstrating using a Gazetteer, along with RecoverNearest to get the full position. (well I can't find any!)

Agree it seems strange to talk about one of the drivers for OpenLocationCodes was be fully functional offline, but then its highly encouraged to create codes that don't work without access to a gazetteer*.

To be fair, short codes do work in offline contexts if the location of the reference point can be assumed to be shared knowledge between all parties involved. This is not necessarily the same requirement as a world-wide gazetteer. For example, one use case of plus codes "in the wild" is enabling postal services to reach people without a proper street address.
If I use an address like CFGH+23, SOMECITY, SOMECOUNTRY, then postal services should be able to first transfer my postcard to the correct city (as long as it is unique in that country), and then it should be able to deliver the postcard to the correct location within that city (as long as the bounding box for the city is not larger than 1°x1°). Both parts of the process work offline - as does the code generation.

Thats a good point! Taking the example further, the delivery driver wanting to input destination into a navigation device, could enter the short code, and the device use the current 'GPS' location in hte 'Recover Nearest' process. no gazetteer involved.
The decoding of the 'city' name etc, has come via the mail system instead, getting the postcard first to the right local depot.

Related to this discussion.. Does anyone know if it's accidental or intentional that Maps has been straight up dropping the first 4 characters and using just the xxxx+xx bit?
My use case is full codes for precise location printed as barcodes in order to quickly navigate to a list of locations. OLC codes are fantastic for this as they are as accurate as Lat,Long while being ~half the barcode length. In the last few months as G has expanded the use of the plus codes, I have noticed that it will map/navigate to the "xxxx+xx, city, st" version even if the full code is entered. When mapping a trip that covers the better part of a state, the 'closest locale' pull from gps isn't guaranteed to be accurate, leading to incorrect locations being mapped. This kind of kills it as an "accurate as Lat,Long" solution.
I can see the various sides you guys are discussing with the best way for a user to find their code/easiest way to display while still being offline-capable, etc., but shouldn't full-code always trump any 'nearest-to' or manual 'city,st' pulls?
Back to the original question, though, of whether this is intentional (and thus belongs as commentary attached to this issue) or unintentional (in which case, I will repost as a new, separate issue)- if anyone knows, please let me know.

@chancegold I believe this was an intentional change, probably coinciding with the change in terminology from "Open Location Code" to "plus code" and subsequent usage in Google Maps. The first four characters are now considered to be an "area code" that can optionally be used instead of a town or city name, instead of the other way around: https://plus.codes/individuals (bottom of the page).
This makes sense for the major use case of plus codes to actually address locations within a city (basically replacing street and number in a regular address) - although, as this thread suggests, there still are other use cases where full codes are a more sensible solution.

So... wouldn't that mean that it is unintentional?

You can use a plus code with a town or city, such as V86P+C8, Paris, or with an area code, such as 8FW4V86P+C8.

If I enter the full code "8FW4V86P+C8", either as an initial destination or an added stop, it should perform the lookup on the full code before/without any area code lookup, no? Preferably, without, since it's not needed and doesn't make much sense to perform an area code look up/conversion just to show the location in the format "V86P+C8, Paris". To me, that seems equivalent to receiving a precise Lat/Long coordinate address, only to look up and display the closest street address, which might be 100km away if the coordinates are in an undeveloped region.
It seems to me that the primary use of OLC/Plus Codes is in areas that either don't have accurate addresses or don't have addresses at all (rural/undeveloped areas). That means that for the areas where OLC/Plus Codes are crucial, there may not be any nearby relevant town/city.
With all of that in mind, while moving towards a more user-friendly "xxxx+xx, city" scheme may seem to work better for some applications, really, is it? It's just somewhat-shortening the existing "street address, city" paradigm while carrying over the issues of such addressing in rural/undeveloped areas if it doesn't offer full-code-first from the ground up. Basically-
if {entered location} = "xxxxxxxx+xxx" or "xxxxxxxx+xx", return offline-algorithm-calculated Lat/Long point/area; if {entered location} = "xxxx+xxx, city" or "xxxx+xx, city", perform online/local-db lookup of area code, return offline-algorithm-calculated Lat/Long of {area code lookup}{entered location sans city}; if {entered location} = "xxxx+xxx" or "xxxx+xx", perform gps lookup, perform online/local-db lookup of area code nearest gps lookup, return offline-algorithm-calculated Lat/Long of {{gps location} area code lookup}{entered location}; else {closest starbucks}
To be honest, though, I can't really figure out the processes going on in order to replicate what I've seen happening. When it happens, I verify the codes entered, and while some have been a case of a typo/barcode capture misfire, most have been related somehow to the "auto-correct" of a full code to a local code with an incorrect city. I'll see if I can find the note for a specific code giving the error once I get home, but attempts to replicate with, for example, the above Paris code seem to all find Paris; so I don't know.
Regardless- full-code and offline-decodable should both be foundational functions. Use-case specific and user-friendly solutions for short/local codes and the methodologies available are important to decide and standardize, but should be exactly that- standards and methods for specific use-cases, not anything foundational.

I think part of the problem might be that Google Maps, although it accepts full plus codes as input, isn't first and foremost a tool for offline navigation.
As an end user, I probably want the result of a search within Google Maps to be as helpful as possible - and when I paste a full plus code into search while my phone is online, a result basically stating that "this cryptic code refers to a location in Paris, France" seems like a generally good idea. :)
Other than that, I don't think that Google Maps first shortens a full code (dropping the first four characters while adding a location string) only to recover the full code and thus lat/long coordinates later. There does seem to be an issue with some individual plus codes being misinterpreted by Maps, which could be the issue you're seeing - but generally speaking, entering a full code should lead to a map pin in the correct location, even if the code itself is shown in a more human-readable format.
If you can provide a code that misbehaves, I'd be interested to see it. Perhaps yours is the same issue that I've been seeing as well.

As Andreas says - if you search for a full code, Google Maps will decode it
and uses that position. If it can, Google Maps will then convert that code
into a plus code address (short code with place name) for display.
We do this because street maps for cities that don't have street names all
look pretty similar, and so it's useful to get confirmation that you're
looking at Conakry, Guinea vs Lagos, Nigeria.
Hope this helps!
Ngā mihi,
Doug Rinckes, Technical Program Manager, Google Switzerland GmbH; 9G8F+6W Zü
rich https://www.google.com/maps/search/9G8F%2B6W Zürich
On Mon, 3 Jun 2019 at 23:51, Andreas<EMAIL_ADDRESS>wrote:

I think part of the problem might be that Google Maps, although it accepts
full plus codes as input, isn't first and foremost a tool for offline
navigation.
As an end user, I probably want the result of a search within Google Maps
to be as helpful as possible - and when I paste a full plus code into
search while my phone is online, a result basically stating that "this
cryptic code refers to a location in Paris, France" seems like a generally
good idea. :)
Other than that, I don't think that Google Maps first shortens a full code
(dropping the first four characters while adding a location string) only to
recover the full code and thus lat/long coordinates later. There does
seem to be an issue with some individual plus codes being misinterpreted by
Maps, which could be the issue you're seeing - but generally speaking,
entering a full code should lead to a map pin in the correct location, even
if the code itself is shown in a more human-readable format.
If you can provide a code that misbehaves, I'd be interested to see it.
Perhaps yours is the same issue that I've been seeing as well.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/open-location-code/issues/343?email_source=notifications&email_token=ACEN46LQLNC2WVR7XKQDOD3PYWG43A5CNFSM4HOE5FV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW2ZNZA#issuecomment-498439908,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACEN46OZB3QJKAIWUN3KCV3PYWG43ANCNFSM4HOE5FVQ
.

Wouldn't it be useful to modify the google maps app as well as the maps web application in a way that they show always the shortend and the full plus code as well as the lat/lon pair? This would give the choice to the user which information he wants to use in his/her use case.
Just a suggestion.

MXQ4+M5  code

87G7MXQ4+M5

Currently rolling out to desktop Google Maps (may even already be active) is a change so that when you search for a plus code, the global code is put into the URL.
If you search for "9G8F+6W Zürich", the URL will start with https://www.google.com/maps/place/8FVC9G8F%2B6W/....

This new feature that @drinckes refers to is appreciated, but it does not really address the core problem, which goes well beyond offline decoding.
The proponents of plus-codes (OLC), with support from Google, claim to be developing an open and free location standard. This is laudable, and I root for a future in which this has materialized.
But at the same time, Google, the dominant mapping application, is seeding the world with short-codes that are advertised as open but for which everyone except Google is at a significant disadvantage when it comes to decoding them, because nobody outside has access to the proprietary gazetteer used to generate them. And the UI makes it very hard for the user to realize this.
As other mapping providers, such as OpenStreetMap, consider whether to celebrate or fight OLC, this issue must be addressed.
It seems to me that Google is facing a trilemma here, in that it must:

Stop seeding the world with short-codes in the way that it is doing
Work to ensure that short codes are decodable in a predictable and equitable manner
Give up on the idea of plus codes as an open location standard (please don't do that :-))

Of these, I would personally prefer 2, since I can understand the advantages of short-codes from a UI standpoint. This could be done (a) by sharing the location database it uses to generate short-codes, (b) by using an openly available location database to generate the short-codes or (c) check all generated short-codes against an openly available location database and only returning the short-code if it correctly decodes to the right long-code.
In short, if short-codes are to be considered a part of OLC, and not an "embrace and extend" tactic, any generated short-code must be predictably decodable to a long-code using open and non-proprietary components in form of a reference implementation. A short-code that does not fulfil that expectation should be considered to represent a bug in the short-code generation program.

Dear Magnus,
Allow me to make a few points.
The decision to include the locality in the plus code address is a conscious one. Plus codes as a project have many goals from voting to emergency response, but the one with the largest unrealized impact is a goal of allowing for effective bootstrapping of postal delivery and logistics in less developed parts of the world, in particular most of sub-Saharan Africa and Southeast Asia. As such, the locality must be a part of the address, because while I can already send a package to WFCM+74 Praia, Cape Verde from Switzerland, I'll never be able to send it to "796RWFCM+74".
Decoding a plus code address with a short plus code is relatively trivial if you have a geocoder. You search for all localities with the given name, and you take into account the distance between the closest matching code and the locality. OpenStreetMap is not a mapping provider but a map repository so I don't think it makes a lot of sense to somehow include plus codes in OSM other than in addresses of buildings that have plus code addresses and so far there are only hundreds of thousands of those. What it makes sense to do is to include plus code search support in OSM-backed geocoders, and some indeed already have it. For example I believe short plus codes work in OsmAnd, they work on mapy.cz or the windymaps app. While these were implemented by the developers themselves, we reached out to a number of OSM based geocoders some time ago offering help and were met with a relatively consistent answer: we'll think about it once people ask for them.
I think you are suggesting that we should publish a definite list of localities that can be used to decode short plus codes. That is not doable for a number of reasons, for instance because our data isn't reliable in poorly mapped areas (in fact, due to HOTOSM hackathons and similar, you'll easily find areas in SSA where OSM data is better), and because we would be releasing postal or political division of a country which is something that is completely out of our authority to do. But the beauty of plus codes is that you don't need to do this. The decoding is "the matching suffix closest to given locality" rather than "the suffix with the same prefix as the locality" and that means that if for a point between villages A and B you choose village A and I choose village B, we will still be able to decode the same point. Ultimately, using plus codes still presents a search problem, just a much easier one. Getting the village is easy, getting the road and house is hard.
That was decoding. The harder part is encoding (or reverse geocoding). Because you need to be able to say "which town/village is most relevant to lat long x,y?". That is challenging because you don't want to skip admin area boundaries, country borders, and need to be mindful of disputed borders. Again, this is implementable on your own in principle (mapy.cz/windymaps are working on it, I believe), but we have been looking at options for publishing a library with associated OSM derived data that would allow one to incorporate this easily in code. A request for this from a platform with large user-base would make us prioritize the work. Anything we do pushes back some other plus codes project.
I hope that made sense, it's a bit too late.

Dear Rasťo,
Thanks for your quick and thoughtful response. It is clear that you have a compelling vision for how OLC can be used. I do think and hope that this vision can be achieved while at the same time allaying the concerns that I voiced and others had voiced before on this thread. You answered a number of questions, and I'd like to follow up on some specific points:

As such, the locality must be a part of the address, because while I can already send a package to "WFCM+74 Praia, Cape Verde" from Switzerland, I'll never be able to send it to "796RWFCM+74".

Agreed, this is a compelling use case - basically you are referring to the fact that although the Cape Verde postal service is set up to handle OLC, the Switzerland postal service is not, but they would manage to forward a package to Cape Verde where it would be processed further.
You also mentioned that OLC already works on mapy.cz and other sites (awesome!). I was not aware of this, I went ahead and tried entering "WFCM+74 Praia, Cape Verde" on mapy.cz and yes, it works like a charm.
However, I also tried another code, "2GH7+95 Skinnastaðir" (AKA: "9CR52GH7+95"). This is the address of the visitor center at Asbyrgi (in Iceland), generated by Google Maps, and correctly decoded with Google Maps, but not with mapy.cz, which means that it is useless without access to GMaps (as an aside, in order to get the long code I had to go to plus.codes, because for this location the long code did not seem to be present even in the url, I'm not sure why that is the case).
The user might feel that this is evidence that an alternative mapping application is inferior, while in reality it comes down to using a proprietary (and perhaps overly inclusive) list of locations that are allowed as anchors for short-codes. Skinnastaðir is a farm, not a town, and I'm pretty sure that no postal service would be able to deliver to that location using this short-code.

I think you are suggesting that we should publish a definite list of localities that can be used to decode short plus codes. That is not doable for a number of reasons ...

But this comes back to my point, that this is an absolute inviolable requirement for an open standard. And perhaps more specifically for solution c ("check all generated short-codes against an openly available location database and only returning the short-code if it correctly decodes to the right long-code"). The best approach if a reference decoder would fail (as it probaly would in the case of Skinnastaðir), would in fact be to fall back to a long-code. And for an open standard, that decision should be based on open data.

... if for a point between villages A and B you choose village A and I choose village B, we will still be able to decode the same point.

Yes, that is true, if both you and I know about the existence and location of villages A and B. The issue is that if you generate a short-code based on village A, but I do not know anything about village A, then I will not be able to decode it.

That was decoding. The harder part is encoding (or reverse geocoding). Because you need to be able to say "which town/village is most relevant to lat long x,y?". That is challenging ...

Agreed, and I don't think a reference implementation for this is a precondition for OLC to be an open standard. In encoding, individual providers can compete to come up with the best option. However, regardless of which reference point is used (village A or B), my suggestion is that before it is delivered to the user, it must be checked against a reference implementation decoder to ensure that it actually is independently and predictably decodable.

... we have been looking at options for publishing a library with associated OSM derived data ... anything we do pushes back some other plus codes project.

That would be awesome. I think a reference decoding library would be the key issue, a reference encoding library would be more of a "nice-to-have". And yes, I am also sympathetic that you must prioritize your work. For me (and I suspect for others who have commented), the key thing is not whether such a library would be promised tomorrow. Rather, I would be hoping for an openness towards having such a reference implementation become tightly integrated into the OLC standard, and treating any short-code that is non-decodable by such an implementation as a bug (or at least a deficiency) in the specific encoder implementation. Such a deficiency could then be resolved either by changing the implementation (and producing a long-code if no suitable short-code can be generated), or by adding the corresponding anchor to the reference database.
In the meantime, I guess the robustness principle applies: "Be conservative in what you send, be liberal in what you accept". And until a reference decoder exists, I also think it would be justifiable to make it easier to get hold of the long-code than asking users to copy it from within the URL.
Thanks again for all your hard work!

@rsramek

WFCM+74 Praia, Cape Verde from Switzerland, I'll never be able to send it to "796RWFCM+74".

But you are omitting the idea that you could send it to "796RWFCM+74 Praia, Cape Verde" from Switzerland and then the extra location information is informative but not required to decode the value.
The shortcodes are undermining this standard due to what @torfason mentions:

stop seeding the world with short-codes in the way that it is doing

The easiest way to get pluscodes all generate and show shortcodes by default, and even copy shortcodes to the clipboard.

"Send me your pluscode", ... "no, that is the short code, my program can't use those, get the long code". ... "how? no, Google maps on your phone only shows the shortcode, go to https://plus.codes and use the find-your-code page" ... "what? no it shows the shortcode too?!?! ... Click the up arrow thingy at the bottom to get a sidebar, then go copy the full code from the left." ... "no, not just the bolded part, the whole thing including the 4 extra characters to the left" ... "no I don't know why it is so hard, sorry, just send me the code."

That doesn't seem to be worth saving 4 characters of typing or copying by just giving the full code from the start.
I suggest that you never seed shortcodes ever, and probably even remove them from the spec as a core feature (make it optional for local focused applications).  Pluscodes are longer but work, short version of pluscodes sometimes work, but likely not outside of Google.
I would rather receive this offline decodable version (not requiring a location database):
766HQ6HC+Q6

Or this longer version of the same (that works for your international mailing scenario):
766HQ6HC+Q6 Santa Rosa de Copan, Honduras

Rather than this ambiguous shortcode that I can't decode without location data of the sender:
Q6HC+Q6

Or this online-only decodable shortcode (with newline) from plus.codes/map:
Q6HC+Q6
Santa Rosa de Copan, Honduras

Or this online-only decodable shortcode from Google maps:
Q6HC+Q6 Santa Rosa de Copan, Honduras

Change Google maps and the https://plus.codes/map/ page to return the full code plus optional informative location, but never use the shortcode by default.

And now it seems like it is utterly impossible to get the full code from Google maps (not possible even to get a full code with some sort of info button). Yuck!

Sort of, it is still possible to get individual plus codes using: https://plus.codes/map Also, remember that site does use Google maps and the 3 horizontal lines icon gives the option of showing the satalite view
Get Outlook for Android

The plus codes specification (to be written) notes that a specification location must be provided in lat/long.
This is a known and documented part of this project.
Recommending to close issue as out of project scope.

A specification location does not have to be provided in general, only for short codes. Full codes are absolutely decodable without a reference location, as has been stated numerous times in this thread.
This issue concerns the fact that Google, the sponsor of plus codes, insisted on first making it hard, then impossible, to get  full codes using Google maps, which is probably the most prolific source of plus codes in the wild. That issue is still very open.
However, I've long ago given up any hope that plus codes are anything other than Google's attempt to gain control of a location standard that looks open but is actually completely proprietary, and usable in practice only within their own apps and APIs. So, if the admins of this repo agree with that strategy, by all means, go ahead and close this issue :-)

I assert that this project doesn't have a (complete) specification even for global codes or for implementations.
Many ideas are here but this is not specification-level work. I am working to fix this by writing an actual, implementable spec and updating all implementation to follow and test for that spec.
You have a valid criticism that short codes are of limited utility until multiple, competing implementations can show cross compatibility.
And I agree with your sentiment that an "open" specification with only one usable implementation is not meeting the spirit of an open implementation. Cough cough Office Open XML, sneeze ISO lost 20 years of reputation with that.
This project is not as bad as Office Open XML. Because anybody can open a paper map (offline) and find an approximate latitude and longitude for Paris. This has O(1) complexity and space requirements assuming you stay near home and only care about locations near you. For arbitrary definitions of "near" this is almost certainly true for most humans.
Therefore we can complain about the lack of competition but I think that criticism for Google is unwarranted.
I think there are two actionable items here:

The new specification (I'm writing it) should clearly express the limitation of short codes in succinct and informative language. I welcome your contributions here.

A new reference location project may be started. Perhaps named "Well Known Places". This could be a dataset of reasonably many (how many?) locations including a center lat-long and names translated into every human language. The "many" should be set "small" if there are limited resources to update the list because names change. Also note that location names are subject to contention. This dataset could be created manually or generously donated by Google, Microsoft, Garmin or others. Such a project is probably not in scope for this repository. But I hope owners here would accept a PR that links to such a project if it were to exist.

I can help with item 1 here.
If there are significant resources and interest available, and a commitment by owners to merge a PR noted above, then I can help to organize that project as manifested as item 2 here.

@fulldecent, I agree with much of what you said.

If there are significant resources and interest available, and a commitment by owners to merge a PR noted above ...

I think this is a key issue. I feel that my posts in 2019 were an attempt to contribute to item 1 (in terms of what the policy should be), and I actually started some experiments implementing item 2 (as an R package, using OSM data). However, the responses to the posts (@apatrida also had some thoughtful insights), coupled by the subsequent decision by Google to remove the ability to get full codes using Google Maps, convinced me that the owners are not at all interested in going this route and that trying to contribute towards this was a dead end.
I hope that you will find that I am mistaken and that you are able to meaningfully contribute to OLC.

Google, as a whole, does not necessarily have consistent interests in its Google Maps product and this OLC project.
(I am an ex-Googler and do not speak for any of the entities involved.)
Because people with commit access here are not completely AFK, I am treating this project with a hopeful optimism and the "GitHub street rules" of "if you show, do all the work, fully understand the project scope, and the committers are fair, then you can lead any project from the bottom."

Regarding item one // https://github.com/google/open-location-code/issues/343#issuecomment-826325385 do you have some proposed wording?

Regarding item two // https://github.com/google/open-location-code/issues/343#issuecomment-826325385 I'll consider this dead until someone with commit access tells me there is interest to link, and resources are available from somewhere.

This issue is fixed at https://github.com/google/open-location-code/pull/463
The statement "Codes should be able to be generated and decoded offline" is removed and appropriate marketing claims are made as to which codes can be decoded offline.
And additionally, the short code specification is tightened, and there are notes to explain under which circumstances a short code could be exchanged between two parties:

A short code Plus Code is meaningful only when the producer and consumer of the Plus Code can agree on the approximate latitude and longitude of a reference location. A short code can be easier to use and remember than a full code.

Dragging this back on topic, yes, it is possible for people to create short codes with localities. No, it is not in scope of this project to generate a prescriptive collection of place names for the world. Why?
The problem is not that people do not know their town or neighbourhood or village name. The problem is they don't have a street name or number. That is the bit that plus codes are solving, both the global codes and the short codes + locality.
So we don't want to generate a set of localities that end up generating short codes with locality names that are not what are in actual use.
Anyway, closing this as we're not going to try to produce a global locality database.


[TW#12669] Examples System/OTA - randomly fails

Followed the instructions, all compiles and runs fine.
Seems to randomly stop in the middle of writing the image.....
Attempt 1
( esp running )
I (3774) ota: Have written image length 10621
I (3784) ota: Have written image length 11645
I (3804) ota: Have written image length 12669
I (3814) ota: Have written image length 13693
I (3814) ota: Have written image length 14413
I (11374) wifi: pm start, type:0
(esp hangs)
Reset and attempt 2
I (7514) ota: Have written image length 183509
I (7534) ota: Have written image length 184533
I (7564) ota: Have written image length 185557
I (7584) ota: Have written image length 186581
I (7604) ota: Have written image length 187605
I (7614) ota: Have written image length 188629
I (7614) ota: Have written image length 188761
I (11154) wifi: pm start, type:0
Reset and attempt 3
I (3804) ota: Have written image length 33697
I (3804) ota: Have written image length 33981
I (3834) ota: Have written image length 35005
I (3834) ota: Have written image length 35441
I (11344) wifi: pm start, type:0
Reset and attempt 4
I (7514) ota: Have written image length 183509
I (7534) ota: Have written image length 184533
I (7564) ota: Have written image length 185557
I (7584) ota: Have written image length 186581
I (7604) ota: Have written image length 187605
I (7614) ota: Have written image length 188629
I (7614) ota: Have written image length 188761
I (11154) wifi: pm start, type:0
This happens on the local server supplied, a local apache server and a remote apache server.....
Any ideas?

Seems its perhaps a timings issue, if I put a vTaskDelay in the while(flag) loop then it has a much better chance of working, still only works now and then....
Or as soon as you put a lot of ESP_LOGI statements in the code, it works.....

Is there any possibility you could take a tcpdump from the server side, as the OTA download is happening?

Is there any possibility you could take a tcpdump from the server side, as the OTA download is happening?

Never mind, I'm able to reproduce this here. Will let you know we find.

Sorry, I was asleep, in the UK here :)
Thanks for the help.

a vTaskDelay(20) fixed it for me, ESP_LOG did nothing for me

doesnt work for me... and it takes forever !
vTaskDelay(3) and some debug statements seems to be best, altho there will be a proper solution I am sure :)

vTaskDelay(20) worked for me, too. Thanks for the advice.
BTW, I'm experiencing a similar issues with OTA on Arduino (some time works, but most of times there are errors during the sketch download). Has anybody the same problem?

i also tried the vTaskDelay and gave me some degree of success but still failed on occasion. What worked for me was to reduce the BUFFER sizes from 1024 to 512... that gave me the best results.

For me it hangs (forever), always in the recv() call but never in the same place in the file, that is, sometimes it reads 20 lines sometime 100, but it NEVER ends and ALWAYS hangs on the recv().
EXACTLY the same application with the OLD library works perfectly EVERY TIME.
Tried the delay but to no avail. Also the buffer size 4096, 2048, 1024 and 512 they all hang in the new version and ALL work in the previous version.
If you compile with this version  the OTA example it works. My routines are a copy of this source code with minor mods.  Very disturbing.

Any news on this ? :)

Hi everyone,
Sorry for the slow update. This bug should be fixed in IDF master branch as of commit 365b66b9.
Angus

Thanks Ill give it a try.

Working as far as I can test it. Thanks

3 Jun, I clone the master, this issue still have...

On Jun 4, 2017, at 7:43 PM, Angus Gratton<EMAIL_ADDRESS>wrote:
Closed #610 https://github.com/espressif/esp-idf/issues/610.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/espressif/esp-idf/issues/610#event-1109298363, or mute the thread https://github.com/notifications/unsubscribe-auth/AIITq-G9v3AAa5T2yZfa2JuVJ40sljyIks5sA08WgaJpZM4Nc2XU.

In my case its working, 700-800k about 10 times.
Regards.

3 Jun, I clone the master, this issue still have...

What is the output of the following two commands:
git describe
git submodule status

@projectgus ,Hi,
Just now, I clone the master, it's work fine!^_^
Thank you.

Tested a few times, works great, thanks :)


ci: remove snyk

:tada: This PR is included in version 4.6.1 :tada:
The release is available on:

v4.6.1
GitHub release

Your semantic-release bot :package::rocket:


Replace Mockito & Kluent with Mockk & Kotest

Why
Currently for CoroutineTemplate we are using mockk for mocking and kotest for assertions.
To align the templates, we should apply the same for RxJavaTemplate as we're still using mockito & kluent there.
Explained in code:
testImplementation("org.amshove.kluent:kluent-android:${Versions.TEST_KLUENT_VERSION}")
testImplementation("org.mockito.kotlin:mockito-kotlin:${Versions.TEST_MOCKITO_VERSION}")

should be replaced with:
testImplementation("io.kotest:kotest-assertions-core:${Versions.TEST_KOTEST_VERSION}")
testImplementation("io.mockk:mockk:${Versions.TEST_MOCKK_VERSION}")

Who Benefits?

Our templates will be more aligned
Our developer experience will improve

We'll no longer make improvements to RxJavaTemplate
Thread: https://nimble-co.slack.com/archives/G41UCTGHL/p1647919906598519


Will the "advanced" Colab file be updated to 0.6?

I have lots of old files not using v2_0_0.wav or the official input filkes and I would like to specify my own x_train and y_train. I don't know how to retrain any of the models without it.

Ah, I had said at the top that I'd remove it, huh?
Just had a look through it and the good news is that if you want to keep on using it, I'm pretty sure you can--just remove the <=0.5.3 bit and everything else seems like it'll still work.
And you can always keep a personal copy of that Colab notebook (or grab it from a previous version here when you need it, or keep a copy in your own repo)--all I'm saying is that I'm not going to be prioritizing keeping it working over here.


Add channel and locale filter to product grid

In products data grid (products table in admin area) only products were displayed that are in the current channel and have the same locale as the app locale value. So, as a channel manager I do not see any products of other channels or other locales.
This PR adds two dropdowns to select channel and/or locale to show products for. If none is selected products for all channels and locales are listed.

GitHub hates me. ;-)
This PR also shows changes already merged into master branch.
Files that I changed within this PR:
packages/Webkul/Admin/src/DataGrids/ProductDataGrid.php
packages/Webkul/Admin/src/Resources/lang/en/app.php
packages/Webkul/Admin/src/Resources/views/catalog/products/index.blade.php


mariadb: 10.2.15 -> 10.2.16

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build mariadb

Success on x86_64-linux (full log)
Attempted: mariadb
Partial log (click to expand)
/nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16/bin/galera_new_cluster: interpreter directive changed from "/bin/sh" to "/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23/bin/sh"
/nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16/bin/mariadb-service-convert: interpreter directive changed from "/bin/bash" to "/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23/bin/bash"
/nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16/bin/wsrep_sst_mysqldump: interpreter directive changed from "/bin/bash -ue" to "/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23/bin/bash -ue"
/nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16/bin/wsrep_sst_xtrabackup-v2: interpreter directive changed from "/bin/bash -ue" to "/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23/bin/bash -ue"
/nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16/bin/mysql_secure_installation: interpreter directive changed from "/bin/sh" to "/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23/bin/sh"
/nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16/bin/msql2mysql: interpreter directive changed from "/bin/sh" to "/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23/bin/sh"
/nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16/bin/wsrep_sst_rsync: interpreter directive changed from "/bin/sh -ue" to "/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23/bin/sh -ue"
/nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16/bin/wsrep_sst_mariabackup: interpreter directive changed from "/bin/bash -ue" to "/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23/bin/bash -ue"
checking for references to /build in /nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16...
/nix/store/n8hlzh37bjzks89663m38c9qsi2yav9r-mariadb-10.2.16

Failure on x86_64-darwin (full log)
Attempted: mariadb
Partial log (click to expand)
[ 98%] Building CXX object libmysqld/CMakeFiles/sql_embedded.dir/__/sql/sql_type.cc.o
[ 99%] Building CXX object libmysqld/CMakeFiles/sql_embedded.dir/__/sql/table_cache.cc.o
[ 99%] Building CXX object libmysqld/CMakeFiles/sql_embedded.dir/__/sql/mf_iocache_encr.cc.o
[ 99%] Building CXX object libmysqld/CMakeFiles/sql_embedded.dir/__/sql/item_inetfunc.cc.o
[ 99%] Building CXX object libmysqld/CMakeFiles/sql_embedded.dir/__/sql/wsrep_dummy.cc.o
[ 99%] Building CXX object libmysqld/CMakeFiles/sql_embedded.dir/__/sql/encryption.cc.o
[ 99%] Building CXX object libmysqld/CMakeFiles/sql_embedded.dir/__/sql/item_windowfunc.cc.o
[ 99%] Building CXX object libmysqld/CMakeFiles/sql_embedded.dir/__/sql/sql_window.cc.o
building of '/nix/store/m973h1qwnaqxfvi9r5zppcnfckz45zdb-mariadb-10.2.16.drv' timed out after 1800 seconds
error: build of '/nix/store/m973h1qwnaqxfvi9r5zppcnfckz45zdb-mariadb-10.2.16.drv' failed

Success on aarch64-linux (full log)
Attempted: mariadb
Partial log (click to expand)
/nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16/bin/mysql_secure_installation: interpreter directive changed from "/bin/sh" to "/nix/store/p0vy17dp9jk2mvqsxsqnb14s3797lay7-bash-4.4-p23/bin/sh"
/nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16/bin/mysql_config: interpreter directive changed from "/bin/sh" to "/nix/store/p0vy17dp9jk2mvqsxsqnb14s3797lay7-bash-4.4-p23/bin/sh"
/nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16/bin/msql2mysql: interpreter directive changed from "/bin/sh" to "/nix/store/p0vy17dp9jk2mvqsxsqnb14s3797lay7-bash-4.4-p23/bin/sh"
/nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16/bin/mysql_install_db: interpreter directive changed from "/bin/sh" to "/nix/store/p0vy17dp9jk2mvqsxsqnb14s3797lay7-bash-4.4-p23/bin/sh"
/nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16/share/mysql/mysql.server: interpreter directive changed from "/bin/sh" to "/nix/store/p0vy17dp9jk2mvqsxsqnb14s3797lay7-bash-4.4-p23/bin/sh"
/nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16/share/mysql/wsrep_notify: interpreter directive changed from "/bin/sh -eu" to "/nix/store/p0vy17dp9jk2mvqsxsqnb14s3797lay7-bash-4.4-p23/bin/sh -eu"
/nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16/share/mysql/binary-configure: interpreter directive changed from "/bin/sh" to "/nix/store/p0vy17dp9jk2mvqsxsqnb14s3797lay7-bash-4.4-p23/bin/sh"
/nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16/share/mysql/mysqld_multi.server: interpreter directive changed from "/bin/sh" to "/nix/store/p0vy17dp9jk2mvqsxsqnb14s3797lay7-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16...
/nix/store/81qbcfdg9bky7m7i3j1wk8sfwd0vy00y-mariadb-10.2.16


⚠ Bofrost.es has degraded performance

In bc59c85, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1479 ms

Resolved: Bofrost.es performance has improved in 6790760 after 6 minutes.


Postgres Migration Questions

Your Question
I've been a long time user of Gorm with sqlite and currently moving to the Postgres driver. I'm noticing a lot of the simplicity of migrations does not work in Postgres and I just want to make sure I'm not doing something stupid.
The AutoMigrate function did not work for me and I needed to move to Migrator().CreateTable() to get the tables created. After using CreateTable, and retrying AutoMigrate, no many-to-many tables are created.
What I thought should work fine:
	if !postgresDB {
		db, err = gorm.Open(sqlite.Open("DBNAME.db"), &config)
	} else {
		dsn := fmt.Sprintf("user=%s password=%s host=%s dbname=DBNAME port=5432 sslmode=enable",
			user, pass, host)
		db, err = gorm.Open(postgres.Open(dsn), &config)
	}
         db.AutoMigrate(
		&Thing1{},
		&Thing2{},
		&Thing3{},
         }

The document you expected this should be explained
https://gorm.io/docs/migration.html
Expected answer
If this is expected behavior and the Postgres driver is less featured, is there any guidance on creating the many-to-many tables? Is the MySQL driver more friendly?
My many-to-many implementation looks as described in the docs:
type Thing1 struct {
     Thing2    []*Thing2 `gorm:"many2many:thing1_thing2;"`
}

Thanks for any guidance and assistance.

I think I found the best way to address my problems. For anyone else who has a similar experience -- on initial table creations for Postgres setting config.DisableForeignKeyConstraintWhenMigrating = true then sending it through db.AutoMigrate() gets the tables created with no errors. Then going over db.AutoMigrate() a second time without that config setting gets the foreign keys and many-to-many tables created.
Just an extra step but I suppose it makes sense, I think there were some errors when trying to set foreign keys for tables that weren't created yet. And maybe SQLite is more willing to allow that than Postgres was.

I think I found the best way to address my problems. For anyone else who has a similar experience -- on initial table creations for Postgres setting config.DisableForeignKeyConstraintWhenMigrating = true then sending it through db.AutoMigrate() gets the tables created with no errors. Then going over db.AutoMigrate() a second time without that config setting gets the foreign keys and many-to-many tables created.
Just an extra step but I suppose it makes sense, I think there were some errors when trying to set foreign keys for tables that weren't created yet. And maybe SQLite is more willing to allow that than Postgres was.


feat(FMP-361): connect to MetaMask on click

Current dependencies on/for this PR:

main

PR #52

PR #53

PR #54   👈

PR #55

This comment was auto-generated by Graphite.

⏳ This pull request is set to merge as part of a Graphite merge job
Stack job ID: VExqmqIMSxeFRAdKggnq.
See details on graphite.dev


Markup different on the server and the client

I'm new with MERN, so not sure if I'm doing something wrong.... but, I get this:
warning.js?8a56:44Warning: React attempted to reuse markup in a container but the checksum was invalid. This generally means that you are using server rendering and the markup generated on the server was not what the client was expecting. React injected new markup to compensate which works but you have lost many of the benefits of server rendering. Instead, figure out why the markup being generated is different on the client or server:
 (client) " data-reactid="1">
 just after
, on the server side.
This seems to happen if I create the project with mern executable and use that and also if I clone the 2.0 branch and use that...

Last time I got this (I wasn't using mern.io) the cause was my server rendering my component without it having the initial data that was retrieved async. But in the client it had time to get the data and rendered differently from the server.

Maybe related https://github.com/Hashnode/mern-starter/issues/139

There is a problem in v2 branch after implementing react-router code splitting. Initially, the client is generating an empty response which is not matching with the server response. That's why this warning comes. But, everything works as expected.

@somus  will be doing pr shortly to fix this, as well as error not rendering header png

PR #181 ready for your review @somus
sorry about to commits to index.js accidently forgot closing curly brace as I  was in rush to grab kids from school :(

@somus closing last PR, only doing one PR to fix this bug, next will be to fix png issue in header, and another PR to fix inital render of client before server

I'm using mern.io with react bootstrap. I'm getting this warning, and source codes are totally different from client and server rendering. is there are way to fix this with react bootstrap?

Small workaround could be something like that:
instead of

${html}

paste this

${process.env.NODE_ENV === 'production' ? html : `<div>${html}</div>`}

at server/server.js#L80
:ok_hand: so you'll get rid of this warning on dev mode (if you are using Redux DevTools) and keep production markup unchanged.

${process.env.NODE_ENV === 'production' ? html : `<div>${html}</div>`}

should be part of the official build..

Use templating on the server side to inject the data into the html view that would be collected on the client side before serving the html to the client.

Fixed by https://github.com/Hashnode/mern-starter/pull/369 and in upgrade/2.4.0 branch.


Add ShazamScrobbler

Project URL
https://github.com/stephanebruckert/ShazamScrobbler
Category
Audio
Description
Scrobble vinyl, radios, movies to Last.fm – ShazamScrobbler was created for people who use Shazam to identify songs played around their Mac and would like to keep an updated playback history using Last.fm's scrobbling service.
Why it should be included to Awesome macOS open source applications  (optional)
For Last.fm and Shazam lovers – only available on Mac
Checklist

[x] Only one project/change is in this pull request
[x] Addition in chronological order (bottom of category)
[x] Appropriate language icon(s) added if applicable
[x] Has a commit from less than 2 years ago
[x] Has a clear README in English

@stephanebruckert Thanks for your contribution!


bug: Back button missing sometimes - works with "ionic": "v1.0.0-rc.3"

Type: bug
Platform: all
See -> http://forum.ionicframework.com/t/back-button-missing-sometimes/28235/9
@yewnork - Its a BUG :frowning:
@cormacodonnell
@DonEllingsworth
@harmonics
See this fork of your Plunker: http://plnkr.co/edit/JVvifeOX5VSDGoa8lrTJ?p=preview
I updated the Plunker to use 1-0-1 (http://forum.ionicframework.com/t/1-0-1-vanadium-vaquita-released/27420) which didn't work so I checked my bower.json and noticed that I am still using:
"ionic": "v1.0.0-rc.3"

which works, index.html:
<link rel="stylesheet" href="http://code.ionicframework.com/1.0.0-rc.3/css/ionic.css" />
<link rel="stylesheet" href="style.css" />
<script src="http://code.ionicframework.com/1.0.0-rc.3/js/ionic.bundle.js"></script>

Note: I also updated from href to ui-sref in several files + fixed 'menu.html' as well :)

Greetings @Robinyo!
My sensors indicate that you need to update your issue through our custom issue form. We are now requiring all issues to be submitted this way, to ensure that we have all of the information necessary to fix them as quickly as possible.
Click Here To Update Your Issue
I will have no choice but to close this issue if it is not resubmitted through the form. Please fill out the rest of the form, so that I may use my friendly robot powers to assist you.
Thank you!

Just tried to:
"ionic": "v1.0.0"

Which works, but:
"ionic": "v1.0.1"

doesn't :(

I have same issue with version 1.0.1
Here is code pen demo. http://codepen.io/anon/pen/wamRVX4
Sequence to reproduce

Select Check in from side menu.
Click on Attendees on main page new page will display back button
Click back which will return to Check in page
Now select check in from side menu (Even if same page is open)
Click on Attendees link new page will not display Back button

Plunker Ionic 1.0.1: http://plnkr.co/edit/wzh9enXseYHfhakdxXhm?p=preview

Plunker "Playlists - Ionic v1.0.0":
http://plnkr.co/edit/pGwucVrTBDxSvw6z092h?p=preview

@Robinyo so whats the difference between these two plunkrs?
This one, http://plnkr.co/edit/pGwucVrTBDxSvw6z092h?p=preview , works as expected.

That's correct, Plunker "Playlists - Ionic v1.0.0" - http://plnkr.co/edit/pGwucVrTBDxSvw6z092h?p=preview displays a '<' back button if you:

Tap 'Reggae'
Tap '<'
Tap 'nav-icon'
Tap 'Playlists' shows Playlist with a  '<' back button

If you follow the same steps with the Ionic v1.0.1 Plunker Ionic 1.0.1 - http://plnkr.co/edit/wzh9enXseYHfhakdxXhm?p=preview it displays nav-icon NOT a '<' back button.
So, the 'bug' was introduced in Ionic version 1.0.1.

There's a small list of changes in 1.0.1, and only two that may be the culprit.
https://github.com/driftyco/ionic/blob/master/CHANGELOG.md#101-vanadium-vaquita-2015-06-30

Okay, so cef6708 is the culprit, looking into a fix

@perrygovier I am still seeing this behavior in ionic v1.1.0 where eventually I will see this in the console:

TypeError: Cannot read property 'expire' of null
    at ionic.bundle.js:47159
    at Scope.$broadcast (ionic.bundle.js:24992)
    at y.transition.M.then.y.transition.y.transition (angular-ui-router.min.js:7)
    at processQueue (ionic.bundle.js:23394)
    at ionic.bundle.js:23410
    at Scope.$eval (ionic.bundle.js:24673)
    at Scope.$digest (ionic.bundle.js:24484)
    at Scope.$apply (ionic.bundle.js:24778)
    at ionic.bundle.js:26545
    at completeOutstandingRequest (ionic.bundle.js:14221)

When using the menu-toggle attribute it is working fine.

This issue remains unresolved.  Using Ionic 1.7.12.  The error occurs in nextViewOption() in the deregisterStateChangeListener.  When a $stateChangeSuccess event occurs it's handler is called.  nextViewOptions is null when the event occurs which triggers the error at nextViewOptions.expire when calling $timeout().
nextViewOptions: function(opts) {
deregisterStateChangeListener && deregisterStateChangeListener();
if (arguments.length) {
$timeout.cancel(nextViewExpireTimer);
if (opts === null) {
nextViewOptions = opts;
} else {
nextViewOptions = nextViewOptions || {};
extend(nextViewOptions, opts);
if (nextViewOptions.expire) {
deregisterStateChangeListener = $rootScope.$on('$stateChangeSuccess', function() {
nextViewExpireTimer = $timeout(function() {
nextViewOptions = null;
}, nextViewOptions.expire);
});
}
}
}
return nextViewOptions;
},

This occurs to me to, on Ionic 1.7.14.

Me too, 1.7.14.
0     226745   error    Error: null is not an object (evaluating 'nextViewOptions.expire') http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:51661:37 $broadcast@http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:29477:33 http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:49321:32 processQueue@http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:27879:30 http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:27895:39 $eval@http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:29158:28 $digest@http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:28969:36 $apply@http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:29263:31 http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:31030:42 completeOutstandingRequest@http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:18706:15 http://<IP_ADDRESS>:8101/lib/ionic/js/ionic.bundle.js:18978:33


Empty region detection fails for placeholders

https://github.com/openfed/kiso/blob/71ff72cad96b8a3da37ed2cd9082220a46eb32d8/includes/preprocess.page.inc#L18
We need a better check to see if the block is empty. For example if the content is a "<drupal-render-placeholder>" or an "<img>" the current check will fail.
It would be nice if strip_tags could allow some tags.
This is a problem, for example, with Indexed Views exposed forms assigned to a region.

Thanks @rutiolma for reporting this issue.
I'm afraid to not be able to make the difference between "" and ""... Any idea?

Comment updated, HTML was stripped :)

This fix has some regressions.
Also, if a region is empty, there's a fatal error.


[SPARK-50480] Extend CharType and VarcharType from StringType

What changes were proposed in this pull request?
Change type hierarchy so that CharType and VarcharType extend StringType.

Why are the changes needed?
This change is a step towards to properly supporting char and varchar types.

Does this PR introduce any user-facing change?
No.

How was this patch tested?
Added tests in DataTypeSuite.

Was this patch authored or co-authored using generative AI tooling?
No.

@mihailom-db

@jovanm-db update title with the component [SQL]

Please merge in new master, so we do not have conflicts

thanks, merging to master!


Chores: Fix SAGEMAKER_APP_TYPE_LOWERCASE, reorganize kernel_launcher

Chores: Fix SAGEMAKER_APP_TYPE_LOWERCASE, reorganize kernel_launcher scripts, Modify kernel launcher for python3
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

LGTM


459 update toggle and numeric field styles

What is the context of this PR?

https://trello.com/c/FcPqON0I/459-update-toggle-switch-and-numeric-field-to-match-new-designs
Updated toggle component to match new designs
Removed increment/decrement buttons from number components

How to review

Run tests
See visual changes in Storybook

 Update Toggle Switch and Numeric field to match new designs

Updated to match attached designs.

@adammahmood @WickyNilliams @SamGodwin2 @samiwel
Should we remove state from our stories as we go? Seems unnecessary to maintain that code every time we change a component. Recently, we've preferred just documenting each state separately and using knobs where appropriate.

@hamishtaplin i think it's a good idea to do that, yeah


API

Exact scope on the API is still coming into view; this issue should be for discussing and designing the 1.0 API.

First cut is done! Leaving open / re-scoping issue to discuss API design.

Replaced by https://github.com/covidatlas/li/issues/224.


Differentiate vehicle ferry routes

Whether or not you can drive onto a ferry is a critically important detail for route-planning and navigation. But OSM maps tend to render all ferry routes the same way. This is a good opportunity to innovate.
In the below example, one route is a massive oceangoing vessel that fits RVs, the other is a motorboat that might fit a couple dozen people, tops. It's impossible to tell the difference in the rendering.

Another consideration is that of tolls. There are a number of toll-free car ferries in the US and Canada, meaning tolls shouldn't be implied. Since toll roads are rendered distinctly in americana, tolled ferries might should be as well.

It looks like the transportation_name layer does not have toll tagging, so we would have to get that added to OpenMapTiles. (OSM object, tile feature).  This should be a pretty straightforward add, though it would help our case to have some examples of paper maps that differentiate this.

It seems like there isn’t very much precedent for distinguishing ferry lines by their characteristics in print maps, at least in the maps in my collection. I don’t have any examples of maps that give auto ferries a different color or dash pattern than passenger ferries, for instance.
In fact, in their maps of New York State and California, Rand McNally treats any ferry as exotic enough to warrant a generic “Ferry” label:

In this map of Long Island, the American Map Company labels every passenger ferry as “Passenger Ferry” instead of by its name. They also provide an inset with phone numbers of automobile ferries to call for fares and schedules.

The Kentucky Transportation Cabinet’s official state highway map does indicate that the ferries they operate are tolled, while the Louisiana Department of Transportation and Development labels free ferries on their map:

Engels Maps, a local publisher in the Cincinnati area, goes a step further, labeling the ferry with the exact fare for cars. This ferry had a famously consistent fare for many years, though they’ve since raised the rate.

The Washington State DOT’s official map shows not only state route shields on its ferry lines but also “Passenger Only” and duration labels:

@1ec5 Thanks for the rundown! Its clear paper maps tend to treat ferries are unusual enough and complicated enough to just describe them with labels. But the labels do specify toll vs. free and vehicle vs. passenger, so I think there's precedent to distinguish them somehow in Americana. Text-only is one option, but I think we can do better with the lines, even if slightly.
Here's another example from NPS (no stroke differences):

Yes, I think there are a few possibilities here:

At a zoom level low enough that a ferry line is barely visible, apply a point-placed label instead of a line-placed label.
Clearly label “passenger” versus “vehicle”. Are these two modes sufficient, or do we need to distinguish permission to carry a bicycle on board? Is the ferry’s name important enough to show alongside this information?
Clearly label free versus tolled, in keeping with our existing treatment of toll roads.
The NPS map indicates seasonality. I think we should consider a “seasonal” gloss for roads as well as ferries: #503.

This possibly calls for several new fields to be exposed through OpenMapTiles. I think it would be the style’s responsibility to come up with the descriptive text, because it needs to be localized and could depend on the available space.

Since the toll attribute already exists in the transportation layer, this is probably the easiest add. However, I think we would need to add ferry tolling information to the transportation_name layer if we wish to use text labels rather than a line treatment. So I think that styling choice needs to be determined before I can submit a PR to OpenMapTiles. I also proposed in the linked ticket that we only tag the toll=no case, which should be the rarer situation.

At a zoom level low enough that a ferry line is barely visible, apply a point-placed label instead of a line-placed label.

I like this a lot, just have to see if it's useful in dense places with lots of ferries.

Clearly label “passenger” versus “vehicle”. Are these two modes sufficient, or do we need to distinguish permission to carry a bicycle on board?

I count 381 ferry routes in OSM with foot=yes and bicycle=no. There must be more without proper access tags. This is an edge case, but it would be useful to display if feasible, at least when bikes are disallowed. No one wants to bike to a ferry just to be denied boarding.

Is the ferry’s name important enough to show alongside this information?

OSM typically just puts "Origin-Destination" (with various formatting) in the name tag. I think this info is quite useful when the route is too long to show entirely on screen at once (based on the zoom). But to improve formatting and localization, it might be better to use the from and to tags instead of name. Though there are cases where name has something useful, like the Staten Island Ferry.

I think we should consider a “seasonal” gloss for roads as well as ferries

Agreed.

We could even decide to represent these attributes using icons beside labels, similar to public transportation maps.

I'd be interested to see what this might look like. We could also adjust the icons for the ferry terminals to indicate access if all the ferries terminating there have the same access. Like this for passenger ferry terminals:

We could even decide to represent these attributes using icons beside labels, similar to public transportation maps.

I'd be interested to see what this might look like. We could also adjust the icons for the ferry terminals to indicate access if all the ferries terminating there have the same access.

Similar to how NPS maps append amenity icons to place labels, we could append mode-of-transportation indicators to the ferry labels using a format expression:

We could keep the icons small and match them to the label color to avoid drawing too much attention. That said, I think glosses would be straightforward enough and well-justified by existing maps. We have a legend now, but we probably shouldn’t rely too heavily on it, since users aren’t guaranteed to consult it.

AAA’s Hawaii map is a rare map that distinguishes auto and passenger ferry lines by color:


Aluminum plate

Can I use the dxf included in the folder with the cases to produce a 1.5 mm thick double-sided plate to snap switches into? I would then 3D print the base part myself.  Or am I better off just 3D printing the whole thing? I have made a macro pad or two and I have reservations on it's longevity.  I did get a number of pcbs made however and so long as I get a matching number of plates the price isn't so bad from xometry.

Hi there,
yes you can but you need to take into account the tolerances required for the switches to snap on correctly. In other words: the dxf file is a good starting point to generate your case files but is not "production ready".


Add LI posts to calendar according xOFU or AARRR

If I click on the schedule action on a single item (in the comparison context view), this will open the calendar and I can choose a date
If I click on the automatci schedule action on a aggregated item (in the aggregated view), this will open a form with the following properties:

the method I want to use among the one which are available (xOFU/AARRR)
the LI account I want to use (personal one vs company one)

The list of files identified for the action will be displayed below.

Post LI.T.1
Post LI.T.2
Post LI.T.3
Post LI.M.1
Post LI.B.1

By clicking on the Schedule action, the following events will be created:

for xOFU:

Monday: TOFU: Post LI.T.1
Tuesday: MOFU: Post LI.M.1
Wenesday: TOFU: Post LI.T.2
Thursday: BOFU: Post LI.B.1
Friday: TOFU: LI.T.3

for AARRR:

Monday: Acquisition: Post LI.T.1
Tuesday: Activation:  Post LI.M.1
Wenesday: Retention: Post LI.T.2
Thursday: Revenue: Post LI.B.1
Friday: Recommandation: Post LI.T.3

TOFU and AARRR are not opposite, they can combine with each other, in respect with best LI publish time, to create more value:

Acquisition in a TOFU and optimistic and energic style on monday, to start week on the right foot
Activation in a MOFU and engaging style on tuesday, to give an opportunity to go deeper
Retention in a TOFU and educational style on wenesday, to give more arguments to buy tomorrow
Revenue in a BOFU and determined style on tuesday, to suscribe and take your destiny in your hands
Recommandation on a TOFU and casual style on friday, to prepare insights (and affiliated links for friends) for the weekend

To identify content types (post LI? TOFU? MOFU? BOFU), we currently use the content name.
In the future, it would be a better idea if content may have custom metadata attached to them during the generation process. These custom metadata can then be used by custom actions such as Publish.


Cannot record region using captura-cli

captura-cli.exe start --source 100,100,300,400
Captura v8.0.0
(c) 2018 Mathew Sachin
Video Buffer Allocated: 0
Press p to pause or resume, q to quit
System.NullReferenceException: Object reference not set to an instance of an object.
at Captura.ViewModels.RecordingViewModel.<>c__DisplayClass67_0.b__0()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Captura.ViewModels.RecordingViewModel.d__67.MoveNext()
I had a look at the source and it seems to be caused by RegionSourceProvider being initialized by MainViewModel before FakeRegionProvider.SelectedRegion is set thereby recording from 0,0,0,0 instead of the one on the command line

Fixed! Thank you for reporting 👍
Here's a dev build to test: https://ci.appveyor.com/api/buildjobs/w7e0a3whmyqsccio/artifacts/Captura-Portable.zip


tools/run_tests/helper_scripts/run_grpc-node.sh is flaky on both macos and linux

Currently failing 6% of our macos dbg builds and flaking on linux too.
Recent mac failures (all dbg):
https://sponge.corp.google.com/invocation?id=24ec39a3-3f1c-46fa-bfc3-8b44e80dee81
https://sponge.corp.google.com/invocation?id=12cb3013-c824-43bc-aa18-c2d83e348568
https://sponge.corp.google.com/invocation?id=2e8305e9-47a0-4fc1-9238-82371b99b901
Recent linux failures (all dbg):
https://sponge.corp.google.com/invocation?id=9935ab4f-0a02-4923-a4c3-e7190afd0a19
https://sponge.corp.google.com/invocation?id=6568a50f-7893-44d8-a5c9-48cc5478a6da

@murgatroid99  is this on your radar?

This looks like a network failure when downloading external resources that are required for the tests. There is only so much we can do about failures like that. And it's difficult to programmatically distinguish between that and a compilation failure, so I don't know if we want to just blindly retry.

I understand this issue is pain to deal with, but currently the flakiness is a bit too high and we should figure out something.
Is there a way we can isolate the "prepare" commands (that download and install the prerequisites) from the actual build of our code and only retry the first group?

Still seeing this (and still on the list of our top few flakes).
https://sponge.corp.google.com/invocation?tab=Build+Log&id=f91bec72-924f-4657-a3c2-088f398ccb11
We really ought to fix this somehow, we can't have this failing forever.
@matt-kwong , do you have any ideas how we could reduce the flakiness rate?

The linked PR will hopefully decrease the frequency of this failure. Until we know what the impact of that PR is I am going to downgrade this to a P2.


remove i18n

i18n is:

large
difficult to use
shaky behind the scenes (see vite bundler not exiting headache)

we can do the str substitutions ourselves rather easily, and chris says sinisa already coded part/all of that. So other than detecting the lang, we don't need i18n. Rolling our own solution / using something else which is simpler and smaller would be best

everything is working, so this is low priority now. Removal reason is above + no dynamic string substitution, which is something we really need for translating target strings (e.g. dog) on the fly. Might be able to use google-translate?

i18n works for our use case. Needs AC if to be done. Moved to low priority for now


Add transaction hashes and uncles to the block that json-rpc returns

What was wrong?
Fix #279
How was it fixed?
To-Do

[ ] Clean up commit history
[x] Test that txn hashes match expected value
[x] ~Add uncles~ Hah right, there are no uncles post-merge.

Ok, ready for review! It's probably easiest to separate reviews out into the linked PRs and then the remaining piece here.
I'll consider a 👍🏻 here as only about the part unique to this PR. I will handle any blocking comments on other PRs first, so the comments on this PR can be focused to the unique part of it. Currently:

chore: add From<H256> for BlockBodyKey
3de9a87
refactor(rpc): fetch & decode any content value
6226964
feat(rpc): add txn hashes in eth_getBlockByHash
5d1a26f

I'm confused, does this PR need review only on two commits: 50e1ce6fcc1f0c34692d645b182367ae8ff9e40a and cd78ef831b000a027f6ecfa3c6504137a83c59ee ?

I'm confused, does this PR need review only on two commits: 50e1ce6 and cd78ef8 ?

Nope, the opposite. :) The three commits I listed are the ones that are for this PR review.

Nice! I guess next is to add support for hydrated txs?

Yup! I'm working on a local copy now. I'll get my WIP for that up today or tomorrow


More precision in the OETH swap/wrap forms

One or two of the stablecoins for OUSD only supported six digits. Now that all of the ETH derivatives support 18, we should allow for greater precision in the dapp. At a minimum, the output field should have the same level of precision as the input field.

While this has been improved, I'm going to open it again for us to consider down the road. As a user, @sumpg41 expects to see greater precision.

The last few digits cut off, says 0.001199 but should say 0.00119955

Related: https://github.com/OriginProtocol/origin-dollar/issues/1395


Unaccessible Administration because of Mixed Content Block

PHP Version
7.4.25
Shopware Version
<IP_ADDRESS>
Expected behaviour
When Shopware is deployed behind a loadbalancer and served through HTTPS, the administration login should proceed.
Actual behaviour
When Shopware is deployed behind a loadbalancer and served through HTTPS, the following error is thrown in https://example.com/bundles/administration/static/js/vendors-node.js:
Mixed Content: The page at 'https://example.com/admin#/login' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://example.com/api/oauth/token'. This request has been blocked; the content must be served over HTTPS.

The TRUSTED_PROXIES environment variable is set to REMOTE_ADDR and config/packages/<myenvironment>/framework.yaml contains the following:
framework:
  trusted_headers: ['x-forwarded-proto', 'x-forwarded-for', 'x-forwarded-host', 'x-forwarded-port']

as per documentation here.
I believe this is caused by misconfugration of the apiContext in JavaScript:
<script nonce="">
        Shopware.Application.start({
        apiContext: {
            host: 'example.com',
            port: 80,
            scheme: 'http',
            schemeAndHttpHost: 'http://example.com',
            uri: 'http://example.com/admin',
            basePath: '',
            pathInfo: '/admin',
            liveVersionId: '0fa91ce3e96a4bc2be4bd9ce752c3425',
            systemLanguageId: '2fbb5fe2e29a4d70aa5854ce7ce3e20b',
            apiVersion: 4,
            assetPath: 'https://cdn.example.com'
        },
        ...
    });
</script>

However, when adding the following after this line, it seems that Symfony is correctly configured:
dump($request->getScheme());
dump($request->getSchemeAndHttpHost());
exit();

which outputs:
https
https://example.com

I'm running Shopware on a Google Cloud Loadalancer with GKE as backend.
How to reproduce
Add the following to config/packages/<myenvironment>/framework.yaml:
framework:
  trusted_headers: ['x-forwarded-proto', 'x-forwarded-for', 'x-forwarded-host', 'x-forwarded-port']

Set TRUSTED_PROXIES to REMOTE_ADDR and add the following to this line in public/index.php:
$_SERVER['HTTP_X_FORWARDED_PROTO'] = 'https';
$_SERVER['HTTP_X_FORWARDED_HOST'] = '<IP_ADDRESS>';
$_SERVER['HTTP_X_FORWARDED_FOR'] = '<IP_ADDRESS>';

Reload the page and inspect the apiContext. After add the following on this line:
dump($request->getScheme());
dump($request->getSchemeAndHttpHost());
exit();

Seems to be fixed in 0371ffc.

Yep it should be fixed by the linked commit.
The linked commit is included in the latest <IP_ADDRESS> release, so an upgrade should fix the issue.
Additionally there was already another issue regarding this: https://github.com/shopware/platform/issues/2099.
Nevertheless thanks for the detailed bug :bug:  description


v2: Wind speed color inverted on white background

Hi,
On white background, the wind speed color is counterintuitive (before it becomes red). Prominently visible dark gray denotes small wind, less prominent light gray numbers denote stronger wind.

Could you maybe consider an option to flip them?

Yes, it is possible. Does the I option works for you? (for the main version) Should it be the same here? (v2)

Am I doing it right: curl 'wttr.in/Budapest?I' ?
I get somewhat different colors: darker clouds and Sun, and darker colors for the temperature. The Sun's color is IMHO too dark, for the rest the result is easier to read.
I can't see such a grayscale thing here though that could help me confirm that this is what I'm exactly looking for. But I trust you that it makes sense to do this for v2, too :)
(Side note: this I isn't mentioned on :help.)

A new idea, @egmontkob what do you think about it?
Display speed of wind in several lines instead of one,
where the height of the plot is depending on the wind speed.
Maybe even the Beaufort number for this windspeed divided
by two (or three), so that the fastest wind with the hurricane speed,
will be displayed with a 6 lines block (or 4 lines) and light breeze with one character.
Something like this:
│          ↙  ←  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙              →  →  → │
│    ↙  ↙  ↙  ←  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙           →  →  →  → │
│ ↙  ↙  ↙  ↙  ←  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↖  ↓  ↘  →  →  →  → │
│ ↙  ↙  ↙  ↙  ←  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↙  ↖  ↓  ↘  →  →  →  → │
│ 14 17 19 23 30 32 35 34 33 32 33 30 31 31 29 24 21 19 18 15 18 22 24 25│

In my opinion, color alone is not enough to represent the windspeed,
and it is better to use something with variable size for that.
What do you think?

I'm not sure. I think I'd prefer the colored, single-line variant.


【v1.4.6】Useless warning  in SyncHttpClient

When use SyncHttpClient in individual Thread（1.4.6）,  i received warnning :
Current thread has not called Looper.prepare(). Forcing synchronous mode.
And I go through the source code to find the reason:
Once I create a responseHandler  it will call  setUseSynchronousMode(false) , but because its in a new thread， so it's looper is null , and execute this code:
@Override
    public void setUseSynchronousMode(boolean sync) {
        // A looper must be prepared before setting asynchronous mode.
        if (!sync && looper == null) {
            sync = true;
            Log.w(LOG_TAG, "Current thread has not called Looper.prepare(). Forcing synchronous mode.");
        }

       ....

        useSynchronousMode = sync;
    }

I don't think it's wrong to use SyncHttpClient in this way, Hope to improve warnning message(^_^)， it's sometimes confused.

Is there any reason, why you opened another ticket?
I'm closing this as duplicate of #828

I'm getting this warning too. I'm calling SyncHttpClient inside onRunTask of a GcmTaskService, if that helps debug it.
Nothing fancy, just a
SyncHttpClient client = new SyncHttpClient();
ByteArrayEntity entity = // some data
client.post(this, "http://service.com/endpoint", entity, "application/json", resonseHandler)


{Core} Remove useless codes in core for ai-did-you-mean-this

Description

This PR removes the useless codes in core for ai-did-you-mean-this, as the extension was already retired.

ai-did-you-mean-this is retired: https://github.com/Azure/azure-cli-extensions/pull/2402
azext_ai_did_you_mean_this is removed from core #15500

@christopher-o-toole @mirdaki for awareness

I think there may be something more to remove. See #12889 for the original change.

I have no problem with this.

Some codes introduce by #12889 are not removed because they are still used in command recomendations

/azp run


move tensorlike operations to builtin operators

Signed-off-by: jie.hou<EMAIL_ADDRESS>

/approve
/lgtm