Negative RAM usage

Bug description
I have been following DCGAN Tutorial with PyTorch, and ran in my jupyter environment.
I tried to show summary ant I got the result with negative RAM usage like Framework & CUDA overhead: -390.33 Mb
I have 4 GPUs but use only 1 GPU for this script by config:
os.environ['CUDA_LAUNCH_BLOCKING'] = "1"
os.environ["CUDA_VISIBLE_DEVICES"] = "0"

here is the define of model.
nz = 200 # Size of z latent vector
ngf = 64 # Size of feature maps in generator
nc = 3 # Number of channels in the training images.

class Generator(nn.Module):
    def __init__(self):
        super(Generator, self).__init__()
        self.main = nn.Sequential(
            nn.ConvTranspose2d(nz, ngf * 8, 4, 1, 0, bias=False),
            nn.BatchNorm2d(ngf * 8),
            nn.ReLU(True),
            nn.ConvTranspose2d(ngf * 8, ngf * 4, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 4),
            nn.ReLU(True),
            nn.ConvTranspose2d(ngf * 4, ngf * 2, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf * 2),
            nn.ReLU(True),
            nn.ConvTranspose2d(ngf * 2, ngf, 4, 2, 1, bias=False),
            nn.BatchNorm2d(ngf),
            nn.ReLU(True),
            nn.ConvTranspose2d(ngf, nc, 4, 2, 1, bias=False),
            nn.Tanh()
        )

    def forward(self, input):
        return self.main(input)

Code snippet to reproduce the bug
the result of summary(netG, (nz, 1, 1)):
__________________________________________________________________________________________
Layer                        Type                  Output Shape              Param #
==========================================================================================
generator                    Generator             (-1, 3, 64, 64)           0
├─main                       Sequential            (-1, 3, 64, 64)           0
|    └─0                     ConvTranspose2d       (-1, 512, 4, 4)           1,638,400
|    └─1                     BatchNorm2d           (-1, 512, 4, 4)           2,049
|    └─2                     ReLU                  (-1, 512, 4, 4)           0
|    └─3                     ConvTranspose2d       (-1, 256, 8, 8)           2,097,152
|    └─4                     BatchNorm2d           (-1, 256, 8, 8)           1,025
|    └─5                     ReLU                  (-1, 256, 8, 8)           0
|    └─6                     ConvTranspose2d       (-1, 128, 16, 16)         524,288
|    └─7                     BatchNorm2d           (-1, 128, 16, 16)         513
|    └─8                     ReLU                  (-1, 128, 16, 16)         0
|    └─9                     ConvTranspose2d       (-1, 64, 32, 32)          131,072
|    └─10                    BatchNorm2d           (-1, 64, 32, 32)          257
|    └─11                    ReLU                  (-1, 64, 32, 32)          0
|    └─12                    ConvTranspose2d       (-1, 3, 64, 64)           3,072
|    └─13                    Tanh                  (-1, 3, 64, 64)           0
==========================================================================================
Trainable params: 4,395,904
Non-trainable params: 0
Total params: 4,395,904
------------------------------------------------------------------------------------------
Model size (params + buffers): 16.78 Mb
Framework & CUDA overhead: -390.33 Mb
Total RAM usage: -373.56 Mb
------------------------------------------------------------------------------------------
Floating Point Operations on forward: 883.95 MFLOPs
Multiply-Accumulations on forward: 442.07 MMACs
Direct memory accesses on forward: 446.39 MDMAs
__________________________________________________________________________________________

and this is result for comparing with other module (torchsummary)
the result of torchsummary.summary(netG, (nz, 1, 1)):
----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
   ConvTranspose2d-1            [-1, 512, 4, 4]       1,638,400
       BatchNorm2d-2            [-1, 512, 4, 4]           1,024
              ReLU-3            [-1, 512, 4, 4]               0
   ConvTranspose2d-4            [-1, 256, 8, 8]       2,097,152
       BatchNorm2d-5            [-1, 256, 8, 8]             512
              ReLU-6            [-1, 256, 8, 8]               0
   ConvTranspose2d-7          [-1, 128, 16, 16]         524,288
       BatchNorm2d-8          [-1, 128, 16, 16]             256
              ReLU-9          [-1, 128, 16, 16]               0
  ConvTranspose2d-10           [-1, 64, 32, 32]         131,072
      BatchNorm2d-11           [-1, 64, 32, 32]             128
             ReLU-12           [-1, 64, 32, 32]               0
  ConvTranspose2d-13            [-1, 3, 64, 64]           3,072
             Tanh-14            [-1, 3, 64, 64]               0
================================================================
Total params: 4,395,904
Trainable params: 4,395,904
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 3.00
Params size (MB): 16.77
Estimated Total Size (MB): 19.77
----------------------------------------------------------------

Error traceback
No error message.
Environment

TorchScan version: 0.1.1
PyTorch version: 1.11.0+cu102
OS: Ubuntu 18.04.5 LTS
Python version: 3.9.4
Is CUDA available: Yes
CUDA runtime version: Could not collect
4 GPUs
Nvidia driver version: 460.27.04

I would prefer not to share environment details, sorry for having problem with security agreement.

Hi @frgfm
I got exactly same problem on kaggle notebook having 1 GPU.
here is the link, you can see the output: https://www.kaggle.com/joonasyoon/wgan-cp-with-celeba-and-lsun-dataset
When there is two model and summary them, first one make zero RAM usage, and second one has negative.

Alright, I think I found a solution in #64 :+1:
@joonas-yoon would you mind trying to install the "negative-ram" branch and check whether that solves your problem?
Side note: what you experienced in Kaggle (that the second model will have a different RAM value) won't be fixed by this as GPU RAM usage is reported as blended between all objects.

oh! that is good news. I will install in directly to check it :)
and thanks for information note.


Call to /api/v1/repos/{:user}/{:repo}/branches/{:branch} fails unexpectedly

Gitea version (or commit ref): 1.12.0+dev-33-gc61b90253 running from Docker
Operating system: Linux gitea 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11) x86_64 GNU/Linux
Database (use [x]):

[ ] PostgreSQL
[x] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[x] No
[ ] Not relevant

Log gist:

2020/03/25 05:01:05 ...ules/context/repo.go:789:func1() [E] GetCommitsCount: Unsupported cached value type: <nil>

Description
The call to /api/v1/repos/{:user}/{:repo}/branches/{:branch} seems to fail unexpectedly and seemingly randomly, possibly due to high I/O load on machine.
It breaks havoc with Renovate Bot making it mark PRs for which request failed as edited.

do this still happen @CirnoT ?

I was unable to reproduce it ever since I've switched to memcached as cache storage and configured higher cache timeouts.


wrong video width and height

i had a issue which video width and height was wrong on android
After looking on code i noticed this
https://github.com/jonataslaw/VideoCompress/blob/e51b3eb0f357f5e478e1fd4e51a87219155ed178/android/src/main/kotlin/com/example/video_compress/Utility.kt#L14
should be
fun isLandscapeImage(orientation: Int) = orientation == 90 || orientation == 270

+1 in portrait videos the width and height are reversed

In my case both portrait and landscape videos give reversed width and height


fix(statefulsets): port name not included in headless service (#3077)

Backport
This will backport the following commits from k8s-28/main to k8s-27/main:

fix(statefulsets): port name not included in headless service (#3077)

Questions ?
Please refer to the Backport tool documentation

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 3991

Questions ?
Please refer to the Backport tool documentation


runtime: sysmon can invoke write barriers without a P

sysmon runs without a P (that is, getg().m.p.ptr() == nil), but I believe it can invoke write barriers in certain unlikely (but possible) scenarios. If this happens, the write barrier will panic with "writebarrierptr_nostore1 called with mp.p == nil".
The common paths are all free of write barriers (sysmon itself, retake, and most of the functions called directly by them), but there are calls to record trace events, which are known to have write barriers when the trace buffer fills up, and other complex things like netpoll and checkdead that may not have write barriers but it's not at all obvious to me.
I don't know what the answer to this is. Possibly sysmon needs something like we were considering for the GC: a special P that is dedicated to sysmon and doesn't count against GOMAXPROCS. It would also need modifications to at least its lengthy usleep calls, which wouldn't be interruptable by STW (but it doesn't matter right now since there's no P to stop anyway).
The most recent specific instance of this I can find was on netbsd-386-minux in commit 41dbcc1:
http://build.golang.org/log/c5098592bfbca60c15071fdf60427372c480a396
This specific one has long since been fixed, but shows evidence of sysmon running without a P.
@dvyukov @RLH @rsc

I've managed the eliminate almost all of the write barriers from sysmon, but, alas, one remains: the tracing code calls gcallers, which certainly has write barriers. So, I still can't apply "go:nowritebarrierrec" to sysmon. However, I think it's safe after the CLs I just mailed because dynamically we can't reach the call gcallers from sysmon because there's never a G.
The pain of all of this really makes me think sysmon should just have a P of its own that it drops when it goes to sleep.


Update CocoaPods to the latest release 1.1.5

Hi guys,
I released 1.1.5 with the following merges:

Fixed intrinsicContentSize size calculation thanks to @lancheg https://github.com/optonaut/ActiveLabel.swift/pull/313
Added email support based on the URL link thanks to @mehulmodihb https://github.com/optonaut/ActiveLabel.swift/pull/354
Added in missing cases for new UITouchPhases, iOS 13.4 thanks to @casperson https://github.com/optonaut/ActiveLabel.swift/pull/360
Adjust the height to zero when there's no text set thanks to @filipealva https://github.com/optonaut/ActiveLabel.swift/pull/367

Everything was set and passed for CocoaPods, just needs someone with permission to publish it.
Many thanks, @efremidze @ejoebstl @schickling
https://github.com/optonaut/ActiveLabel.swift/releases/tag/1.1.5

Just to check back if you guys missed the notification (I don't have permission to push it on CocoaPods): @efremidze @ejoebstl @schickling
Thanks again


Discuss the possibility of adding device properties with object aggregation

Some usecases seem to map naturally to this pattern. While they can be implemented also with the existing infrastructure, it can be useful to discuss the possibility of adding the ability of installing interfaces that publish device properties with an object aggregation.

Duplicate of #503


[Feature Request] Add more readline compatible keybinding feature.

Following is most useful keybinding AFAIK.
You should be able to reproduce them in any linux/mac shell (e.g. konsole, iTerm2) use bash as shell.
Features

[x] C-f → move cursor forward char
[x] A-f → move cursor forward word
[x] C-b → move cursor back forward char
[x] A-b → move cursor back forward word
[x] C-d → delete after char on cursor. (exit to REPL only no char in current line)
[x] A-d → delete after word on cursor.
[x] A-Backspace → delete before word on cursor.
[x] C-k → delete to end of line
[x] C-u → delete to beginning of line.

C-k, C-u seem like works well on my linux laptop,  i marked as completed.
thank you.

Thank you for making the list!, the others are coming soon :)

Everything done!

Everything done!

Cool, it works! maybe a new release?


mc panning

[ ]  pan8 is actually 'spat'? Rename it, make it compiled and with arbitrary N of outs? That is, delete it and have a new object?

[ ] - New object: pan~ (like PanAz, basically SpreadAz with one input....) add level in and .mc~ verson or add cartesian point and make it like a 'spat~'?

[ ] a new [vbap~] object

[ ] - add 'spread' parameter to spread~ (when output channels is 2, spread~ is like pan.spread~ or Splay.ar in SC! It is more versatile, but it doesn't have the center/spread thing which is cool. I want a spread~ object with that function.)... SplayAz can spread in more channels, like I wanted spread~ to do so. Need to test if it can behave the same as Splay when output is 2 channels! ==> No, It Can't, spread works quite differently...

[ ] - xgate2~ add '-mc' flag

[ ] - rethink, recheck [xselect2~], make it like xgate2~, add 'spread' parameter, etc (see SelectXFocus from SC)

[ ]  think about a way to do xgate with 'threno' paradigm of fade in/out

also ambisonics
ambipan~ cicm tools
and HOA

https://ssr.readthedocs.io/en/0.6.1/renderers.html#vbap

Ville Pulkki’s Vector-base amplitude panning (VBAP) is a quite robust and generic algorithm that works on nearly any surrounding loudspeaker layout. VBAP activates the smallest possible number of loudspeakers, which gives a directionally robust auditory event localization for virtual sound sources, but it can also cause fluctuations in width
and coloration for moving sources.
Multiple-direction amplitude panning (MDAP) is a modification that increases the number of activated loudspeakers. In this way, more direction-independence is achieved at the cost of an increased perceived source width and reduced localization accuracy at off-center positions. As vector-base panning methods rely on convex hull triangulation, irregular loudspeaker layouts yielding degenerate vector bases can become a problem.

vamos re-escrever mtx~ e autofades... e essas coisas de como copiar as entradas.

vamos re-escrever mtx~ e autofades... e essas coisas de como copiar as entradas.


Incredibly slow in powershell 7

Doing something as simple as this results in 4.3510603 seconds wait time:
Measure-Command {Write-Debug "Next"
$Element = Find-SeElement -Driver $Driver -XPath "/html/body/div/div/div/div/div/div/button"
Invoke-SeClick -Element $Element}

Whereas in PowerShell 5, it takes 0.3669819 seconds, so using powershell 7 is significantly impacting performance.
I will add, there is some mention to 7 being slower here, but it's surrounding web sessions using the invoke-restmethod command, and there's a fix for it:
https://github.com/PowerShell/PowerShell/issues/12764

That's quite a long time.... hmm. Out of curiosity, can you double check the type of $Element there when it comes back ($Element.GetType().FullName)?


模式模式是广播还是组播，能否自定义

Is default broadcast or mutilcast？

default is broadcast

hi, when i start the config, i obtain this on ios  : " client: sendto fail, but just ignore it: No route to host" and device not found.

HI @rvalentini1 , please see it. I will fix it soon.
https://github.com/EspressifApp/EsptouchForIOS/issues/53


echo-bot existed with error

[discord-websockets.cpp:477] ws_send_heartbeat()
        HEARTBEAT_PAYLOAD:
                {"op":1,"d":1}
[discord-websockets.cpp:298] ws_on_text_cb()
        ON_TEXT:
                {"t":null,"s":null,"op":11,"d":null}
        OP:             GATEWAY_HEARTBEAT_ACK
        EVENT_NAME:     NULL
        SEQ_NUMBER:     1
        EVENT_DATA:     null
[discord-websockets.cpp:341] ws_on_text_cb()
        PING: 93 ms
[http-common.c:236] perform_request()
        ERROR:  Assert Failed:  Error
        Expected:       CURLE_OK == ecode
Aborted

Really strange error which I assume not to be on our side. But we should be able to handle it

https://curl.se/libcurl/c/CURLOPT_ERRORBUFFER.html
This may be useful.

Another error
[discord-websockets.cpp:361] ws_on_text_cb()
        ON_TEXT:
                {"t":null,"s":null,"op":10,"d":{"heartbeat_interval":41250,"_trace":["[\"gateway-prd-main-d8bm\",{\"micros\":0.0}]"]}}
        OP:             GATEWAY_HELLO
        EVENT_NAME:     NULL
        SEQ_NUMBER:     1
        EVENT_DATA:     {"heartbeat_interval":41250,"_trace":["[\"gateway-prd-main-d8bm\",{\"micros\":0.0}]"]}
[discord-websockets.cpp:111] ws_send_identify()
        ERROR:  Reach identify requests threshold (1 every 5 seconds)
Aborted

Another error
[discord-websockets.cpp:361] ws_on_text_cb()
        ON_TEXT:
                {"t":null,"s":null,"op":10,"d":{"heartbeat_interval":41250,"_trace":["[\"gateway-prd-main-d8bm\",{\"micros\":0.0}]"]}}
        OP:             GATEWAY_HELLO
        EVENT_NAME:     NULL
        SEQ_NUMBER:     1
        EVENT_DATA:     {"heartbeat_interval":41250,"_trace":["[\"gateway-prd-main-d8bm\",{\"micros\":0.0}]"]}
[discord-websockets.cpp:111] ws_send_identify()
        ERROR:  Reach identify requests threshold (1 every 5 seconds)
Aborted

Did you try to run two bots with the same token in parallel?

No. They use different tokens.

On Feb 8, 2021, at 5:53 AM, Lucas Müller<EMAIL_ADDRESS>wrote:

Another error
[discord-websockets.cpp:361] ws_on_text_cb()
ON_TEXT:
{"t":null,"s":null,"op":10,"d":{"heartbeat_interval":41250,"_trace":["["gateway-prd-main-d8bm",{"micros":0.0}]"]}}
OP:             GATEWAY_HELLO
EVENT_NAME:     NULL
SEQ_NUMBER:     1
EVENT_DATA:     {"heartbeat_interval":41250,"_trace":["["gateway-prd-main-d8bm",{"micros":0.0}]"]}
[discord-websockets.cpp:111] ws_send_identify()
ERROR:  Reach identify requests threshold (1 every 5 seconds)
Aborted
Did you try to run two bots with the same token in parallel?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

I don't see the error again.


Chore: Improve Engine Adapter Abstraction for Handling DataFrames

The core change is to create a single method (_df_to_source_query) that can be overridden by any engine to take a Dataframe and create a query. They just need to return a generator that creates expressions. Since it is a generator they can also define their own setup/teardown logic around each expression. This covers all known implementations of either converting pandas to SQL, loading pandas to temp table and then reading from the table, or create a tempView out of a PySpark Dataframe.
One known regression is that for BigQuery and Snowflake we now always load to a temp table from a Pandas Dataframe and then copy those contents into the target table when in some cases we could have loaded the target directly. Since Pandas Dataframes are smaller data I doubt this have much actual impact and can be improved in a follow-up PR.
TODO:

Run end-to-end on some engines (BQ, Snowflake, Databricks)

Follow Up:

Make columns_to_types required in SourceQuery

I don't like SourceQuery 😿

let's discuss this PR tomorrow in a meeting, it will probably be faster


Memory table revamp.

I'm dissatisfied with several features of the way memory tables are handled.

All documents and vocabulary must be stored in memory, although it would be nice to allow non-memory builds especially on non-dedicated servers.
Memory tables must be reloaded, but this requirement is hard to automate and non-obvious to all users.
The queries that create memory tables rely on some complicated joins that can take a while (I've seen hours) to execute. This means a system restart takes a bookworm offline for hours.

Current situation
The memory tables speed things up, or did when I benchmarked things five years ago.
There are currently up to four tables for each variable. Let me take the language field from hathipd as an example of them.
On disk tables
The first is the raw data imported from the user. This is keyed to some id variable--usually, but not necessarily, bookid. The field itself is text.
MariaDB [hathipd]> SELECT * FROM languagesDisk LIMIT 3;
+--------+-----------+
| bookid | languages |
+--------+-----------+
|      1 | English   |
|      2 | English   |
|      3 | English   |
+--------+-----------+
3 rows in set (0.00 sec)

A SQL query then generates a table of ids and counts for each language.
The ids are useful in queries because they make grouping and filtering queries significantly faster.
The ids are in decreasing prevalence; this isn't strictly necessarily, but enables a very useful query restraint I use a lot that uses only the most common elements. {"languages__id":{"lte":10}}.
In order to get this property, the __id field is actually created by this query. It might be possible to create in python.
The __count field is not, AFAIK, used anywhere and could safely be deleted.
SELECT * FROM languages__id LIMIT 3;
+---------------+-----------+------------------+
| languages__id | languages | languages__count |
+---------------+-----------+------------------+
|             1 | English   |          2793265 |
|             2 | German    |           578369 |
|             3 | French    |           541092 |
+---------------+-----------+------------------+
3 rows in set (0.00 sec)

In memory tables
Those tables persist through restart, but are not used in queries. Instead, two other queries are used.
The first is the Lookup table that does a hashmap from ids to keys.
If you search for "languages":["English"], this is the field that will be checked during the search for each book. (I assume, though I'm not sure, that after the first hit mysql will actually cache the proper languages__id and this table isn't actually consulted 10 million times).
This table is just about the same as languages__id, except that

It lacks a languages -> languages__id index, because those sometimes confuse the query optimizer.
It lives in memory
It lacks the useless count field.

SELECT * FROM languagesLookup LIMIT 3;
+---------------+-----------+
| languages__id | languages |
+---------------+-----------+
|             1 | English   |
|             2 | German    |
|             3 | French    |
+---------------+-----------+
3 rows in set (0.00 sec)

In order to hit that lookup table, there needs to be another memory table that maps from bookid (or other id) to the language id. This is called heap.
This memory table is created with a merge of two of the above tables.
SELECT * FROM languagesheap LIMIT 3;
+--------+---------------+
| bookid | languages__id |
+--------+---------------+
|      1 |             1 |
|      2 |             1 |
|      3 |             1 |
+--------+---------------+
3 rows in set (0.00 sec)

Solution
There's potentially a single solution for all three of the above problems.

If the Lookup table is empty, use the __id table instead (with index hints to avoid paralyzing wrong-way lookups).
Create a new table on disk with the bookid->languages__id map (like languagesheap). This will make creation of the languagesheap table faster.
On db queries, try to use the memory tables; if they don't exist (or have no rows, indicating a restart) fall back to using the on-disk versions.

Turning on memory tables then becomes a specific performance optimization you can turn on specific bookworms, without requiring a huge memory overhead for every bookworm.
Namespace issues
It would be nice to do a full rename of all these tables, but that might break some existing bookworms. So instead I think the path of least resistance is to add two new disk tables, called
languagesheap_ and languages__id_. (i.e., trailing underscores). If the memory table is empty or nonexistent, they'll be used instead; and the creation of the memory table will be from this table, rather than from the other on-disk tables. (The old on-disk tables could be deleted, I guess).
base tables
This process can apply to the base tables as well. The disk -> memory pattern also exists in the table pairs

catalog->fastcat
words -> wordsheap

The creation of these tables (especially fastcat) is going to be a big pain on the new hathi bookworm; so the existence of a fastcat_ table will help there a lot.

Tagging @organisciak to make sure that this is compatible with the current state of the hathi build.

Preliminary version pushed to memory_tables branch.
Still to do:

Allow bookworm_reload_memory only materialize certain memory tables. (fastcat is epecially important).
Make sure that overwriting an existing variable with new values isn't broken (this should be a new unit test.)
??? Looks relatively complete to me right now.


AttributeError: 'NoneType' object has no attribute 'findAll'

Got an error collecting the cliffnotes summaries (see below). Looks as if there is a problem, when some element (here article) is missing:
Traceback (most recent call last):
  File "scripts/data_collection/cliffnotes/get_summaries.py", line 95, in <module>
    section_paragraphs = list(filter(None, scrape_section_continuation(soup, section_header)))
  File "scripts/data_collection/cliffnotes/get_summaries.py", line 41, in scrape_section_continuation
    section_paragraphs = [paragraph.text.strip() for paragraph in section_data.findAll("p", recursive=False)]
AttributeError: 'NoneType' object has no attribute 'findAll'

Thank you for creating the issue, we will look into it as soon as possible.

@fotisj we've fixed this issue with collecting cliffnotes data. Thanks for bringing this to our attention.


Add demonstrating proof of possession

This is the second part of splitting up #1361 - hopefully it is easier to review.
Addresses/highlights https://github.com/authts/oidc-client-ts/issues/1360
As per the issue, this is an implementation of Demonstrating Proof of Possession (DPoP) to constrain access/refresh tokens to devices.
This is mostly spec complete - though I have left out some parts that I don't think are appropriate to the oidc-client-ts use case (such as Pushed Authorisation Requests). I have also left out support for Authorization Server supplied nonces, which I would like to address in a separate PR in the future (this is optional in the spec).
I've used the react-odic-client library example app and npm linked it to my local version of this PR. I used a fork of IdentityServer's DPoP examples to develop and test against, if you are interested in trying it out for yourself.
Things of note:

This adds a new set of options specifically for DPoP to the UserManagerSettings and introduces a new service - the DPoPService. This is off by default.
It makes use of IndexedDb, which allows storing non extractable key material used to generate the DPoPProofs. This means we can still use the stored keys to generate DPoP proofs, even though the key values themselves can't be accessed by the browser directly. This is critical to this functionality and without it DPoP is pointless in the browser context as the keys can just be exfiltrated by a bad actor alongside any tokens if stored in plain text. Storing key material this way prevents an attack where a bad actor may be able to exfiltrate tokens, because the actor cannot generate the required proof outside of the browser/session context.

Checklist

[ ] This PR makes changes to the public API
[ ] I have included links for closing relevant issue numbers

Will work on getting the code coverage up over the next week or so.

I am thinking about if it makes sense to add an extended userStore for dpop. Within that store we could handle/ isolate stuff like await IndexedDbCryptoKeyPairStore.remove("oidc.dpop");. We might even merge this new userStore  class with IndexedDbCryptoKeyPairStore...
export class DPopStorageStateStore extends WebStorageStateStore {
  // ... handles indexeddb stuff and dpop related things needed to be done differently than without dpop
}
```

I am thinking about if it makes sense to add an extended userStore for dpop. Within that store we could handle/ isolate stuff like await IndexedDbCryptoKeyPairStore.remove("oidc.dpop");. We might even merge this new userStore  class with IndexedDbCryptoKeyPairStore...
export class DPopStorageStateStore extends WebStorageStateStore {
  // ... handles indexeddb stuff and dpop related things needed to be done differently than without dpop
}

I'll give it a try and see what I come up with 👍

I am thinking about if it makes sense to add an extended userStore for dpop. Within that store we could handle/ isolate stuff like await IndexedDbCryptoKeyPairStore.remove("oidc.dpop");. We might even merge this new userStore  class with IndexedDbCryptoKeyPairStore...
export class DPopStorageStateStore extends WebStorageStateStore {
  // ... handles indexeddb stuff and dpop related things needed to be done differently than without dpop
}

Okay, I had a Quick Look...
When you say extend the WebStorageStateStore class, what benefit do we get from this?
Because we are dealing with the CryptoKeyPair type when dealing with IndexedDb (this is the type required to be able to store the key material as non extractable and also the type/supertype used by the crypto.subtle library), we can't override the set, get, remove and getAllKeys methods on the class because the StateStore interface specifies the types <string | null>.
So adding additional methods, such as getDPoPKeyPair is fine, but we don't really need the methods set, get, etc that are exposed by WebStorageStateStore because we don't need the session/local storage for anything DPoP currently (we might do if/when we add support for server side nonces though).
We could change the StateStore interface to allow for CyptoKeyPair, e.g. something like:
export interface StateStore<T extends string | null | CryptoKeyPair> {
    set(key: string, value: T): Promise<void>;
    get(key: string): Promise<T>;
    remove(key: string): Promise<T>;
    getAllKeys(): Promise<string[]>;
}

And implement that interface directly e.g. export class DPoPStorageStateStore implements StateStore<CryptoKeyPair | null> but that would mean making slight changes to the classes that already implement it as well I think (to specify the types). If we did this and decided to add the ability to also store DPoP nonces as strings in local/session storage through the same class, we could then widen the accepted types to <string | null | CryptoKeyPair> and add logic to put nonces in local/session storage and key material in IndexedDB.
Anyway, I suppose I'm not fully clear on what you are after as far as extending the WebStorageStateStore class.

Anyway, I suppose I'm not fully clear on what you are after as far as extending the WebStorageStateStore class.

I was more thinking about this:
your MR:
public async removeUser(): Promise<void> {
        const logger = this._logger.create("removeUser");
        await this.storeUser(null);
        logger.info("user removed from storage");
        if (this.settings.dpopSettings.enabled) {
            await IndexedDbCryptoKeyPairStore.remove("oidc.dpop");
            logger.debug("removed dpop cyptokeys from storage");
        }
        await this._events.unload();
    }

idea:
public async removeUser(): Promise<void> {
        const logger = this._logger.create("removeUser");
        await this.storeUser(null);
        logger.info("user removed from storage");
        /* not needed, as its part of the new `userStore` and done automatically when calling "await this.storeUser(null);"
        if (this.settings.dpopSettings.enabled) {
            await IndexedDbCryptoKeyPairStore.remove("oidc.dpop");
            logger.debug("removed dpop cyptokeys from storage");
        }*/
        await this._events.unload();
    }

...

Anyway, I suppose I'm not fully clear on what you are after as far as extending the WebStorageStateStore class.

I was more thinking about this:
your MR:
public async removeUser(): Promise<void> {
        const logger = this._logger.create("removeUser");
        await this.storeUser(null);
        logger.info("user removed from storage");
        if (this.settings.dpopSettings.enabled) {
            await IndexedDbCryptoKeyPairStore.remove("oidc.dpop");
            logger.debug("removed dpop cyptokeys from storage");
        }
        await this._events.unload();
    }

idea:
public async removeUser(): Promise<void> {
        const logger = this._logger.create("removeUser");
        await this.storeUser(null);
        logger.info("user removed from storage");
        /* not needed, as its part of the new `userStore` and done automatically removed when calling "await this.storeUser(null);"
        if (this.settings.dpopSettings.enabled) {
            await IndexedDbCryptoKeyPairStore.remove("oidc.dpop");
            logger.debug("removed dpop cyptokeys from storage");
        }*/
        await this._events.unload();
    }

// untested pseudo code only
export class DPopStorageStateStore extends WebStorageStateStore {
   public async remove(key: string): Promise<string | null> {
     const ret = super.remove(key);
     await IndexedDbCryptoKeyPairStore.remove("oidc.dpop");
     return ret;
   }
   ...
}

Okay, yep - and would you like to try and apply the same idea to all the methods on the WebStorateStateStore class - e.g. set, get, getAllKeys such that it can handle DPoP related data as well as he usual user attributes stored in local/session storage?

Okay, yep - and would you like to try and apply the same idea to all the methods on the WebStorateStateStore class - e.g. set, get, getAllKeys such that it can handle DPoP related data as well as the usual user attributes stored in local/session storage?

if possible yes. the idea is to simplify the dpop code paths and the not dpop code paths.
Stuff like:

User.ts (BTW: not used anywhere)

public async dpopProof(url: string, httpMethod?: string): Promise {
return await DPoPService.generateDPoPProof({ url, accessToken: this.access_token, httpMethod: httpMethod } );
}
- it contains directly code of `export class IndexedDbCryptoKeyPairStore` or hold the only reference to that object.

Okay, yep - and would you like to try and apply the same idea to all the methods on the WebStorateStateStore class - e.g. set, get, getAllKeys such that it can handle DPoP related data as well as the usual user attributes stored in local/session storage?

if possible yes. the idea is to simplify the dpop code paths and the not dpop code paths. Stuff like:

User.ts (BTW: not used anywhere)

public async dpopProof(url: string, httpMethod?: string): Promise { return await DPoPService.generateDPoPProof({ url, accessToken: this.access_token, httpMethod: httpMethod } ); }
- it contains directly code of `export class IndexedDbCryptoKeyPairStore` or hold the only reference to that object.

So I gave this a go but it's not possible to extend the WebStorageStateStore class with a DPoPStorageStateStore class without changing the types on the WebStorageState class (and the interface) to allow returning a CryptoKeyPair on the get method... which seems a bit gross, e.g.
public async get(key: string): Promise<string | null | CryptoKeyPair> {
        if (key === this.dpop_key) {
            return await IndexedDbCryptoKeyPairStore.get(this.dpop_key);
        } else {
            return await super.get(key);
        }
    }

doesn't work unless I change the signature of the parent WebStorageStateStore class to also return <string | null | CryptoKeyPair>
One option might be to alter the WebStorateStateStore to handle DPoP as well as what it already does (we'd still need to alter the StateStore interface).
What do you think?

One option might be to alter the WebStorateStateStore to handle DPoP as well as what it already does (we'd still need to alter the StateStore interface).

Maybe by adding a "new/additional" get like getDpop(): Promise<null | CryptoKeyPair>

One option might be to alter the WebStorateStateStore to handle DPoP as well as what it already does (we'd still need to alter the StateStore interface).

Maybe by adding a "new/additional" get like getDpop(): Promise<null | CryptoKeyPair>

Ok, I have had a rough go at adding a DPoPStorageStateStore for you to look at. I altered the StateStore interface to accept the CryptoKeyPair type, hopefully that makes sense. Also added the additional getDPoP method and inject the instance of the store into the DPoPService now.
Have a look and let me know what you think. If you're happy with the approach, I will move the IndexedDBCryptoKeyPairStore logic directly into the DPoPStorageStateStore and incorporate any other feedback you may have.
I'll be travelling for the next few weeks so may not be able to do too much until May 👍

Have a look and let me know what you think. If you're happy with the approach, I will move the IndexedDBCryptoKeyPairStore logic directly into the DPoPStorageStateStore and incorporate any other feedback you may have.
I'll be travelling for the next few weeks so may not be able to do too much until May 👍

I will have a deeper look at this in the next days. Thanks for keeping up on this.

Have a look and let me know what you think. If you're happy with the approach, I will move the IndexedDBCryptoKeyPairStore logic directly into the DPoPStorageStateStore and incorporate any other feedback you may have.
I'll be travelling for the next few weeks so may not be able to do too much until May 👍

I will have a deeper look at this in the next days. Thanks for keeping up on this.

Cool - I am back from travelling so can start working on this again when you have had a look 👍

Any thoughts on the latest commit @pamapa ? I am keen to progress this.

Was busy the last weeks, I have checked out your branch and I am now looking at this code and have some thoughts and time in the future to work with you...
General design

I am afraid of the additional code size for people who do not need DPoP, thus my first idea was to move everything into a sub class of UserManager, like DPoPUserManager, but when looking at the new code this does not seem feasible. The additional code is tight into the existing code.
By design OidcClient is for the protocol and UserManager is for "user related" stuff. DPoP is mostly protocol stuff except the access token from the user.
Thus maybe it really makes sense to have all but public async dpopProof( (access token needed), in the OidcClient class like you had in the beginning (sorry about that).

DPoPService:

i am not so sure it is worth to have this service at all, at the end its a bunch of crypto utils and getting the keys.
we could split the utils and getting the keys.
crypto utils code could be added to the CryptoUtils and adding a parameter to pass the keyPair

DPoPStorageStateStore/IndexedDbCryptoKeyPairStore:

when i look at the current (from me previously proposed idea) implementation of extending the WebStorageStateStore, its really hard to read and seems over complicated...
at the end is probably simpler if we simply add an additional optional store like dpopStore.
like this we can merge DPoPStorageStateStore & IndexedDbCryptoKeyPairStore into one and keep its as simple as possible
if somebody needs a different implementation of dpopStore he can easily do so.

DPoPSettings:

lets rename dpopSettings to dpop (it is already within the settings)
drop enable property (if dpop is undefined its like false if not its enabled)
add store

Something like:
let dpopJkt: string | undefined;
if (this.settings.dpop && this.settings.dpop.bind_authorization_code) {
  const keys = await this.settings.dpop.store.get();
  dpopJkt = await CryptoUtils.generateDPoPJkt({ keys });
}

Was busy the last weeks, I have checked out your branch and I am now looking at this code and have some thoughts and time in the future to work with you...
General design

I am afraid of the additional code size for people who do not need DPoP, thus my first idea was to move everything into a sub class of UserManager, like DPoPUserManager, but when looking at the new code this does not seem feasible. The additional code is tight into the existing code.
By design OidcClient is for the protocol and UserManager is for "user related" stuff. DPoP is mostly protocol stuff except the access token from the user.
Thus maybe it really makes sense to have all but public async dpopProof( (access token needed), in the OidcClient class like you had in the beginning (sorry about that).

DPoPService:

i am not so sure it is worth to have this service at all, at the end its a bunch of crypto utils and getting the keys.
we could split the utils and getting the keys.
crypto utils code could be added to the CryptoUtils and adding a parameter to pass the keyPair

DPoPStorageStateStore/IndexedDbCryptoKeyPairStore:

when i look at the current (from me previously proposed idea) implementation of extending the WebStorageStateStore, its really hard to read and seems over complicated...
at the end is probably simpler if we simply add an additional optional store like dpopStore.
like this we can merge DPoPStorageStateStore & IndexedDbCryptoKeyPairStore into one and keep its as simple as possible
if somebody needs a different implementation of dpopStore he can easily do so.

DPoPSettings:

lets rename dpopSettings to dpop (it is already within the settings)
drop enable property (if dpop is undefined its like false if not its enabled)
add store

Something like:
let dpopJkt: string | undefined;
if (this.settings.dpop && this.settings.dpop.bind_authorization_code) {
  const keys = await this.settings.dpop.store.get();
  dpopJkt = await CryptoUtils.generateDPoPJkt({ keys });
}

Thank you for the detailed feedback @pamapa, that is excellent and exactly what I was looking for. At a glance, that all makes sense and I will update this PR accordingly over the next week.
One minor clarifying implementation question for you regarding the new dpopStore: does it matter if it implements the storageStage interface? It's minor, but will require extending the interface as I have already done (to handle the CryptoKeyPair type).

One minor clarifying implementation question for you regarding the new dpopStore: does it matter if it implements the storageStage interface? It's minor, but will require extending the interface as I have already done (to handle the CryptoKeyPair type).

If we create a new storage type we have some flexibility. I guess when we decouple from StateStore its getting simpler for existing string cases and CryptoKeyPair.
My hope is that when someone does not use dpop, that store is removed when doing tree-shaking in production builds, as its not used/referenced directly. Its only passed via settings.

@pamapa I believe I have integrated most, if not all, of your most recent suggestions. I have also rebased with main. Please let me know your thoughts 🙏

Looks awesome, its very close to be merged. I have found some little style stuff (see below). Thanks a lot for being patient and taking care!
Question:

in OidcClientSettings.ts i see this.dpop.store = new IndexedDbDPoPStore(); It is done now like the other stores. The difference is that the other stores are essential, where the dpop is not. When we would not create here a new object, for everybody who does not use dpop, the IndexedDbDPoPStore could be tree-shaked (can you verify this?). This would mean when you use dpop you simple must pass that store via config, which is very easy... What do you think?
The above would also make this if (this.settings.dpop && this.settings.dpop.store) { into this if (this.settings.dpop) { as store is no longer an optional property in dpop...

@pamapa I believe I have integrated most, if not all, of your most recent suggestions. I have also rebased with main. Please let me know your thoughts 🙏

Looks awesome, its very close to be merged. I have found some little style stuff (see above & below). Thanks a lot for being patient and taking care!
Question:

in OidcClientSettings.ts i see this.dpop.store = new IndexedDbDPoPStore(); It is done now like the other stores. The difference is that the other stores are essential, where the dpop is not. When we would not create here a new object, for everybody who does not use dpop, the IndexedDbDPoPStore could be tree-shaked (can you verify this?). This would mean when you use dpop you simple must pass that store via config, which is very easy... What do you think?
The above would also make this if (this.settings.dpop && this.settings.dpop.store) { into this if (this.settings.dpop) { as store is no longer an optional property in dpop...

I see what you are saying: we would just need to be clear that an implementation of the dpopStore must be supplied, and that we provide an implementation that makes use of IndexDb as a default. I don't have a problem with that really, and you are right that in its current form the store is not tree shaked and adds about 1kb:

As far as overall build size goes, there is a 5kb difference between master and this branch when I bundle it
into a sample application using Parcel.

EDIT: I can confirm that changing the dpopStore to a non optional configuration parameter when dpop is enabled does allow the IndexedDbDPoPStore to be tree shaked. Do you think we should add a check that the store is actually configured when drop is enabled, and throw an exception if not?

We can explicitly throw an exception if the store is undefined, despite the type would enforce to not be undefined. Maybe hidden in the settings class...

I've also fixed all the style issues you high lighted. Would like me to add some documentation regarding the feature and how to use it to this PR?

Would be nice if you could add a new section in the docs/index.md. This can also be done in a next MR when this one has been merged...

EDIT: I can confirm that changing the dpopStore to a non optional configuration parameter when dpop is enabled does allow the IndexedDbDPoPStore to be tree shaked. Do you think we should add a check that the store is actually configured when drop is enabled, and throw an exception if not?

We can explicitly throw an exception if the store is undefined, despite the type would enforce to not be undefined. Maybe hidden in the settings class...

I've also fixed all the style issues you high lighted. Would like me to add some documentation regarding the feature and how to use it to this PR?

Would be nice if you could add a new section in the docs/index.md. This can also be done in a next MR when this one has been merged...

No problem, happy to do the docs in a different PR. For now I have annotated the IndexedDbDPoPStore as @public with a short description.
I have also updated with main and added a check in OidcSettings to throw an exception if drop is configured without a store.

Thank you for the contribution.  I noticed in your comments you removed the nonce option.  We implement it here within our organization.  Is it possible you could add the nonce back in?  We get passed the DPoP Proof but then are warned we need to supply the nonce value.
Can you provide me with a workaround if you do not add it to the source?  Any help is greatly appreciated and I do thank you for the contribution.

Thank you for the contribution. I noticed in your comments you removed the nonce option. We implement it here within our organization. Is it possible you could add the nonce back in? We get past the DPoP Proof but then are warned we need to supply the nonce value.
Can you provide me some help or a workaround? Any help is greatly appreciated and I do thank you for the contribution.

Hi there, yes I originally added a rough implementation but opted to remove it in favour of adding it in a separate PR (the PR was getting quite large, there was disagreement regarding the implementation and it is optional in the spec).
I will open another PR next week. Perhaps you would be willing to help test and review it?

Thank you for the contribution. I noticed in your comments you removed the nonce option. We implement it here within our organization. Is it possible you could add the nonce back in? We get past the DPoP Proof but then are warned we need to supply the nonce value.
Can you provide me some help or a workaround? Any help is greatly appreciated and I do thank you for the contribution.

Hi there, yes I originally added a rough implementation but opted to remove it in favour of adding it in a separate PR (the PR was getting quite large, there was disagreement regarding the implementation and it is optional in the spec).
I will open another PR next week. Perhaps you would be willing to help test and review it?

Thank you once again, great work!   I found your oidc-client-context and was able to get it working in my code.  But got stuck on the DPoP nonce requirement.
Yes absolutely I can test it and help anyway I can.  Again, appreciate your work and the author of the library - great work !

@psgfaa please take a look at #1569 🙏


Delayed BGM on resume for MBP

I am not sure if this is related with ebiten or it should be part of oto but let me write the issue here
The BGM resumes after certain delay after sleeping MacbookPro. The amount of the delay seems to correlate with the duration of the sleep.

This might be an issue in Oto. Let me take a look.


Use tags to execute test in firebase test lab with flutter/integration_test

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Use case
When I execute test with flutter/integration_test in firebase test lab, I compile the apk or xctesrun and I have to pass a target with the dart file that contains the tests.
For example I have the following files:
- group_test1.dart
- group_test2.dart
- main_test.dart (Contains both tests scenarios)

If I want to execute a group of test 1 I have to compile with the dart file "group_test1" and if I want to execute a group of test 2 I have to compile with the dart file "group_test2". This compilation in my case is slowly and it can take 50 minutes for each dart file.
Can I compile only with the "main_test.dart" and when I execute in firebase test lab pass a flag with a Tag or Regex that only execute a group of test?
Proposal
Can I compile only with the "main_test.dart" and when I execute in firebase test lab pass a flag with a Tag or Regex that only execute a group of test?
For example I compile:
pushd android
# flutter build generates files in android/ for building the app
flutter build apk
./gradlew app:assembleAndroidTest
./gradlew app:assembleDebug -Ptarget=main_test.dart
popd

I execute in firebase test lab something like
gcloud firebase test android run --type instrumentation \
  --app build/app/outputs/apk/debug/app-debug.apk \
  --test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk\
  --timeout 2m \
  --results-bucket=<RESULTS_BUCKET> \
  --results-dir=<RESULTS_DIRECTORY> \
 ---tags=@Group_test1 -> new parameter

@anfruiz
Can you take a look at this and see if it helps in your case or not ?

@darshankawar thank you, it can help me but only for android, do you know an example with gcloud firebase test ios run?

@anfruiz
See if this helps ?

@darshankawar do you have an example using the parameter --environment-variables and get this variables in the code? I'm using Platform.environment and String.fromEnvironment and these doesn't works

Thanks for the update. I'll treat this issue as a proposal to add tags to execute tags.

Hi @anfruiz, I think you may be interested in Patrol. It's an improved version of the default integration_test plugin that has better integration with device farms like Firebase Test Lab. Its biggest advantage is you can build all your tests once, which sounds important to you since you're saying the builds take a long time.
Then, when running the tests, they all execute in full isolation, so you don't have to worry about selecting only one or another to prevent overlap.

Hi @bartekpacia, thank's it sounds interesting... I'll create a PoC with Patrol and tell you the result.


[typescript] Enable generic props for certain components

This revives the effort started in #11731, which ran into difficulty with callback parameter types no longer being inferred, thus breaking backwards compatibility. This new approach circumvents the issue by using overloads; in the simple case where no override component is provided, callback parameter types are fully inferred with no extra work. In the case where an override component is provided, you can either

provide the component type also as a generic parameter:<Button<'div'>
  component="div"
  onClick={e => { /* e is inferred as MouseEvent<HTMLDivElement> */ }}
/>

provide an explicit type for the callback parameter which agrees with the component prop:<Button
  component="div"
  onClick={(e: MouseEvent<HTMLDivElement>) => {}}
/>

So far in this PR only a handful of components have been converted to use this new typing strategy:

Avatar
Button
Fab
ListItem
Tabs

The results seem promising, but I wanted to solicit feedback before diving in too much deeper. One unfortunate issue is that error messages can be somewhat cryptic; for instance with
const Override = (props: {}) => <div />;

<Avatar component={Override} onClick={e => {}} />;

The error we get is that Property 'component' does not exist on type .... There is an error here, which is that we are providing a prop, onClick, which does not exist on the overriding component, however the message doesn't help us understand that.
Upgraded TypeScript to 3.2 because it contains a bugfix for https://github.com/Microsoft/TypeScript/issues/26004 which is necessary to make this work.

I would've thought github pull requests for vscode would bundle those review comments. Sorry for the noise.
We still need to target next with this change. It still requires explicit callback annotation unless one allows implicit any.

We still need to target next with this change. It still requires explicit callback annotation unless one allows implicit any.

Do you mean targeting the next branch of MUI, or typescript@next?

We still need to target next with this change. It still requires explicit callback annotation unless one allows implicit any.

Do you mean targeting the next branch of MUI, or typescript@next?

Our next branch. As far as I know passing component does not cause implicit any in callback with 3.x so It's probably better to target 4.x.

We still need to target next with this change. It still requires explicit callback annotation unless one allows implicit any.

Do you mean targeting the next branch of MUI, or typescript@next?

Our next branch. As far as I know passing component does not cause implicit any in callback with 3.x so It's probably better to target 4.x.

I see. Yeah, it's technically a breaking change although in the past we've treated "type-only" backwards incompatibilities as not requiring a major rev. If I target next, it brings in a bunch of other changes from master that aren't on next yet, should I rebase then?

netlify was not configured to build previews for next PRs. Should be working on the next git push.

@pelotom I did commit some changes from my own review that I thought weren't that controversial. This includes some code comments for the types.
Could you go over some unaddressed review comments? I think we can start converting the other declarations and get some feedback on a alpha release.

@eps1lon very sorry I've been AWOL on this, work has been crazy. I should have time to look this over tonight.

@eps1lon very sorry I've been AWOL on this, work has been crazy. I should have time to look this over tonight.

No worries. I think I only fixed some nitpicks I had. Looks like we can start applying this to the rest of the components which hopefully is pretty straight forward and doesn't break any tests.

Looks like we can start applying this to the rest of the components which hopefully is pretty straight forward and doesn't break any tests.

I assume you mean merge this and then start to gradually migrate other components to the new system, yeah?


Add Grammar's rexGlorp to the pool of dino bodies.

No testing done because I'm doing it through the web interface.

Thanks for glorping my glorp Zuzu


Fix nil pointer deref in metrics exporter (#1187)

This is a very targeted fix for the NPE shown in #1187 since it's a known problem. The general pattern of guarding status lookups may be worth thinking about. Maybe we just need an intermediate type that decouples the application level usage of this state from its serialization/deserialization, at which point we can use basic language features like field visibility to reduce the probability of future unsafe accesses.
Important items to consider before making a Pull Request
Please check items PR complies to:

[x] All commits in the PR are squashed. More info
[x] The PR is made into dedicated next-release branch, not into master branch1. More info
[x] The PR is signed. More info

Meta: it would seem to be going against the trend to add some highly specific unit test for this, so I'm going to let integration tests "cover it" ... although they didn't catch its first introduction (possibly due to the self-healing / temporal element of the "status" state).

Whoops:
133.0 pkg/apis/metrics/exporter.go:295:8: invalid operation: operator ! not defined on chi.EnsureStatus().GetNormalizedCHICompleted() (value of type *"github.com/***/clickhouse-operator/pkg/apis/clickhouse.***.com/v1".ClickHouseInstallation)

I'll fire this up on my actual development machine and fix that now


Error invoking action

I'm getting an error running the script
~/Library/Application Support/Keyboard Maestro/Keyboard Maestro Actions/Pushover Notifications/PushoverNotification.scpt: execution error: Can’t get text 1 thru 14 of "Test title". (-1728)
When invoking the following action
Send a notification via Pushover
Priority: Normal
Title: Notification Title
Notification Message:
HTML formatted notification message. Go here Here
URL Title:
Test title
URL:
http://google.com
Notification Sound: User Default
Display trimmed results in a window.

Well it's many years late, but I did update this in my code. I hope you found one of the clones where people had fixed this.
my latest update adds support for the new custom sounds and file attachments.


Add MAC address hash for non-Windows platforms

This PR enables obtaining a MAC address hash for telemetry purposes on non-Windows platforms.

Reminder for myself to investigate ifaddrs.h for macos

I've added a fallback to ioctl() if getifaddrs() is not available, this should work on other systems besides macOS and Linux (BSD for example).
I don't like relying on build system detection, but it is the best we can do right now. If people don't like it I'm willing to revert the last commit.

Given the issues Robert found here, please double check on an actual Linux box that what you get matches a VS Code on the same machine.


Probably don't need an intermediary type for serialization

The NNTSerde struct makes it easier to write because of the derive macro, but it is also a lot slower than just converting directly from the neural network topology.

If #60 does end up rewriting everything without Arc spam, this will become significantly easier.


Automatic branch update and forced pushes can potentially flood CI

As discussed here: https://github.com/playframework/playframework/pull/9159#issuecomment-479859679
We are facing an issue in which Travis is being flooded with new builds over and over. That happens because we have had a few PRs opened by @scala-steward and on each new merge on master, @scala-steward updates the branch and sends a new push.
I personally don't think that this should be the default behaviour because it can potentially hijack CI for long periods.
Maintainers have a few options to enforce this, for instance, they can configure branch protection which will force them to update the branch via GitHub or they can use something like Mergify that does it automatically and sequenced without triggering many builds at the same time.
So, ideally, either @scala-steward simply leave the responsibility to the project maintainers or over a flag in .scala-steward.conf to disable it.

Btw, this is mainly done to resolve merge conflicts which easily happen with scala-steward's PR since version numbers tend to be defined relatively adjacent in build files.
If scala-steward could figure out if a PR has conflicts with the base branch we could update PRs only if there are conflicts. That sounds like a good compromise between keeping PRs automatically mergeable and not uneccessarily stressing CI systems.

https://developer.github.com/v3/git/#checking-mergeability-of-pull-requests
But I'm not sure if "mergeability" means pure git (i.e. has merge conflicts or not) or PR mergeability (requires review, etc).

Thanks for the hint. Since this is just another GitHub call it would be straightforward to implement.
Another option might be just using Git as shown in https://stackoverflow.com/a/501461/460387. The benefit of just using Git is that it would also work if scala-steward is ported to Bitbucket or GitLab


Cast type when melting wide data

Coverage decreased (-0.04%) to 74.975% when pulling 1f19df5fdf0189dc3052d9c6e04aa70108f6d77f on melt_typing into cabb29c7b180defbe090f0421ec837016ff63a4d on master.


Add alembic to requirements.txt

Add alembic>=0.8.0 to requirements.txt

for the master trunk - it looks like the a10-neutron-lbaas-db-manage requires a specific version of alembic looks like > 0.8.0 and as such seems like its targeted towards liberty and newer?
>=**
would be nice if that requirement was done in requirements.txt or test-requirements.txt
As it needs: alembic.runtime.environment import EnvironmentContext which was added in 0.8.0 - are you guys expecting most of these deps to be satisfied by neutron?

Even if we were to rely on neutron satisfying dependencies, the use of EnvironmentContext in cli.py can't rely on Neutron having installed a specific version of alembic.

no, i don't think we want this, or we have to have a branch for every openstack release. Does the stable/kilo and master split not cover this?

Reworked to use older alembic.


Where can I find the color palette info for the ground truth images?

Could you please provide the color code for the groundtruth images, so that we can find which color corresponds to which class? Thanks!

You can find all labels, colors and other meta information here
https://github.com/mcordts/cityscapesScripts/blob/master/scripts/helpers/labels.py


Adds active_subsidized_units & percent_units_subsidized to zone_facts.

This revision gets the job done. Eventually, when we call _get_residential_units from within _populate_zone_facts_table, then the one-time query used here to filter and sum the column values proj_units_assist_max can ultimately be generalized and made into its own separate method. Then we also can trim back the argument res_units_by_zone presently passed to subsequent methods. For now, however, this one-time wrangling produces the results we want.
That said, some data integrity problems also are brought to light. For example, in neighborhood cluster 38, 104% of the residential units are counted as actively "subsidized." Something clearly isn't being counted right, and we need to find out why not. More dramatically, in census tract<PHONE_NUMBER>4, 106 out of, yes, 13, units are counted as subsidized. So, some sleuthing is needed to see if residential units (from the mar table) are being under-counted or subsidized units (from the project table) are being over-counted..

I'm putting review of this on hold until after our Friday demo to DHCD, since there's some data integrity problems and so I can focus on making sure everything is working for the demo. Will pick it up after likely on Monday.

@pfjel7 I think the biggest source of error might be double counting on the projects when we can't appropriately de-duplicate the records from our various sources. Here's a good example of several projects that are likely double counting:

To find those records you can run this query:
select * from project where proj_name ilike '%elvan%' or proj_addre ilike '%elvan%'
Two of them come from the preservation catalog ("People's Cooperative" and "Elvans Road"), so those are likely to be two separate projects. However, there are two additional records that appear to match each of those, one from the DHCD database and one from the DCHousing dataset. Only one of those has proj_units_assist_max. Just one example, though. Our current de-dupe method relies on being able to parse the addresses, but here it looks like slightly different addresses are throwing things off

?That makes sense. I will follow up on your suggestions for how we might check the validity of the data. I will take up that task next week. Will be in touch. Per

From: Neal Humphrey<EMAIL_ADDRESS>Sent: Wednesday, September 20, 2017 10:21 PM
To: codefordc/housing-insights
Cc: Fjelstad, Per; Author
Subject: Re: [codefordc/housing-insights] Adds active_subsidized_units & percent_units_subsidized to zone_facts. (#574)
I'm putting review of this on hold until after our Friday demo to DHCD, since there's some data integrity problems and so I can focus on making sure everything is working for the demo. Will pick it up after likely on Monday.

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_codefordc_housing-2Dinsights_pull_574-23issuecomment-2D331032371&d=DwMFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=tVTgE3T5prTv3rUavIYrDmLWdIgRBDlY0lM4qXkELL8&m=-tPdmL0I96qJE6KwChFbIoidOLU0EoUMOrKR7eoqjVA&s=IPFo55zhMhrplJx_6MAnzJ2eYNqdAVQxC8nffUbo96c&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ASuoBH984-2DxE1S0bSPf0wgLXGKrccfm0ks5skcgrgaJpZM4PegPL&d=DwMFaQ&c=c6MrceVCY5m5A_KAUkrdoA&r=tVTgE3T5prTv3rUavIYrDmLWdIgRBDlY0lM4qXkELL8&m=-tPdmL0I96qJE6KwChFbIoidOLU0EoUMOrKR7eoqjVA&s=ACS1Q7PJXxCqxH-j6KE85WoGNW_xHluFQwTdQuvALho&e=.

@pfjel7 FYI, I made another issue that will help us troubleshoot the numerator portion of this - #577 . In addition, @jkwening added a logger to our deduplication code that gives us additional data about how the deduplication of project records was performed so that we can troubleshoot the ones that are slipping through. Finally, I'm hoping @ptatian can take a look at our data with both of these tools to help us identify duplicates since he has the most familiarity with the preservation catalog data set - he may know off hand whether some of the ones that are supposedly unique to DHCD are actually already in the preservation catalog. We can use this to either improve our dedupe logic and/or create a manual override column during dedupe that uses a list of known pairs and/or send corrections to DHCD with respect to the addresss that they have stored in the database (our main way of identifying duplicated projects).


GEOMESA-2485 Fix SimpleFeatureSimpleFeatureConverter field transforms

Signed-off-by: Emilio Lahr-Vivaz<EMAIL_ADDRESS>

@elahrvivaz thanks for verifying that!.

Ok.... I get 20 files when I run that command.  Do the ones where individual tests are annotated work?

Did you run it on this branch? There should be fewer now - all the other tests were running as expected, but I figured it would be good to standardize on specs2. There was one test that I didn't update from straight junit, as it was fairly complex. And there are several in hbase and lambda that get invoked differently to avoid spinning up the embedded clusters multiple times.


Remove console.log for logger library

Update the to a logger so that we can do away with console logging.
Lets us temporarily look at debug logs at various levels.

Squashing this in new PR with better fixes


add GatherND and ScatterND to onnx ops

need to still ~verify correctness~ (wait still wrong) and check if there's a better implementation.

There's probably a simpler implementation but the implementation currently is correct.
These two ops are another case of onnx test suite not testing enough cases 🫠
Will add the tests in once I figure out how to superset onnx tests.

i think it's fine to add test cases that's lacking, maybe the ones you had bug that was not caught, by calling the onnx function like other Tensor functions to start with.


Debugging an extension does not produce console output in dev host window

VSCode Version: 1.0.1-alpha
OS Version: Windows 10

Steps to Reproduce:

Create an extension
Under the activate function in extension.js add:

 console.log('Congratulations, your extension "MyTestExtension" is now active!');

Launch extension debugging.
Press F1 in the host to launch the extension.

Actual:
The console output is being generated in the original Code window, but no output is showing in the dev host window.

I believe this is designed behavior and also how we behave in 1.0.0.
The original code window is used for debugging, so IMHO all debugging information should be displayed in the original window, like the console.log. Otherwise the experience would be disconnected for the user - to inspect variables he woudl go to original window but to inspect log to the dev host window.
@bpasero opinions welcome
Closing this for now as designed.


Quick Entry

Allow for some quick add functionality, where items can be entered quickly
If adding items, keep the same pack!

Probably an animation should be done, instead of just clearing the Item Name field


Syntax Highlighting, Example Hello World policy

Closes #65

Adds syntax highlighting to wherever code is shown, including the text field that you type your rego policy code into.
Adds an example policy that gets auto-populated to the create policy form.

some things to note with the addition of syntax highlighting...
The highlighter sometimes takes a moment to show the colors (viewing the occurrence JSON ,for example). I have added this to an issue to address later since it's not really breaking any functionality, it's just a weird experience.
With the change in to highlight code, the policy playground now shows the entire rego policy code, which is not a good experience if you have a lengthy policy. I plan on addressing this issue in my next PR.
The prism.js file included this PR is the code to do syntax highlighting. There is an npm package for Prism, but the latest release does not include the ability to parse rego. Once a new release of Prism and the babel plugin are cut, we can switch over to pulling down those dependencies instead of including them directly. The file is small and the parsing shouldn't change so not a huge deal, in my opinion. I have ignored the Prism file for linting and coverage.

With the light theme, the text contrast with a background is a little difficult to read IMO

Otherwise, LGTM, just had a few questions

I updated the colors for both themes a bit so now there should be a distinction between comments and regular code. Let me know if you still think the contrast is a bit too low.


Review blocks including {% endfor %}

A few students in this year's (2022) lesson had a consistent issue where there was an extra {% endfor %} tag in their index.html files. Since it's consistent, this is probably our mistake, rather than the students. Take a look through the docs and see where this may be confusing

Fixed by #117


[ACR] Enable to run black in ci

API change check
API changes are not detected in this pull request.

Hi @msyyc , can you help to run black on mgmt package? Or do you know who can help if you are not the best contact?

Hi @msyyc , can you help to run black on mgmt package? Or do you know who can help if you are not the best contact?

Mgmt python sdk is generated automatically and have been blacked. Maybe the difference is caused by the different black configuration. Why do you want to open format check in CI?

Hi @msyyc , can you help to run black on mgmt package? Or do you know who can help if you are not the best contact?

Mgmt python sdk is generated automatically and have been blacked. Maybe the difference is caused by the different black configuration. Why do you want to open format check in CI?

We run black with -l 120 option to allow lines up to 120 characters.


Installation error on Windows<EMAIL_ADDRESS>install

Description:
Installation fails on windows, either via Atom GUI or cmd (apm). I noticed that the same error happens when running:
npm install zeromq

Steps to Reproduce:
apm install hydrogen --python [path-to-python2]

Versions:
OS Version: Windows 7 64bit
Atom Version: 1.38.2
Electron Version: 2.0.18
Node Version: 8.9.3
Hydrogen Version: 2.9.0 (latest)
Logs:
9678 verbose stack Error<EMAIL_ADDRESS>install: `node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)`
9678 verbose stack Exit status 1
9678 verbose stack     at EventEmitter.<anonymous> (C:\Users\[user]\AppData\Local\atom\app-1.36.0\resources\app\apm\node_modules\npm\node_modules\npm-lifecycle\index.js:304:16)
9678 verbose stack     at emitTwo (events.js:126:13)
9678 verbose stack     at EventEmitter.emit (events.js:214:7)
9678 verbose stack     at ChildProcess.<anonymous> (C:\Users\[user]\AppData\Local\atom\app-1.36.0\resources\app\apm\node_modules\npm\node_modules\npm-lifecycle\lib\spawn.js:55:14)
9678 verbose stack     at emitTwo (events.js:126:13)
9678 verbose stack     at ChildProcess.emit (events.js:214:7)
9678 verbose stack     at maybeClose (internal/child_process.js:925:16)
9678 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
9679 verbose pkgid<EMAIL_ADDRESS>9680 verbose cwd C:\Users\[user]\AppData\Local\Temp\apm-install-dir-119315-20252-tmh3rj.1qhh
9681 verbose Windows_NT 6.1.7601
9682 verbose argv "C:\\Users\\[user]\\AppData\\Local\\atom\\app-1.36.0\\resources\\app\\apm\\bin\\node.exe" "C:\\Users\\[user]\\AppData\\Local\\atom\\app-1.36.0\\resources\\app\\apm\\node_modules\\npm\\bin\\npm-cli.js" "--globalconfig" "C:\\Users\\[user]\\.atom\\.apm\\.apmrc" "--userconfig" "C:\\Users\\[user]\\.atom\\.apmrc" "install" "https://www.atom.io/api/packages/hydrogen/versions/2.9.0/tarball" "--runtime=electron" "--target=2.0.18" "--dist-url=https://atom.io/download/electron" "--arch=x64" "--global-style" "--msvs_version=2015"
9683 verbose node v8.9.3
9684 verbose npm  v6.2.0
9685 error code ELIFECYCLE
9686 error errno 1
9687 error<EMAIL_ADDRESS>install: `node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)`
9687 error Exit status 1
9688 error Failed at the<EMAIL_ADDRESS>install script.
9688 error This is probably not a problem with npm. There is likely additional logging output above.
9689 verbose exit [ 1, true ]

Thank you!

Could you give me what apm --version says on your machine ?

apm  2.1.7
npm  6.2.0
node 8.9.3 x64
atom 1.38.2
python
git
visual studio 2015

Seems fine. Can you try to reinstall with python 3 ?

Python 2 has so known issues with hydrogen you can search all our issues to see info about it. Also just note that 2.10.2 is the latest version of hydrogen.

I use Python 3 but I am using Python 2 for the installation as I otherwise get the same issues as in #1431. Regarding the version, I get the same results with 2.10.2.
I cannot seem to specify a specific version in apm, sadly, as this feature is still WIP (PR: https://github.com/atom/apm/pull/773), but I remember having managed to install Hydrogen 2.6.0 (using Python 2) on the same system. This error happens either when updating, or on a fresh install.

Just a quick question. When you launch atom are you able to run any hydrogen commands and get the red bug as described in our troubleshooting guide?

On the working version of Atom + Hydrogen 2.6.0, Hydrogen works just fine, but if I run the update via GUI, I get this:

With error message:
><EMAIL_ADDRESS>install C:\Users\gpepe\AppData\Local\Temp\apm-install-dir-11961-17348-jxs4ir.j0zq\node_modules\Hydrogen\node_modules\zeromq
> node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)

prebuild-install WARN install self signed certificate in certificate chain

Downloading libzmq for Windows
Download finished

C:\Users\[user]\AppData\Local\Temp\apm-install-dir-11961-17348-jxs4ir.j0zq\node_modules\Hydrogen\node_modules\zeromq>if not defined npm_config_node_gyp (node "C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\bin\\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )

gyp info it worked if it ends with ok
gyp verb cli [ 'C:\\Users\\[user]\\AppData\\Local\\atom\\app-1.38.2\\resources\\app\\apm\\bin\\node.exe',
gyp verb cli   'C:\\Users\\[user]\\AppData\\Local\\atom\\app-1.38.2\\resources\\app\\apm\\node_modules\\node-gyp\\bin\\node-gyp.js',
gyp verb cli   'rebuild' ]
gyp info using<EMAIL_ADDRESS>gyp info using<EMAIL_ADDRESS>| win32 | x64
gyp verb command rebuild []
gyp verb command clean []
gyp verb clean removing "build" directory
gyp verb command configure []
gyp verb check python checking for Python executable "C:\Python27\python.exe" in the PATH
gyp verb `which` failed Error: not found: C:\Python27\python.exe
gyp verb `which` failed     at getNotFoundError (C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\which\which.js:13:12)
gyp verb `which` failed     at F (C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\which\which.js:68:19)
gyp verb `which` failed     at E (C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\which\which.js:80:29)
gyp verb `which` failed     at C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\which\which.js:89:16
gyp verb `which` failed     at C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\isexe\index.js:42:5
gyp verb `which` failed     at C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\isexe\windows.js:36:5
gyp verb `which` failed     at FSReqWrap.oncomplete (fs.js:152:21)
gyp verb `which` failed  C:\Python27\python.exe { Error: not found: C:\Python27\python.exe
gyp verb `which` failed     at getNotFoundError (C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\which\which.js:13:12)
gyp verb `which` failed     at F (C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\which\which.js:68:19)
gyp verb `which` failed     at E (C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\which\which.js:80:29)
gyp verb `which` failed     at C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\which\which.js:89:16
gyp verb `which` failed     at C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\isexe\index.js:42:5
gyp verb `which` failed     at C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\isexe\windows.js:36:5
gyp verb `which` failed     at FSReqWrap.oncomplete (fs.js:152:21) code: 'ENOENT' }
gyp verb could not find "C:\Python27\python.exe". checking python launcher
gyp verb could not find "C:\Python27\python.exe". guessing location
gyp verb ensuring that file exists: C:\Python27\python.exe
gyp ERR! configure error
gyp ERR! stack Error: Can't find Python executable "C:\Python27\python.exe", you can set the PYTHON env variable.
gyp ERR! stack     at failNoPython (C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\node-gyp\lib\configure.js:449:14)
gyp ERR! stack     at C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\node-gyp\lib\configure.js:404:11
gyp ERR! stack     at C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\app\apm\node_modules\graceful-fs\polyfills.js:282:31
gyp ERR! stack     at FSReqWrap.oncomplete (fs.js:152:21)
gyp ERR! System Windows_NT 6.1.7601
gyp ERR! command "C:\\Users\\[user]\\AppData\\Local\\atom\\app-1.38.2\\resources\\app\\apm\\bin\\node.exe" "C:\\Users\\[user]\\AppData\\Local\\atom\\app-1.38.2\\resources\\app\\apm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\[user]\AppData\Local\Temp\apm-install-dir-11961-17348-jxs4ir.j0zq\node_modules\Hydrogen\node_modules\zeromq
gyp ERR! node -v v8.9.3
gyp ERR! node-gyp -v v3.4.0
gyp ERR! not ok
npm WARN enoent ENOENT: no such file or directory, open 'C:\Users\[user]\AppData\Local\Temp\apm-install-dir-11961-17348-jxs4ir.j0zq\package.json'
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| ^15.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of vega@* but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| ^4.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN apm-install-dir-11961-17348-jxs4ir.j0zq No description
npm WARN apm-install-dir-11961-17348-jxs4ir.j0zq No repository field.
npm WARN apm-install-dir-11961-17348-jxs4ir.j0zq No README data
npm WARN apm-install-dir-11961-17348-jxs4ir.j0zq No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>install: `node scripts/prebuild-install.js || (node scripts/preinstall.js && node-gyp rebuild)`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

I recommend reading nodejs/node-gyp#1337 and nodejs/node-gyp#1268. They both seem to have work arounds for now. Seems like gyp needs to update from Python 2.7 but seems like they don't want/care to.

Most workarounds in those issues involve the use of Python 2. I seem to successfully do that in my case (original log in the first post), and I still get that error.
I have found a similar issue: https://github.com/nteract/hydrogen/issues/1352, but the solution does not work for me (I do not have that key in the registry).
I think the problem runs deeper than this, as the log looks almost identical to others I have found around other node packages:

https://github.com/zeromq/zeromq.js/issues/177
https://github.com/zeromq/zeromq.js/issues/245
https://github.com/wavebox/waveboxapp/issues/454
https://github.com/nodegit/nodegit/issues/1471

It looks like it is some issue due to the compiler/visual studio. Since that is something I cannot change from my side, I might give up with updating Hydrogen. Thank you for your help, anyway.

If you don't have that key you should set it. That why the error is showing up because it can't find the key

In any case you could always download the files from the repo and install. There are plenty of ways to upgrade

gyp verb check python checking for Python executable "C:\Python27\python.exe" in the PATH
≥ Error: Can't find Python executable "C:\Python27\python.exe", you can set the PYTHON env variable.

@pepicello The log in this comment reports node-gyp fails because C:\Python27\python.exe doens't exist. Is that so?
Would it be an option for you to install Python 2 in C:\Python27\python.exe?
You my be affected by https://github.com/atom/apm/issues/818 .

Sorry for the confusion, the two issues mixed up! To clarify: I cannot choose the python version when trying to install via GUI. Since I use Python 3, the error from the GUI is related to not finding Python 2. But I am not attached to the GUI installation, so I tried to install from CLI, using apm, setting the python executable in apm, which I described in the first comment: https://github.com/nteract/hydrogen/issues/1689#issue-462645172. Here is a longer log of the same issue. For some reason, the console output showed more error than the log, so here is the console output:

gyp verb check python version `C:\Users\[user]\AppData\Local\Continuum\anaconda3\envs\py2\python.EXE
-c "import platform; print(platform.python_version());"` returned: "2.7.16\r\n"
gyp verb get node dir compiling against --target node version: 2.0.18
gyp verb command install [ '2.0.18' ]
gyp verb install input version string "2.0.18"
gyp verb install installing version: iojs-2.0.18
gyp verb install --ensure was passed, so won't reinstall if already installed
gyp verb install version is already installed, need to check "installVersion"
gyp verb got "installVersion" 9
gyp verb needs "installVersion" 9
gyp verb install version is good
gyp verb get node dir target node version installed: iojs-2.0.18
gyp verb build dir attempting to create "build" dir: C:\Users\[user]\AppData\Local\Temp\apm-install-d
ir-11963-6488-1jhv6l9.v57x\node_modules\Hydrogen\node_modules\zeromq\build
gyp verb build dir "build" dir needed to be created?  C:\Users\[user]\.atom\.node-gyp\.node-gyp\iojs-2.0.18\Re
lease
gyp verb copying "iojs.lib" for x64 C:\Users\[user]\.atom\.node-gyp\.node-gyp\iojs-2.0.18\Release\ioj
s.lib
gyp info spawn C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe
gyp info spawn args [ 'build/binding.sln',
gyp info spawn args   '/clp:Verbosity=minimal',
gyp info spawn args   '/nologo',
gyp info spawn args   '/p:Configuration=Release;Platform=x64' ]
gyp ERR! build error
gyp ERR! stack Error: `C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe` failed with exit c
ode: 1
gyp ERR! stack     at ChildProcess.onExit (C:\Users\[user]\AppData\Local\atom\app-1.38.2\resources\ap
p\apm\node_modules\node-gyp\lib\build.js:276:23)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Windows_NT 6.1.7601
gyp ERR! command "C:\\Users\\[user]\\AppData\\Local\\atom\\app-1.38.2\\resources\\app\\apm\\bin\\node
.exe" "C:\\Users\\[user]\\AppData\\Local\\atom\\app-1.38.2\\resources\\app\\apm\\node_modules\\node-g
yp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\[user]\AppData\Local\Temp\apm-install-dir-11963-6488-1jhv6l9.v57x\node_modules\
Hydrogen\node_modules\zeromq
gyp ERR! node -v v8.9.3
gyp ERR! node-gyp -v v3.4.0
gyp ERR! not ok
npm WARN enoent ENOENT: no such file or directory, open 'C:\Users\[user]\AppData\Local\Temp\apm-insta
ll-dir-11963-6488-1jhv6l9.v57x\package.json'
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| ^15.0.0 but none is installed.
You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>|| ^4.0.0 but none is installed. You must in
stall peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of vega@* but none is installed. You must install peer de
pendencies yourself.
npm WARN apm-install-dir-11963-6488-1jhv6l9.v57x No description
npm WARN apm-install-dir-11963-6488-1jhv6l9.v57x No repository field.
npm WARN apm-install-dir-11963-6488-1jhv6l9.v57x No README data
npm WARN apm-install-dir-11963-6488-1jhv6l9.v57x No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>install: `node scripts/prebuild-install.js || (node scripts/preinstall.js && n
ode-gyp rebuild)`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

As you can see, Python 2 is picked up, but the installation still fails.
Among the extra interesting lines in this log, this one seems to indicates a compiler error:
gyp ERR! stack Error: `C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe` failed with exit c
ode: 1

OK, it looks like<EMAIL_ADDRESS>is now choking on your installation of VS build tools.
The usual recommendation is to install npm install --global --production windows-build-tools (though, this hasn't always worked for me).

OK, it looks like<EMAIL_ADDRESS>is now choking on your installation of VS build tools.
The usual recommendation is to install npm install --global --production windows-build-tools (though, this hasn't always worked for me).

This did not work, but I have eventually fixed it by re-installing VS and VS C++ Build Tools. Thanks anyway for the help!


Flutter Roboto doesnt look like current Google Roboto Font

Hello,
Today I ran into a problem with the Roboto Font that is set as a default in the Flutter framework.
Somehow the Roboto Font on the google fonts website (https://fonts.google.com/specimen/Roboto) looks different than the one that Flutter provides.
To solve my problem I imported the font found on the google fonts website through the pubspec.yaml file to overwrite the version of the font that Flutter provides as a default.
It would be great if that issue is fixed!
Thank you :))

Can you please share screenshots of the specific problem, and a code to reproduce, so we can actually see the difference in the same text?
(To identify, if it's a font-weight problem or different font problem etc.)

import 'package:google_fonts/google_fonts.dart';
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) => MaterialApp(
    home: MyHomePage(),
  );
}

class MyHomePage extends StatelessWidget {

  @override
  Widget build(BuildContext context) => Scaffold(
    body: Column(
      mainAxisAlignment: MainAxisAlignment.center,
      crossAxisAlignment: CrossAxisAlignment.center,
      children: <Widget>[
       const Text('"hello world" w/ default fonts',
        textScaleFactor: 0.95,
       ),
       Text(
         '"hello world" w/ google fonts',
         style: GoogleFonts.roboto(),
       ),
      ],
    ),
  );
}

screenshot
Hi @khalithartmann
I can observe the issue you are describing using google_fonts.
Thank you

Yes digging a little bit, after I use the same links provided in the package, I am able to get the correct fonts. It must be related to set up of the fonts.

Same issue.
We have had to move to using https://pub.dev/packages/google_fonts to work around the problem.

flutter doctor -v
[✓] Flutter (Channel dev, 1.20.0-3.0.pre, on Mac OS X 10.15.5 19F101, locale en-GB)
    • Flutter version 1.20.0-3.0.pre at /Users/taha/Code/flutter_dev
    • Framework revision 0af027f805 (4 days ago), 2020-07-04 12:19:20 -0700
    • Engine revision a751393804
    • Dart version 2.9.0 (build 2.9.0-20.0.dev 22da8934ac)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
    • Android SDK at /Users/taha/Code/sdk
    • Platform android-30, build-tools 30.0.0
    • ANDROID_HOME = /Users/taha/Code/sdk
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.5)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.5, Build version 11E608c
    • CocoaPods version 1.9.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 47.1.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] VS Code (version 1.46.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.12.1

[✓] Connected device (3 available)
    • Taha’s iPhone (mobile) • 00008020-001059882212002E • ios            • iOS 13.5.1
    • Web Server (web)       • web-server                • web-javascript • Flutter Tools
    • Chrome (web)           • chrome                    • web-javascript • Google Chrome
      83.0.4103.116

• No issues found!

cc @darrenaustin. This is a little tricky. Roboto is included in a fonts.zip. I think updating it straight out would be a major visual breaking change, which we may want to do? Alternatively, we'd need to find a mechanism to support multiple versions of Roboto.

This is still a problem in latest flutter

Hi @khalithartmann, I verified the issue on stable 3.3 and master 3.7 and it is no longer reproducible, I believe this issue is fixed.

code sample
import 'package:flutter/material.dart';
import 'package:google_fonts/google_fonts.dart';

void main() => runApp(const MyApp());

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) => const MaterialApp(
        home: MyHomePage(),
      );
}

class MyHomePage extends StatelessWidget {
  const MyHomePage({super.key});

  @override
  Widget build(BuildContext context) => Scaffold(
        body: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          crossAxisAlignment: CrossAxisAlignment.center,
          children: <Widget>[
            const Text(
              '"hello world" w/ default fonts',
            ),
            Text(
              '"hello world" w/ google fonts',
              style: GoogleFonts.roboto(),
            ),
          ],
        ),
      );
}

flutter doctor -v (mac)
[✓] Flutter (Channel master, 3.7.0-3.0.pre.33, on macOS 12.6 21G115 darwin-arm64, locale en-IN)
    • Flutter version 3.7.0-3.0.pre.33 on channel master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision<PHONE_NUMBER> (38 minutes ago), 2022-12-05 18:27:21 -0800
    • Engine revision a309d239c4
    • Dart version 2.19.0 (build 2.19.0-463.0.dev)
    • DevTools version 2.20.0
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly
      to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0-rc4)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0-rc4
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14A400
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 61.2.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.70.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.55.20221129

[✓] Connected device (3 available)
    • iPhone 12 Pro (mobile) • 026D5789-9E78-4AD5-B1B2-3F8D4E7F65E4 • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • macOS (desktop)        • macos                                • darwin-arm64   • macOS 12.6 21G115 darwin-arm64
    • Chrome (web)           • chrome                               • web-javascript • Google Chrome 108.0.5359.94

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

[✓] Flutter (Channel stable, 3.3.9, on macOS 12.6 21G115 darwin-arm, locale en-IN)
    • Flutter version 3.3.9 on channel stable at /Users/mahesh/Development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision b8f7f1f986 (24 hours ago), 2022-11-23 06:43:51 +0900
    • Engine revision 8f2221fbef
    • Dart version 2.18.5
    • DevTools version 2.15.0

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0-rc4)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0-rc4
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14A400
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 61.2.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.70.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.53.20221101

[✓] Connected device (3 available)
    • iPhone 12 Pro (mobile) • 026D5789-9E78-4AD5-B1B2-3F8D4E7F65E4 • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • macOS (desktop)        • macos                                • darwin-arm64   • macOS 12.6 21G115 darwin-arm
    • Chrome (web)           • chrome                               • web-javascript • Google Chrome 107.0.5304.110

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!

Closing as fixed, Incase anyone disagrees feel free to write in the comments and we will reopen it.
Thank you.


License of example figma file

Hi,
Thank you for this plugin. I find it useful.
I would like to know if I have permission to redistribute the API dump of the example Figma file in my repo?
https://www.figma.com/file/2MQ759R5kJtzQn4qSHuqR7/Design-Tokens-for-Figma
This repo is MIT licensed, but I don't think that applies to the file hosted on Figma. My repo is also MIT licensed.
I would like to use the Figma API dump of the file as a test case in my project.

Hey, sure, feel free to do that.

Thanks :)