Add property that there is a point whose only neighborhood is the full space

Adds a property as discussed in #847. I think the main point of discussion left to be had is regarding the name of the property and listing of equivalent characterizations. I opted for "Has a point with a trivial neighborhood filter" as the name, but there were other ideas raised. Another idea that just came to mind would be something along the lines of "has a universal closure point" in that such points are also closure points of any nonempty set.

"Has a point more specialized than every other point" "Has a point that is a specialization of every other point" "Has a point with X as only neighborhood" "Has a point contained in all nonempty closed sets" "Has a point contained in all closed sets" (ignoring "nonempty", which only needs to be in the text) etc, etc.
Is there an article somewhere that makes use of that notion, so we can see what they did?

Variations on "Has a point more specialized than every other point" don't really seem to be in use. I found  only one document which uses something like it, and the usage seems to be ad hoc though.
I still like the idea of the name emphasizing the point, so it's more clear that the property is dual to 'Has a generic point'. (This might not be a valid justification for choosing the name though.) Is 'closure point' more common than 'adherent point'? I just saw 'adherent' in use in a paper recently.

"Trivial neighborhood filter" doesn't connect with me as quickly as "global/universal closure/adherent/limit point" does.
We don't always follow Counterexamples terminology, but "adherent point" appears there on page 5. So I'd lean "universal/global adherent point".

That seems sensible to me. The meaning of closure point is more obvious to me than adherent point since I've spent a lot of time thinking about the closure algebras of topological spaces, but I could not tell you which term is more common.
@prabau any thoughts?

"Has a point with a single neighborhood" or
"Has a point with a unique neighborhood"

I'm okay with these as well. Maybe add "closure" point or similar description as an alias.

"Has a point with a single neighborhood" or
"Has a point with a unique neighborhood"

I vote for unique.

As for "Has a most specialized point", apart from the fact that it's a little involved, I like that it's short.  But it is really clear?
A point that is a minimum in the specialization order, i.e., a point that is a specialization of every other point, is what we want.  It is "most specialized in that sense.
But consider a point $p$ that is only minimal in the specialization order, i.e, a point such that any point $x$ that is a specialization of $p$ is actually topologically indistinguishable from it.  Could one also think that it is "most specialized", in the sense that there is no strictly more specialized point?  But that's not what we want.

"A point with one neighborhood": short and unambiguous!

But consider a point p that is only minimal in the specialization order, i.e, a point such that any point x that is a specialization of p is actually topologically indistinguishable from it. Could one also think that it is "most specialized", in the sense that there is no strictly more specialized point? But that's not what we want.

I was wondering if in future commits we might be interested in adding other properties satisfied by the specialization preorder (having multiple or no minimal / maximal points etc...).

I vote for unique as well. As my linear algebra students point out when discussing independence and solution sets, saying you have "one" thing isn't always clear you mean "exactly one" or "at least one".

That's actually my inclination as well, but in this case I think once I see the word "one" the thought that comes to mind is "this would be both redundant and trivial if they just wanted to say 'has a neighborhood'".

But consider a point p that is only minimal in the specialization order, i.e, a point such that any point x that is a specialization of p is actually topologically indistinguishable from it. Could one also think that it is "most specialized", in the sense that there is no strictly more specialized point? But that's not what we want.

I was wondering if in future commits we might be interested in adding other properties satisfied by the specialization preorder (having multiple or no minimal / maximal points etc...). (Not to speculate out of turn here, but perhaps that has some influence on the decision.)

I'd say let's worry about this if we need to in the future.  Terminology can be tweaked if desired, since it's our own invention here.
Speaking of which, adding the current property seems a natural thing as it is exactly the dual of the "Has a generic point property" and we have a good set of associated theorems.  But at the same time, I think one of principles of pi-base was that we wanted to reflect properties of interest to topologist that have been used in the literature.  Once a while we deviated from that by adding utility properties that were very convenient in tightening up theorems and traits.  That does not mean we want to add any random property out there.  @StevenClontz Is the above still a guiding principle for pi-base?
Anyway, question: What does the current property add exactly in terms of new automated deductions of traits for spaces currently in pi-base?  (I have not reviewed the PR yet.)

Speaking of which, adding the current property seems a natural thing as it is exactly the dual of the "Has a generic point property" and we have a good set of associated theorems. But at the same time, I think one of principles of pi-base was that we wanted to reflect properties of interest to topologist that have been used in the literature. Once a while we deviated from that by adding utility properties that were very convenient in tightening up theorems and traits. That does not mean we want to add any random property out there. @StevenClontz Is the above still a guiding principle for pi-base?

That's my opinion still. "Has a point with a unique neighborhood" feels like a good utility that matches what a mathematician would do: even though it's not named, it's a very easily verified property that immediately implies multiple named things, so by including it in pi-Base we are able to assert fewer properties and reduce manual deduction in lieu of automated deduction.

Anyway, question: What does the current property add exactly in terms of new automated deductions of other traits for spaces currently in pi-base? (I have not reviewed the PR yet.)

Aside from values for the new property, which is derived for all but one space, there are three properties for which there are new traits: Strongly paracompact, Ultraparacompact, and Contractible. I just whipped this up to see exactly what traits are added by a PR if you'd like to see for yourself: https://observablehq.com/d/3b91415189f648d9

Very nice comparison tool!

Should the descriptions of P201 and P202 refer to each other, so the user sees that pi-base has the dual property?

Good idea.

The last characterizations of P201/P202 you added are so cool.
I'm really sorry for just noticing this, but did we decide not to add

$p$ is a limit point of every nonempty proper subset of $X$.

as one of the equivalent characterizations? Or alternatively,

$p$ is a closure point of every nonempty subset of $X$.

(Or adherent / contact point)? This conveys something like "every set of values/points/space elements approximates $p$". Knowing any information about my space tells me about $p$, or something...
I think this should be on the parallel line in the list of equivalences to P201's " $\{p\}$ is dense in $X$ ", since that conveys something like "all of the values/points/space elements are approximated by $\{p\}$ ". The substantial information about my space comes from knowing about $p$, or something...
(I'd love to hear a more refined moral description on what generic point / the dual property mean.)

"every set of values/points/space elements approximates p ".

(For P202)  Didn't you mean the opposite: "$p$ approximates every point/set/..."?

(consider moving this discussion out to a discussion or issue so it's not lost)

(I edited my comment but prabau edited it simulataneously and I don't want to lost my paste / override his edits.)
The last characterizations of P201/P202 you added are so cool.
I'm really sorry for just noticing this, but did we decide not to add

$p$ is a limit point of every nonempty proper subset of $X$.

as one of the equivalent characterizations? Or alternatively,

$p$ is a closure point of every nonempty subset of $X$.

(Or adherent / contact point)? This conveys something like "every set of values/points/space elements approximates $p$". Knowing any information about my space tells me about $p$, or something...
I think this should be on the parallel line in the list of equivalences to P201's " $\{p\}$ is dense in $X$ ", since that conveys something like "all of the values/points/space elements are approximated by $\{p\}$ ". The substantial information about my space can be known from knowing about $p$, or something...
Sorry this is not super well thought out. I find myself rather distracted. Is the following another equivalence to being a generic point $p \in X$?

There exists a space $\widehat{X}$ with a dense subset $P$, and a quotient map $\widehat{X} \to X$ sending points of $P$ to $p$, and which otherwise has singleton fibers.

Is a particular point topology a terminal object with respect to sets of a given cardinality having this property? This would be a kind of terminal object characterization, compared to the one danflapjax added, which is a kind of initial object characterization.
(I'd love to hear a more refined moral description on what generic point / the dual property mean.)

My edit was inconsequential.  Can you delete one of the near identical comments?  I'll delete this particular one afterwards.  This discussion is becoming unwieldy. :-)

"every set of values/points/space elements approximates p ".

(For P202) Didn't you mean the opposite: "$p$ approximates every point/set/..."?

No. I wanted to think of $p$ is a limit point of a set  $A$ to mean I can approximate the value of $p$ with points from $A$.

Maybe I am confused about the meaning of "approximates". When we have x ∈ A ― , i.e., every nbhd of x meets A , i.e., there are values of A as close as desired from x , that would mean x can be approximated by points of A , i.e., A approximates x . Is that right?

Yes.

Variations on characterizations danflapjax added

The following are additional characterizations P201, which could be nice to include. danflapjax's characterization guides the reader to the first, which perhaps guides the reader to the second.

There exists a space $P$ with a particular point topology, and a surjective continuous map $P \to X$.

For one direction, continuous images of dense subsets are dense, so the generic point of $P$ maps to a generic point. For the converse, the characterization danflapjax added gives one such $P$. Note that $P \to X$ is necessarily a closed map by one of the characterizations of generic point in P201.

There exists a space $Y$ and a closed map $Y \to X$ which maps a dense subset to a singleton.

^ I just wanted to think about what generic points are, and I guess one way they implicitly arise is when you're thinking about a dense subset of a space, and treating points of that dense subset as just the same as any other for some reason. I have to say it seems kind of weird to me, but I also think it's interesting.

For P202, the parallel characterizations are

There exists a space $E$ with an excluded point topology, and a surjective continuous map $E \to X$.

For one direction, continuous maps send closure points to closure points. For the converse, the characterization danflapjax added gives one such $E$. Note that $E \to X$ is necessarily an open map by the naming characterization of P202.

There exists a space $Y$ and an open map $Y \to X$ which maps a codense subset to a singleton. A subset $A \subset Y$ is codense if for each closed, nonempty subset $B \subset Y$, $A \cap B \neq \emptyset$.

This one doesn't really illuminate anything for me. I currently find the excluded point topologies unintuitive. I was disappointed that I couldn't find any usage of 'codense' in the literature though (hopefully I'm not kidding myself in thinking it is distinct from having a dense complement). So it may just be something nlab uses, and maybe this one isn't very helpful to include. Nlab's talk page shows some of the contributors want to find some examples of codense sets though.

What do we think about these? Hopefully I haven't made an error.

Hmm, I have not read all this (yet, but I will).  But we don't necessarily need to add all possible equivalent characterizations of a given property.  Compare with https://en.wikipedia.org/wiki/T1_space, which has a large number of equivalent conditions for T1 and for R0; yet we only have listed a few of those in pi-base.  I think it makes sense to add to pi-base those characterizations that are helpful in proving some of the associated theorems or deriving some of the traits for spaces.  But pi-base is not meant as a complete repository of all topological information about a property. My $0.02.
Maybe discussing on mathse would reach a wider audience and get interesting answers from people, and serve as a more accessible repository for some these ideas?

@prabau I would move this there. I just have a lot of anxiety with posting on mathse, to be honest. I'm not sure how to format a question like this.

Yeah, it's more of a discussion that a punctual question.  Sometimes phrasing in terms of a comparison between P201 and P202 properties (without necessarily referring to pi-base) and asking some fuzzy "wondering about" question gets some good answers from mathse folks.

Yeah, it's more of a discussion that a punctual question. Sometimes phrasing in terms of a comparison between P201 and P202 properties (without necessarily referring to pi-base) and asking some fuzzy "wondering about" question gets some good answers from mathse folks.

I think I'm going to give it a shot. I'm happy to move on from this either way (I still think P202 would benefit from having the parallel property to P201's ${p}$ is dense).

I still think P202 would benefit from having the parallel property to P201's { p } is dense

Yeah, that sounds reasonable.  Whatever that property turns out to be.

The last characterizations of P201/P202 you added are so cool.

Thank you :)

I still think P202 would benefit from having the parallel property to P201's ${p}$ is dense

Along with what you already discussed, here's another possible approach to a corresponding notion. I don't know if this has a name, but there is a similar notion to closure using open sets: just as closure is the intersection of all closed sets containing a given set, one can take the intersection of all open sets containing a given set. This is, in general, open itself only if the space is Alexandrov, and it becomes the identity on T1 spaces, but it's still a well-defined operator. P202 holds iff there is a singleton for which this operator returns the whole space.


Update the snowplow-mini UI to make it more functional

Proposed UI:

Looks awesome - scheduling!

Pushing back


how do i hide all object broder line,thanks~

All object in pdf document had an dotted line border, In example and wiki i can't how to hide it.

sorry,I find it at PDFGenerator debug variable~~~


DRAFT: YJDH-645 | KS-Backend: Stream Excel export

Description :sparkles:
KS-Backend: Add xlsx-streaming Python package
Refs YJDH-645 (The package to be used for streaming .xlsx files)
KS-Backend: Make Excel export really stream XLSX files
DRAFT! Do not merge this as is!
Refs YJDH-645
Issues :bug:
YJDH-645
Testing :alembic:
Screenshots :camera_flash:
Additional notes :spiral_notepad:

Codecov Report

Merging #1345 (0d06d2d) into develop (69e72dd) will increase coverage by 29.44%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##           develop    #1345       +/-   ##
============================================
+ Coverage    64.87%   94.32%   +29.44%
============================================
  Files         1024      143      -881
  Lines        28091     6450    -21641
  Branches      2950        0     -2950
============================================
- Hits         18225     6084    -12141
+ Misses        9424      366     -9058
+ Partials       442        0      -442

Impacted Files
Coverage Δ

...esaseteli/applications/exporters/excel_exporter.py
99.09% <100.00%> (+0.09%)
:arrow_up:

...kesaseteli/applications/tests/test_excel_export.py
98.66% <100.00%> (-0.04%)
:arrow_down:

backend/kesaseteli/applications/views.py
96.49% <100.00%> (+0.65%)
:arrow_up:

...nd/shared/shared/service_bus/service_bus_client.py
27.65% <0.00%> (-57.45%)
:arrow_down:

backend/shared/shared/yrtti/yrtti_client.py
39.39% <0.00%> (-51.52%)
:arrow_down:

backend/shared/shared/common/lang_test_utils.py
80.00% <0.00%> (-20.00%)
:arrow_down:

backend/shared/shared/audit_log/admin.py
87.50% <0.00%> (-12.50%)
:arrow_down:

backend/shared/shared/audit_log/viewsets.py
88.60% <0.00%> (-7.60%)
:arrow_down:

...end/kesaseteli/youth/src/hooks/useRegisterInput.ts

.../tet/admin/src/query-client/create-query-client.ts

... and 879 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Please could you @nikomakela or @tuomas777 code review this? Thank you! We can code review this live together also if you prefer.

Ok, so now tested more locally, added testing instructions for testing locally (See the description's testing section). This is ready for code review and testing! Please anyone of you @mhieta, @nikomakela or @tuomas777 if you have time could you code review this? Thank you!

Alright, now tested this locally more extensively and I'm quite satisfied that this should work correctly now. Please could you @mhieta do a rereview? Commit b398f526636394b290822f9d1e7bf22bcdc49107 is new which should fix the sorting order in the Excel output. We can also do a live code review tomorrow.


[Feature] Static build for ARM

Please add a static build for arm systems, I think it will be really useful for those small systems / empedded hw were you can't install properly.

Yeah Apple users would love an Arm build. M1 and M2 are the default at this point and having to install Rosetta to use Micro is not great.

MacOS arm binaries will now be available for all future releases and nightly releases.


Getting exception when click on Apply rules if conditions are left empty while creating catalog rule.

Framework Version: 0.1.6

Fixed

Framework Version: 0.1.6

Not Fixed

Framework Version: 0.1.6

Fixed


Option to remove spaces between tuple indexes.

When indexing a tuple, a chain of more than 1 index is formatted to have spaces in between every index, like so:
let x = a.0.1.2;

becomes
let x = a.0 .1 .2;

In my opinion this looks worse than having them condensed, and it also means, in the case of VS code, they no longer have any syntax highlighting. It would be nice if there was an option to disable the spaces.
That being said, I don't know if this is a bug. On one hand it feels like something that shouldn't happen because it's not very nice looking, but then again if it wasn't bug, it's a common thing to do so it feels weird that there's no option to disable it.

Never mind, that's actually correct. Was discussing something sort of related and they explained that the spaces means it gets lexed properly otherwise it can look like a float, which I did not realise.


Can we do a link attribute rel configuration?

Howdy,
Trying to figure out how to configure for import in a link tag. So how to make this:
        <!-- build:[href] components -->
        <!-- /build -->

do this:
<link rel="import" href="../../polymer-components/src/components/input/some-input.html">

I'm getting closer:
 <!-- build:process components -->
        <script rel="import" href="<%= ACTUAL-SOURCE %>">
        </script>
        <!-- /build -->

how do i replace  ACTUAL-SOURCE with the url specified in the gruntfile?

<!-- build:[href] components/ -->
  <link rel="import" href="../../polymer-components/src/components/input/some-input.html">
<!-- /build -->

When I do that, i get href="" how do i get it to insert the path from the config/path scanner into the href? I can't hard code it as you show it, I thought that was the whole point of using html build.


Tearing of textures

This is tracking https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=44141 specifically.

Tracks two separate internal issues:
https://bugs.fuchsia.dev/p/fuchsia/issues/detail?id=44141
and
https://b.corp.google.com/issues/148504495.
cc @iskakaushik @chinmaygarde

@cbracken As explained here, the two issues are unrelated (till I can prove otherwise). I suggest we find someone else to investigate this in parallel so as not to stall progress here.

@chinmaygarde Thanks for updating! Will un-de-dupe.

Sounds like @iskakaushik is working on this. Thanks!

fixed by https://github.com/flutter/engine/pull/16548


chore(main): release 0.39.0

:robot: I have created a release beep boop
0.39.0 (2023-10-31)
Features

release new version (9f8a6e1)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/samuele-cozzi/obsidian-marp-slides/releases/tag/0.39.0 :sunflower:


Add lo.hn

Please add my personal site https://lo.hn
6.2kb uncompressed
https://gtmetrix.com/reports/lo.hn/RLchIhIl/

Wow, that domain name! Lucky shot? And nice to see that you self-host all the things! Definitely added.

Thanks! I got it when Honduras lost control of the .hn to ICANN about ten years ago.
And yes. I like to self host everything.


Add required support for Combobox

The Combobox doesn't support a built it option to mark it as required when included in a form.

+1 i have been searching for it in all docs, cant find. any workaround available?

I couldn't find a workaround.

Since the combobox component is actually composed of the popover and the command component, it will probably be difficult to add a required field to the combobox component, since neither the popover nor the command component are typical form elements.
However, if you use react-hook-form and (more importantly) zod, as suggested in the documentation, you can mark a field as required as follows:
const FormSchema = z.object({
  language: z.string({
    required_error: "Please select a language.",
  }),
})

Here is the implemented example.
However, if you are more concerned with the UX, a common option is to mark required fields with a specific symbol (*) and indicate at a specific point that marked fields are required. Hope that helps!

This issue has been automatically closed because it received no activity for a while. If you think it was closed by accident, please leave a comment. Thank you.

I don't think it was solved....

You can always add a hidden input and forward the required to it together with all other form related props
<input type="hidden" required={required} value={value} name={name} onChange={onChange} .... />
I use it like that for any SSR for example remix-run


Looking for new maintainers

For my personal sake, I am too busy to contribute in the future for a while,
and this repository is looking for new maintainers now.
Any one who would like to take over this plugin and make improvements for it
can feed me back.

@skywind3000 fair enough, we've all got busy lives. Until maintenance for this repo picks up again, for those looking for a package to run similar functionality, process-palette looks like it's still being maintained regularly.

I have a fork going, which I will release today under dqs-shell-packages. It should fix most of these issues, including allowing a .shell-commands.cson file to be placed in the project's root directory to specify commands, instead of just the user's config file.

@skywind3000 抱歉 , 最近在准备考研，也没太有时间。


Change enclave signature to be over proposal rather only args

Description
The enclave signature for a transaction should be other the whole signed proposal (rather only over the (encrypted) args to also securely bind the creator. (This also requires changing edl to pass proposal to enclave ...)
Link to feature or bug
As discussed during PR #457 review.
Link to dependent issues

Implemented in #501 (note though also #504)


I2C may not respond properly to a repeated start condition

When encountering repeated START conditions, the I2C device FSM (as drawn and implemented) appears to return to the idle state and doesn't prepare the device to listen for its address to be repeated and continue operating. The documentation also appears to show that after a repeated start condition the device is intended to stop listening for addresses which conflicts with the I2C spec for handling of a repeated START.
https://docs.opentitan.org/hw/ip/i2c/doc/#design-details The device FSM shows STOP and RESTART both returning the state machine to the same status
1172 Shows the issue in RTL where the state machine doesn't respond to the difference between START and STOP conditions as it should.

I'll make sure the i2c_device test I'm working on hits this behavior to confirm/demonstrate the issue

thanks @drewmacrae !

actually looking at this a bit more.
When we get a read, the sequence is something like..
START..ADDR.. R/W bit... ACK START... Data bytes STOP
If you follow the fsm diagram (and i think the implementation matches at a quick glance), upon the completion of the ADDR ACK, we are at the ADDR_ACK state at the top.  This state itself does not look at START / STOP, instead it just assumes the following clocks should be used for byte transmission.  Its only at the end of that THAT ack, does it observe STOP / START again.
So I think the sequenced above "should" work no?
it's a sequence like this
START .. ADDR .. R/W bit .. ACK START .. DATA bytes ...START that would cause it to return to idle.

The sequence that would fail is the typical addressable read case, where a write transaction gives the command / address, then a repeated start, then a read command to acquire the data.
So it looks like this:

H: START
H: Addr / Wr
T: ACK
At least once

H: Data
T: ACK

H: START (repeated)
H: Addr / Rd
T: ACK
Zero or more times

T: Data
H: ACK

T: Data (last byte)
H: NAK
H: STOP

I'd worry about using or relying on the sequence:
START..ADDR.. R/W bit... ACK START... Data bytes STOP as it might address another device if the Data matches its address.

Figure 13. shows a repeated start https://www.nxp.com/docs/en/user-guide/UM10204.pdf that would return to idle when the address should be checked.

yep my bad, I got this confused in my head.
I was somehow only thinking about the latter half of this sequence, and also mistakenly assumed the repeated start was between the second "device address" and the data bytes return.

I feel like START and STOP are the highest priority events (other than an out-of-band reset). They probably ought to be automatic arrows to AddrRead and Idle, respectively. Then, what happens to the in-progress byte depends on which state we're leaving. For example, a START that is not in the appropriate place suggests a misbehaving or recently-online host causing bus contention in a multi-host environment.

yeah agreed. I think there's some similar issue with this on the host side also, remember the glitch onscl i saw?
i'll bring that wave up again today and have a closer look. Maybe add to this issue.


TextFormField deletes 2 characters at once on Android.

-TextFormField is deleting 2 characters with the default android keyboard.
-Expected behaviour should be that backspace only deletes 1 character at a time.
-Issue is not appearing on Iphone devices.
Issue is related to issue 70744, but that has been closed.
I was able to reproduce this bug on the following platforms:
-VFD 510 Android 7.1.1 (physical device)
-Nexus 5 API 29 Android 10 (emulator)
I am attaching a modified flutter demo file (main.dart) with the relevant code to reproduce the bug, and also flutter doctor output.
Steps to reproduce:
-Run the attached main.dart on an Android device.
-Type a lengthy text to the TextField.
-Delete the text with the default Android keyboard backspace button.

Flutter doctor -v output:
D:\AndroidProjects\double_delete_bug>flutter doctor -v
[√] Flutter (Channel beta, 1.24.0-10.2.pre, on Microsoft Windows [Version 6.1.7601], locale hu-HU)
• Flutter version 1.24.0-10.2.pre at D:\Android\flutter
• Framework revision 022b333a08 (4 months ago), 2020-11-18 11:35:09 -0800
• Engine revision 07c1eed46b
• Dart version 2.12.0 (build 2.12.0-29.10.beta)
[√] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
• Android SDK at D:\Android
• Platform android-29, build-tools 29.0.2
• Java binary at: D:\Android\Android Studio\jre\bin\java
• Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b04)
• All Android licenses accepted.
[√] Android Studio (version 3.6)
• Android Studio at D:\Android\Android Studio
• Flutter plugin version 47.1.1
• Dart plugin version 192.8052
• Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b04)
[√] Connected device (2 available)
• VFD 510 (mobile)                   • 8639d414      • android-arm • Android 7.1.1 (API 25)
• Android SDK built for x86 (mobile) • emulator-5554 • android-x86 • Android 10 (API 29) (emulator)
• No issues found!

main.zip

I have a different bug to report with the same files.
Steps to reproduce:
-Run the attached main.dart on an Android device.
-Type a lengthy text to the TextField.
-Use the Floating action button to reveal the typed text. (all should be fine so far)
-Delete some of the text with the default Android keyboard backspace button.
-Use the Floating action button to reveal the typed text. (the text is now stars instead of the expected actual string)
Should I open a new issue for this?

Hi @Qwampa
Your flutter version is four months old, please upgrade and try again.
Thank you

Should I open a new issue for this?

If you face this issue after the upgrade, the yes.

Indeed I was using an old version. It works as expected with 2.0.2. Thank you for your time and quick response.


Crash when making a long cluster script

I was making a 2D parameter scan (delay1, delay2) on a sequence, and it crashed under windows, reproducibly

Found the cause: if the currently-loaded simulation is very large (this one was 2810 total simulation in the batch), the count timer will allocate new memory, and fail when the system runs out.

This job was so large that when I worked around the problem (run a small simulation first), it ran out of memory on the cluster. Maybe it would be good to have a max size warning


how to turn off auto serialization?

Description
I have an endpoint that returns an html page. How do I turn off serialization so that it renders in the browser?
E.g.,
def my_handler():
    return render_template(...)

Output of the commands:

python --version
3.6.3
pip show connexion | grep "^Version\:"
Version: 1.1.10

From the documentation, it looks like you can either return a Response object, which will be used as-is, or set the endpoint to have a different mime-type in the OpenAPI file (it only serializes if it's a JSON endpoint). See https://connexion.readthedocs.io/en/latest/response.html.


Testing a pip install doesn't work

🐛 Bug
Running pytest --pyargs napari should run the napari test suite, but right now, this fails:
 $ pytest --pyargs napari
============================= test session starts ==============================
platform darwin -- Python 3.7.6, pytest-5.3.2, py-1.8.1, pluggy-0.13.1
PySide2 5.14.0 -- Qt runtime 5.14.0 -- Qt compiled 5.14.0
rootdir: /Users/jni
plugins: ordering-0.6, qt-3.3.0, cov-2.8.1
collected 149 items / 4 errors / 145 selected

==================================== ERRORS ====================================
_ ERROR collecting conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/_qt/tests/test_qt_viewer.py _
ImportError while importing test module '/Users/jni/conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/_qt/tests/test_qt_viewer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/_qt/tests/test_qt_viewer.py:6: in <module>
    from napari.tests.utils import (
E   ModuleNotFoundError: No module named 'napari.tests'
_ ERROR collecting conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/layers/image/tests/test_big_image_timing.py _
ImportError while importing test module '/Users/jni/conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/layers/image/tests/test_big_image_timing.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/layers/image/tests/test_big_image_timing.py:2: in <module>
    import zarr
E   ModuleNotFoundError: No module named 'zarr'
_ ERROR collecting conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/layers/image/tests/test_image.py _
ImportError while importing test module '/Users/jni/conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/layers/image/tests/test_image.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/layers/image/tests/test_image.py:4: in <module>
    import xarray as xr
E   ModuleNotFoundError: No module named 'xarray'
_ ERROR collecting conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/layers/tests/test_serialize.py _
ImportError while importing test module '/Users/jni/conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/layers/tests/test_serialize.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
conda/envs/napari-0.2.10/lib/python3.7/site-packages/napari/layers/tests/test_serialize.py:4: in <module>
    from napari.tests.utils import layer_test_data
E   ModuleNotFoundError: No module named 'napari.tests'

To Reproduce
conda create -y -n napari-0.2.10 python=3.7
conda activate napari-0.2.10
pip install --pre napari
pip install pytest pytest-cov pytest-ordering pytest-qt pytest-timeout
pytest --pyargs napari

Note also, in a related issue, that I think our tests should be able to run without our optional dependencies, zarr and xarray, ie, we should add pytest.skip or pytest.importorskip to the relevant tests (and they should be in their own files if the latter because that function only works for full files).

Expected behavior
pytest --pyargs napari should run our complete test suite using the installed napari module. This makes it easier to test releases, among other things.

Environment
napari: 0.2.10rc0
Platform: Darwin-19.2.0-x86_64-i386-64bit
Python: 3.7.6 (default, Jan  8 2020, 13:42:34)  [Clang 4.0.1 (tags/RELEASE_401/final)]
Qt: 5.14.0
PySide2: 5.14.0
VisPy: 0.6.4
NumPy: 1.18.1
SciPy: 1.4.1
scikit-image: 0.16.2
Dask: 2.9.1

Please copy and paste the information at napari info option in help menubar here:

Any other relevant information:

Additional context

@kne42 @ttung do either of you know why we might be getting the
E   ModuleNotFoundError: No module named 'napari.tests'

error and how to fix? Is something not getting included in the manifest? Do we want to just import all our tests in some private way in the init? Any suggestions welcome

This is because napari.tests is not a package (no __init__.py), and as such, is not bundled into the napari package.
In my development environment, I run pip install -e . (or python setup.py develop), followed by pytest --pyargs napari and that works.

I strongly suspect that if you run pytest --pyargs napari.utils.test, everything would work, and that is because napari.utils.test is an actual package.

@kne42 and I chatted about making all our tests available under napari._tests, she agreed that was a good approach

Why? I haven't seen that used in any other scipy-ish packages... I think it's easier for navigation to place tests next to their respective packages.

I think it's easier for navigation to place tests next to their respective packages.

We've done that (in starfish) and I think it's great.

Why? I haven't seen that used in any other scipy-ish packages... I think it's easier for navigation to place tests next to their respective packages.

Ah @jni if that is directed at me there was probably some misunderstanding as I was just trying to repeat something you had suggested in the zulip.
I feel like I may be just getting in the way here and @ttung @jni and @kne42 probably agree with each other on the best approach. My apologies.
It would be great though if one of you could claim the issue and then at least we know we have a plan for who will fix this.

The only suggestion I remember making is to replace napari.tests.utils with napari.utils._testing, ie, never cross import from things in tests folders.


UPDATE: improve docs

:tada: This PR is included in version 5.3.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add rcu-ish component building to the ractive bin - RFC

Description of the pull request:
This splits the ractive bin script up into modules that could conceivably be used by third parties. It stays dep-free, so there are a few things that rcu offers that aren't present here, namely source mapping and mangling various forms of dependency loading.
Instead of being very clever, this takes the dumb approach to compiling a component. It merely collects up the relevant files, converts them to the appropriate string form, does a string substitution on the script portions, and plops the result back out to STDOUT or the designated file. This leaves any module craziness up to the component author, but I'd recommend using es module format, as it translates reasonably well into other formats where needed. It's also a bit less cluttered.
Here's what this will do:

Read in a base template file from STDIN, a given file, or a given directory with an index.html or index.ractive and parse it as a template
Iterate through the top-level bits of template looking for script, template, style, and link tags

script tags that are text/ractive or text/html and have an id or name are turned into partials - if the script has a src, it is read from the designated file relative to the component file
script tags that are text/javascript (or untyped) are concatenated in order and used as the basis of the component. It's generally only recommended to have one, but this doesn't argue about having more.
template tags that have a name or id are parsed as partials
style tags have their content added to the component css string
link tags that are type text/css or component and rel ractive or component with an href will have the file at href read and appended to the component css string
any processed tags are removed from the template

The extra partials are parsed and have their template set to the appropriate partial id/name and their expressions added to the main template expressions
The css has its whitespace collapsed and it is JSON.stringify-ed
The script has any $TEMPLATE and $CSS identifiers replaced with the stringified template and css respectively

The whole chunk of relevant code ended up being ~120 lines, which I think is acceptable for a component compiler. External deps should be as simple (when using ES module format) as import Ractive from 'ractive'; import OtherComponent from '../OtherComponent;or whatever, and the result can be shared with the world byexport`ing it. I believe module bundlers like rollup would work nicely with this sort of format.
Fixes the following issues:
#1051 #2786 and it could fix #1892 by allowing a $PARTIALS sub that spit out a stringified object with all of the partials, but I'm not sure how useful that is in the context of csp and more powerful yielding.
Is breaking:
Nope.
Reviewers:
@fskreuz and anyone else interested in having a Ractive CLI for (hopefully) less painful component building and template pre-parsing.

Reading lodash earlier to see how it split up its functions into separate modules while keeping one with everything.
We can have Ractive loaded as is:
// The lib gets returned
var Ractive = require('ractive');

But then have rcu functions be loaded like this:
// Submodules that comprise rcu. They require('ractive') on their own.
var parse = require('ractive/parse');
var make = require('ractive/make');
var generateSourceMap = require('ractive/generateSourceMap');

Then the bin just becomes a CLI for those same functionality.
// stdin input, stdout output
ractive make input.html

// file input, file output
ractive make -i input.html -o output.html

// All the rest
ractive parse ...
ractive sourcemap ...

That way, it can both serve those who prefer the CLI way of doing things and the ones who make Ractive part of their node tools. Two birds with one stone n_n)b

I think I'm only one step off of a reasonably consumable API stuck right in the ractive module then. I just need to move the output handling up a layer and let the component module work directly with strings. It will still have to load files in some way. I suppose the polite way to do that would be to accept a loading function like rcu and promisify everything. Then the bin wrapper can provide a default node file loader that handles adjusting paths to be relative to the input.
I don't think this will ever quite replace rcu, as it doesn't quite have the same intentions. It looks like rcu is mostly isomorphic, but I don't see a whole lot of point to that for bundling with the ractive module. There's just so much more mileage you can get out of pre-building and bundling app and library code. I keep hearing that http2 makes having multiple requests actually preferable to bundling, but it seems like the added complexity of having a server smart enough to know which scripts it needs to push out with certain requests would be trounced by simply bundling the scripts up as a build step. I say simply because rollup is relatively simple :smile:. I still haven't muddled through a webpack or browserify build.
I also don't think sourcemap support here is particularly useful. The only actual source that ends up in the output is the script source, which is recommended to come from a single script tag. Everything else, being styles and template chunks, is neither executable nor sourcemap-able. And generating/interacting with the various module formats is way beyond the scope of what I think this should handle.
So, I think that leaves something like require('ractive/lib/component') handing back an object with a function that takes a few options (the primary string template, a file loader function, parser options) and returns a promise of a string. The the much simpler require('ractive/lib/parse') would hand back an object with a parse function, which takes a string template and hands back a string of compiled template, and a stringify function, which takes a compiled template and stringify-s it. Does that sound reasonable?
I think adjusting the modules like that will make this reasonably testable, which is something I was trying to figure out how best to approach.

Oh snap :facepalm: Your second paragraph reminded me that rcu is more than just a node module/build time dependency. It's actually an embeddable script. I was looking at everything from a node build tool dev dependency perspective. They might have to stay separate in the end. D:

I've adjusted the library functions so that they could conceivably be used by third parties:

require('ractive/lib/parse').parse('some template', { parse: options }).then(str => 'do something with the stringified parsed template');
require('ractive/lib/component').build('some template', { parse: options }, function readFile(path) { return aPromiseForTheFileContent; }).then(str => 'do something with the component js');

I also added the $PARTIALS replacement, which just subs the JSON.stringify-ed partials from script tags, and adjusted the stringified template output so that it doesn't quote stuff that doesn't really need to be quoted. That saves a fair amount of space for parsed templates.


mistserver_alpine_minimal Docker Image ships with incompatible libgcc

mistserver_alpine_minimal container ships with custom libgcc that conflicts with stock libraries and makes it impossible to extend the container with stock Alpine packages - notably ffmpeg.
In the console capture below - /MistController runs fine until libgcc is installed via apk. The /usr/lib/libgcc_s.so.1 file that ships with the image works fine with MistServer executables but break anything else in Alpine that depends on it.
[m1tk4@BLUEY ~]$ docker pull ddvtech/mistserver_alpine_minimal
Using default tag: latest
latest: Pulling from ddvtech/mistserver_alpine_minimal
Digest: sha256:64a2f7f7273488ac3ea6d605a7c8b21b2b801a0a3e8a4f9bd3c64cf1c3993219
Status: Image is up to date for ddvtech/mistserver_alpine_minimal:latest
docker.io/ddvtech/mistserver_alpine_minimal:latest
[m1tk4@BLUEY ~]$ docker run --detach --shm-size=4g --network=host ddvtech/mistserver_alpine_minimal
c915c7322f8fe6d964e8ef712f1c880ceec04ecbd88c2ed107a7361804ae42c9
[m1tk4@BLUEY ~]$ docker ps
CONTAINER ID   IMAGE                               COMMAND             CREATED         STATUS         PORTS     NAMES
c915c7322f8f   ddvtech/mistserver_alpine_minimal   "/MistController"   4 seconds ago   Up 2 seconds             zen_brattain
[m1tk4@BLUEY ~]$ docker exec -it zen_brattain /bin/sh
/ #
/ # cd /
/ # ./MistController
Account not set, do you want to create an account? (y)es, (n)o, (a)bort: [2024-12-27 14:49:05] MistController (80) WARN: Your shared memory is half or less of your RAM (4096 / 31950 MiB). Mist heavily relies on shared memory: please ensure your shared memory is set to a high value, preferably ~95% of your total available RAM.
[2024-12-27 14:49:05] MistController (80) WARN: Tip: In most cases, you can change the shared memory size by running `mount -o remount,size=30353m /dev/shm` as root. Doing this automatically every boot depends on your distribution: please check your distro's documentation for instructions.
^C[2024-12-27 14:49:07] MistController (80) CONF: Controller started
Killed all processes, wrote config to disk. Exiting.
/ #
/ # ls -al /usr/lib/libgc*
-rw-r--r--    1 root     root        182408 Nov 20 11:18 /usr/lib/libgcc_s.so.1
/ #
/ #
/ #
/ # apk add libgcc
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/x86_64/APKINDEX.tar.gz
(1/1) Installing libgcc (13.2.1_git20240309-r0)
OK: 8 MiB in 15 packages
/ #
/ # ls -al /usr/lib/libgc*
-rw-r--r--    1 root     root        141096 Mar 12  2024 /usr/lib/libgcc_s.so.1
/ #
/ # /MistController
/MistController: error while loading shared libraries: libc.musl-x86_64.so.1: cannot open shared object file: No such file or directory
/ #
/ # apk info -L libgcc
libgcc-13.2.1_git20240309-r0 contains:
usr/lib/libgcc_s.so.1

/ #

follow up - the image built via https://github.com/DDVTECH/mistserver/tree/feature/alpine_docker_builds branch does not have this problem, just the official one at https://hub.docker.com/r/ddvtech/mistserver_alpine_minimal

Yeah - this is a known thing. The official image is build from scratch on a independent system and then copied to the docker images with all dependencies as-is. The alpine version merely does this same thing on top of alpine instead. Which means that indeed, it will have an incompatible libgcc in there (and probably several other incompatible libraries).
The feature/alpine_docker_builds branch is work towards an alpine-compatible version, but it's not quite ready for prime time yet. (As you may have noticed from trying to run it after compiling it yourself: it doesn't actually work yet.)

Thanks for the response and re feature/alpine_docker not being ready for prime time - can you please elaborate on what's broken? I've actually build alpine docker images both based on master and development branches and (in my minimal testing and with my limited scenarios) both worked fine. Anything specific that is broken in Alpine builds?
Thanks again.

Never mind, I see the problem with alpine.

Yeah it definitely looks like it works at first glance, but any closer inspection and you'll notice almost every feature is broken. 🙃 We're working on fixing that though! 🛠


Finetuning process

@jik876 -  I would like to ask about your finetuning process. Specifically, what are some of the things to consider?
Let's say I'm training from the LJSpeech model to another single speaker model.

Do I consider using the learning rate scheduler at all, or just use a constant learning rate for the generator and discriminator?

For the learning rate themselves (and I guess this really depends on the dataset too), do you use a much lower learning rate?

Do you only train the generator, the discriminator, or both, when fine-tuning?

Thanks for your interest.

In our fine-tuning experiments, the effect of the scheduling may have been relatively small, but if you use a different speaker dataset from the original and adjust the initial learning rate, using the learning rate scheduling can lead to better quality.
We used a learning rate calculated by the scheduler according to training step. You are using a different speaker dataset from the original, so this may need to be adjusted.
We trained both.


[python3] Connect server with a wrong ip but  succeeded unexpectedly

What version of gRPC and what language are you using?
grpcio v1.24.1
grpcio-tools v1.24.1
python 3.6.9
What operating system (Linux, Windows,...) and version?
Ubuntu 18.04
What runtime / compiler are you using (e.g. python version or version of gcc)
python 3.6.9
What did you do?
I launched a server locally, and process pytest for my test case connecting server with a wrong ip <IP_ADDRESS>, strangely client connected successfully.  I set grpc environment varialbes export GRPC_VERBOSITY=DEBUG export GRPC_TRACE=http,api,client_channel_routing,cares_resolver and the output message is:
I1112 20:08:26.871317464   15110 init.cc:164]                grpc_init(void)
I1112 20:08:26.871340186   15130 completion_queue.cc:682]    cq_end_op_for_next(cq=0x55f8b8d7cbe0, tag=0x7f504699eef0, error={"created":"@1573560506.871317449","description":"Timed out waiting for connection state change","file":"src/core/ext/filters/client_channel/channel_connectivity.cc","file_line":142}, done=0x7f50482286c0, done_arg=0x7f5030015850, storage=0x7f5030015938)
I1112 20:08:26.871348393   15110 completion_queue.cc:504]    grpc_completion_queue_create_internal(completion_type=0, polling_type=0)
I1112 20:08:26.871354424   15110 completion_queue.cc:504]    grpc_completion_queue_create_internal(completion_type=0, polling_type=0)
I1112 20:08:26.871362007   15110 channel_create.cc:97]       grpc_insecure_channel_create(target=<IP_ADDRESS>:19530, args=0x7f50469b2390, reserved=(nil))
I1112 20:08:26.871368363   15126 completion_queue.cc:1060]   RETURN_EVENT[0x55f8b8d7cbe0]: OP_COMPLETE: tag:0x7f504699eef0 ERROR
I1112 20:08:26.871372004   15110 init.cc:164]                grpc_init(void)
I1112 20:08:26.871417625   15110 client_channel.cc:1367]     chand=0x55f8b8dad3d8: creating client_channel for channel stack 0x55f8b8dad1f8
I1112 20:08:26.871566469   15126 channel_connectivity.cc:232] grpc_channel_watch_connectivity_state(channel=0x55f8b8da8290, last_observed_state=3, deadline=gpr_timespec { tv_sec:<PHONE_NUMBER>, tv_nsec: 71561216, clock_type: 1 }, cq=0x55f8b8d7cbe0, tag=0x7f504699ee80)
I1112 20:08:26.871580786   15126 completion_queue.cc:960]    grpc_completion_queue_next(cq=0x55f8b8d7cbe0, deadline=gpr_timespec { tv_sec:<PHONE_NUMBER>, tv_nsec: 71579166, clock_type: 1 }, reserved=(nil))
I1112 20:08:26.871604159   15131 channel_connectivity.cc:43] grpc_channel_check_connectivity_state(channel=0x55f8b8dad190, try_to_connect=1)
I1112 20:08:26.871625571   15131 resolving_lb_policy.cc:196] resolving_lb=0x7f5028000f60: starting name resolution
I1112 20:08:26.871630817   15131 client_channel.cc:1273]     chand=0x55f8b8dad3d8: update: state=CONNECTING picker=0x7f50280010e0
D1112 20:08:26.871636256   15131 dns_resolver_ares.cc:176]   (c-ares resolver) resolver:0x7f5028001100 AresDnsResolver::StartLocked() is called.
D1112 20:08:26.871640254   15131 grpc_ares_wrapper.cc:605]   (c-ares resolver) request:0x7f50280015e0 c-ares grpc_dns_lookup_ares_locked_impl name=<IP_ADDRESS>:19530, default_port=https
D1112 20:08:26.871688129   15131 dns_resolver_ares.cc:427]   (c-ares resolver) resolver:0x7f5028001100 Started resolving. pending_request_:0x7f50280015e0
I1112 20:08:26.871693731   15131 client_channel.cc:1551]     chand=0x55f8b8dad3d8: created resolving_lb_policy=0x7f5028000f60
I1112 20:08:26.871698317   15131 resolving_lb_policy.cc:441] resolving_lb=0x7f5028000f60: got resolver result
I1112 20:08:26.871716241   15131 resolving_lb_policy.cc:339] resolving_lb=0x7f5028000f60: Creating new child policy pick_first
I1112 20:08:26.871722791   15131 resolving_lb_policy.cc:393] resolving_lb=0x7f5028000f60: created new LB policy "pick_first" (0x7f5028001be0)
I1112 20:08:26.871726090   15131 resolving_lb_policy.cc:356] resolving_lb=0x7f5028000f60: Updating child policy 0x7f5028001be0
I1112 20:08:26.871751023   15131 client_channel.cc:831]      chand=0x55f8b8dad3d8: creating subchannel wrapper 0x7f50280039f0 for subchannel 0x7f5028002d20
I1112 20:08:26.871772593   15131 client_channel.cc:992]      chand=0x55f8b8dad3d8: connectivity change for subchannel wrapper 0x7f50280039f0 subchannel 0x7f5028002d20 (connected_subchannel=(nil) state=CONNECTING); hopping into combiner
I1112 20:08:26.871833742   15131 chttp2_transport.cc:839]    W:0x7f50280057e0 CLIENT [ipv4:<IP_ADDRESS>:19530] state IDLE -> WRITING [TRANSPORT_FLOW_CONTROL]
I1112 20:08:26.871840412   15131 chttp2_transport.cc:839]    W:0x7f50280057e0 CLIENT [ipv4:<IP_ADDRESS>:19530] state WRITING -> WRITING+MORE [INITIAL_WRITE]
I1112 20:08:26.871854286   15131 subchannel.cc:1086]         New connected subchannel at 0x7f5028001af0 for subchannel 0x7f5028002d20
I1112 20:08:26.871862865   15131 chttp2_transport.cc:1900]   perform_transport_op[t=0x7f50280057e0]: ON_CONNECTIVITY_STATE_CHANGE:p=0x7f5028002958:from=READY BIND_POLLSET_SET
I1112 20:08:26.871867873   15131 client_channel.cc:992]      chand=0x55f8b8dad3d8: connectivity change for subchannel wrapper 0x7f50280039f0 subchannel 0x7f5028002d20 (connected_subchannel=0x7f5028001af0 state=READY); hopping into combiner
I1112 20:08:26.871871851   15131 client_channel.cc:1040]     chand=0x55f8b8dad3d8: processing connectivity change in combiner for subchannel wrapper 0x7f50280039f0 subchannel 0x7f5028002d20 (connected_subchannel=(nil) state=CONNECTING): watcher=0x7f5028002930
I1112 20:08:26.871875330   15131 client_channel.cc:1273]     chand=0x55f8b8dad3d8: update: state=CONNECTING picker=0x7f5028001810
I1112 20:08:26.871879886   15131 client_channel.cc:1040]     chand=0x55f8b8dad3d8: processing connectivity change in combiner for subchannel wrapper 0x7f50280039f0 subchannel 0x7f5028002d20 (connected_subchannel=0x7f5028001af0 state=READY): watcher=0x7f5028002930
I1112 20:08:26.871883198   15131 client_channel.cc:1273]     chand=0x55f8b8dad3d8: update: state=READY picker=0x7f5028004fc0
I1112 20:08:26.871887166   15131 client_channel.cc:1477]     chand=0x55f8b8dad3d8: updating subchannel wrapper 0x7f50280039f0 data plane connected_subchannel to 0x7f5028001af0
I1112 20:08:26.871894206   15131 writing.cc:116]             CLIENT: Ping sent [ipv4:<IP_ADDRESS>:19530]: 2/2
I1112 20:08:26.871897812   15131 chttp2_transport.cc:839]    W:0x7f50280057e0 CLIENT [ipv4:<IP_ADDRESS>:19530] state WRITING+MORE -> WRITING [begin write in current thread]
I1112 20:08:26.871908089   15131 chttp2_transport.cc:2660]   ipv4:<IP_ADDRESS>:19530: Start BDP ping err="No Error"
I1112 20:08:26.872195762   15131 channel_connectivity.cc:232] grpc_channel_watch_connectivity_state(channel=0x55f8b8dad190, last_observed_state=0, deadline=gpr_timespec { tv_sec:<PHONE_NUMBER>, tv_nsec: 72190720, clock_type: 1 }, cq=0x55f8b8d59b40, tag=0x7f50469ab470)
I1112 20:08:26.872209883   15131 completion_queue.cc:682]    cq_end_op_for_next(cq=0x55f8b8d59b40, tag=0x7f50469ab470, error="No Error", done=0x7f50482286c0, done_arg=0x7f50280018f0, storage=0x7f50280019d8)
I1112 20:08:26.872216585   15131 completion_queue.cc:960]    grpc_completion_queue_next(cq=0x55f8b8d59b40, deadline=gpr_timespec { tv_sec:<PHONE_NUMBER>, tv_nsec: 72215344, clock_type: 1 }, reserved=(nil))
I1112 20:08:26.872221769   15131 completion_queue.cc:1060]   RETURN_EVENT[0x55f8b8d59b40]: OP_COMPLETE: tag:0x7f50469ab470 OK
I1112 20:08:26.872287176   15131 channel_connectivity.cc:43] grpc_channel_check_connectivity_state(channel=0x55f8b8dad190, try_to_connect=0)
I1112 20:08:26.872413586   15131 channel_connectivity.cc:232] grpc_channel_watch_connectivity_state(channel=0x55f8b8dad190, last_observed_state=2, deadline=gpr_timespec { tv_sec:<PHONE_NUMBER>, tv_nsec: 72408832, clock_type: 1 }, cq=0x55f8b8d59b40, tag=0x7f50469ab470)
I1112 20:08:26.872428649   15131 completion_queue.cc:960]    grpc_completion_queue_next(cq=0x55f8b8d59b40, deadline=gpr_timespec { tv_sec:<PHONE_NUMBER>, tv_nsec: 72426872, clock_type: 1 }, reserved=(nil))

My client code is:
class GrpcMilvus(ConnectIntf):

    def __init__(self):
        self._channel = None
        self._stub = None
        self._uri = None
        self.server_address = None
        self.status = None

    def set_channel(self, host=None, port=None, uri=None):

        if host is not None:
            _port = port or "19530"
            _host = host
        elif port is None:
            try:
                config_uri = urlparse(config.GRPC_URI)
                _uri = urlparse(uri) if uri else config_uri

                if _uri.scheme != 'tcp':
                    raise ParamError(
                        'Invalid parameter uri: `{}`. Scheme `{}` '
                        'is not supported'.format(_uri, _uri.scheme))

                _host = _uri.hostname
                _port = _uri.port

            except Exception:
                raise ParamError("`{}` is illegal".format(uri))
        else:
            raise ParamError("Param is not complete. Please invoke as follow:\n"
                             "\t(host = ${HOST}, port = ${PORT})\n"
                             "\t(uri = ${URI})\n")

        if not is_legal_host(_host) or not is_legal_port(_port):
            raise ParamError("host or port is illegal")

        self._uri = str(_host) + ':' + str(_port)
        self.server_address = self._uri
        self._channel = grpc.insecure_channel(
            self._uri,
            options=[(cygrpc.ChannelArgKey.max_send_message_length, -1),
                     (cygrpc.ChannelArgKey.max_receive_message_length, -1)]
        )

    def connect(self, host=None, port=None, uri=None, timeout=3):
        if self._channel is None:
            self.set_channel(host, port, uri)

        elif self.connected():
            return Status(message="You have already connected!", code=Status.CONNECT_FAILED)

        try:
            grpc.channel_ready_future(self._channel).result(timeout=timeout)
        except grpc.FutureTimeoutError:
            raise NotConnectError('Fail connecting to server on {}. Timeout'.format(self._uri))
        except grpc.RpcError as e:
            raise NotConnectError("Connect error: <{}>".format(e))
        except Exception as e:
            raise NotConnectError("Error occurred when trying to connect server:\n<{}>".format(e))
        else:
            self._stub = milvus_pb2_grpc.MilvusServiceStub(self._channel)
            self.status = Status(message='Successfully connected! {}'.format(self._uri))
            return self.status

    def connected(self):
        if not self._stub or not self.status or not self._channel:
            return False

        try:
            grpc.channel_ready_future(self._channel).result(timeout=2)
        except (grpc.FutureTimeoutError, grpc.RpcError):
            return False
        else:
            return True

My test case code is:
def test_connect_with_invalid_uri(self, get_invalid_uri):
        milvus = Milvus()
        uri_value = get_invalid_uri
        with pytest.raises(Exception) as e:
            milvus.connect(uri=uri_value, timeout=1)
        assert not milvus.connected()

And the error output messge is:
    def test_connect_with_invalid_uri(self, get_invalid_uri):
        milvus = Milvus()
        uri_value = get_invalid_uri
        with pytest.raises(Exception) as e:
>           milvus.connect(uri=uri_value, timeout=1)
E           Failed: DID NOT RAISE <class 'Exception'>

What did you expect to see?
I was troubled by this problem for several days. Thanks for your any help!!!!
What did you see instead?
Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).
See TROUBLESHOOTING.md for how to diagnose problems better.
Anything else we should know about your project / environment?

@BossZou In my local reproduce, I found your code raises NotConnectError exception as expected. There might be some network setting in your environment that is causing this unexpected behavior. I recommend to run your code in a clean slate VM, and see if problems goes away.

@lidizheng .The bug doesn't reappear every time。 In my CI environment, This bug reappear occasionally too.

I Found the bug appear when a client object connected exists in the same process.

@BossZou Can you provide more info about how to reproduce it? With your snippet, I still can't reproduce this error, even with your latest update, adding a connected channel at side. The ideal format is a single script that can trigger this error.

Closing this down since reporter has not responded in quite some time. Please feel free to re-open if the bug persists.


feat: connect most active contributors graph to API v2 endpoint

Description

Connects the Most Active Contributors graph to the V2 API endpoint
Adds Reviewed PRs, Issues Created, and Comments to the chart data

What type of PR is this? (check all applicable)

[x] 🍕 Feature
[ ] 🐛 Bug Fix
[ ] 📝 Documentation Update
[ ] 🎨 Style
[ ] 🧑💻 Code Refactor
[ ] 🔥 Performance Improvements
[ ] ✅ Test
[ ] 🤖 Build
[ ] 🔁 CI
[ ] 📦 Chore (Release)
[ ] ⏩ Revert

Related Tickets & Documents

Mobile & Desktop Screenshots/Recordings

Steps to QA

Added to documentation?

[ ] 📜 README.md
[ ] 📓 docs.opensauced.pizza
[ ] 🍕 dev.to/opensauced
[ ] 📕 storybook
[ ] 🙅 no documentation needed

[optional] Are there any post-deployment tasks we need to perform?
[optional] What gif best describes this PR or how it makes you feel?

Let's hold off merging this till: https://github.com/open-sauced/app/pull/2469 is ironed out (don't want to accidentally overwrite your changes with a force push)

don't want to accidentally overwrite your changes with a force push

Disregard, we can merge this: there are some merge conflicts that will manually need to be fixed from alpha --> beta. So, when the time comes, we'll merge this the alpha branch to beta with a side branch based off alpha that has those conflicts resolved 👍🏼

So we can continue getting stuff onto alpha as long as we don't mind getting the latest and hottest from beta there as well

That's fine with me. We've merged in from main after a release in the past also to keep things relatively up to date, but I think that has to be done manually.

I haven't been able to check this out because of auth. I also see this screen first when clicking the deploy preview.

Curious if anyone else is seeing these errors

Probably due to issue mentioned here by @jpmcb https://github.com/open-sauced/app/pull/2470#issuecomment-1889747698

Ah. Did realize the target was alpha.

Yeah, I'm not sure if there's a better approach vs. pulling the PR locally and setting the API endpoint to alpha.api.opensauced.pizza - wish there was a more granular way to let deploy previews targeting a certain branch use certain env vars

Yeah, I'm not sure if there's a better approach vs. pulling the PR locally and setting the API endpoint to alpha.api.opensauced.pizza - wish there was a more granular way to let deploy previews targeting a certain branch use certain env vars

Working on setting this up now and see the alpha endpoint already there - https://github.com/open-sauced/app/blob/feat-most-active-contributors-api-v2/.env#L3
I don't have any lists on alpha, and getting a 401 unauthorized when trying to create one to test this feature. With two other approvals, we should move this forward and I can figure out my environment later though.


[Enhancement] API documentation in PT and EN

Since we are aiming at having the API publicly available to everyone, no matter what background they have, I think it would be useful to also have the API's documentation in EN.

You are right, we will update the documentation as soon as possible.

Closing issue has last update already includes EN/PT instructions.


Watching value only from context

I was exploring the possibility of adding riverpod to some parts of my existing project which uses provider at the moment.
One of my current providers from the provider package is the theme of the app with custom colors and such.
From any of my widgets I can do Provider.of<MyTheme>(context). I was wondering if there is a way of watching a specific riverpod provider by its name, without relying on Consumer or ConsumerWidget.
Something along the lines of:
static MyTheme of(BuildContext context){
  return  ProviderScope.containerOf(context).watch(_myThemeProvider);
}

I find migrating my current MyTheme dependency to riverpod very hard or nearly impossible if using Consumer/ConsumerWidget/useProvider is the only solution because many of my widgets depend on the signature from above.
Migrating to using Consumers can be doable but not for moderately sized projects.

This is not feasible and is in fact one of the reasons why Riverpod and Provider are separated.
This would both significantly reduce the performances and remove some features (like ".autoDispose")
The question would instead be:
Why is it hard for you to migrate to Consumer/ConsumerWidget/useProvider?

I totally understand if this is not feasible at all to do. I just wanted to share my experience from an existent Provider project standpoint.

The question would instead be:
Why is it hard for you to migrate to Consumer/ConsumerWidget/useProvider?

Simply put:

Big project with many different widgets. Error prone to update all that code.

Big project with many different widgets. Error prone to update all that code.

How so?
You'd have a compile error if you made a mistake

I guess that's true thanks to the compile safety from riverpod.

In some situations it's not so linear.
If you have StatefullWidget's where you mix setState and Provider it's a little bit harder to just migrate to ConsumerWidget or Consumer
In my case, when i had just a simple inner state, i converted to a ConsumerWidget and created a private StateProvider and used it instead of setState. In other cases where that provider state could be used in isolation, i just used a Consumer widget.
But i ended up being able to migrate our App to river_pod, replacing Provider and get_it with it! Thanks allot for your work on all of this, by the way!

@porfirioribeiro Couldn't you just turn your StatefulWidget into a ConsumerWidget? I think you could still use setState because ConsumerWidget extends from StatefulWidget :sweat_smile:

In some situations it's not so linear.

Could you give examples of those situations? One long-term plan is to make a script to automatically convert from provider to riverpod, so that would be really helpful

Also, in the short term, I want to release a migration guide. "Riverpod for Provider users" or something like that
So if you could list your pain points, I'll make sure to cover them

In some situations it's not so linear.

Could you give examples of those situations? One long-term plan is to make a script to automatically convert from provider to riverpod, so that would be really helpful

Well, provider can be injected in any widget type, so you could access it either on StatelessWidget or StatefulWidget
For river_pod you need to change your widget either to ConsumerWidget/HooksWidget or wrap it in a Consumer
When using StatelessWidget was really straightforward to migrate to ConsumerWidget, (i didn't want to also bring flutter_hooks to the project, maybe later, need to investigate it more )
Now when you have a SatefulWidget because of some setState or some controllers its a little more complicated.
@davidmartos96  Even tho ConsumerWidget extends StatefulWidget you are extending the Widget not the State, so it does not work as a StatefulWidget, in my understanding.

@porfirioribeiro Oh you are totally right.

Could you expand on the difficulties encountered with StatefulWidget?

I think the issue stems from the docs not really every explaining how to use this with a Stateful Widget.
If I understand right, this is functionally equivalent to a ConsumerWidget??
class _MyWidgetState extends State<MyWidget> {
  Widget build(BuildContext context) {
    return Consumer(
      builder: (context, watch, _) {
          bool value = watch(myModel).value;
          return ...;
      }
    )
  }
}

If that's true, then migrating from Provider shouldn't be tooo hard.
The two main issue here are

The nesting is unfortunate, at least it's only one level because one watch() method can presumably be used on multiple providers. But still, lots of lines, spaces and brackets compared to context.watch
Lack of a Select API renders this extremely unoptimized for architectures that prefer to have complex changeNotifiers as their provided dependencies.

In general I just find it very odd that we have a context.read for button handlers, but then can't have context.watch or context.select inside a stateful widget.
I also still don't full understand how we're supposed to access a provider from outside the widget tree completely, for instance if we want to implement our Application Logic using simple Command objects.
There's just so much going on, and so many options, and so many flavors it just makes it really hard to see the clear path forward when learning riverpod.

@esDotDev Yes accessing provider from outside widget tree using the Container is not clear to me either, and there is not much on the docs regarding that.
cc @rrousselGit

Using flutter_hooks, even if only for hooks_riverpod, will really simplify the process of using riverpod for many users. It eliminates the nesting among other pain points that seem to be coming up with adopting riverpod.
There is StatefulHookWidget for those who really want to maintain verbose syntax using StatefulWidget for their state while gaining simpler usage of riverpod.  Then, when you are ready to embrace concise state management, it would be possible to remove Stateful widget by widget as you convert the remaining state to use hooks.

The argument isn't that switching over to a fully hooks-based approach is not going to simplify usage, it's that it greatly complicated understanding required before usage can occur.
First should be clear demonstrations of the 'un-simplified' process, so riverpod can be evaluated on it's own terms, using some common workflows. And then, if examples are provided of accelerators using additional libs, that would be great.

Fair enough, I would still stand by the point that many if not all concerns I've seen are solved by using hooks though. Yes, more examples of non-hooks riverpod would be great, but it's definitely a case of stepping backwards to move forwards.
It seems more to me like a Flutter problem than a riverpod/hooks problem. The copious boilerplate required by vanilla Flutter (that vanilla riverpod is directly affected by) is all solved when using the simpler, more concise, hooks approach to building widgets. There are also really complex architectures that are being popularized for whatever reason that get people confused when working with riverpod/provider as well.
All that said, I'm sure everyone would welcome a contribution of an example using riverpod without hooks.

Tutorials will be available in due time (not just examples, but code with the explanation)
I'll make sure that all the tutorials I write will be available both with and without hooks. Not just so that people can start slowly, but also compare approaches.
I probably won't be using ChangeNotifier though, and focus on StateNotifier + Freezed, and instead will make a single "ChangeNotifier vs StateNotifier" article

@rrousselGit Would it be feasible to implement a custom hook called useWatch? It would return a ScopedReader, same as ConsumerWidget. That would make migrating StatefulWidgets easier with the help of StatefulHookWidget + avoiding unnecessary nesting in the process as @esDotDev mentioned previously.

What would be the benefit over useProvider?

Oh, nevermind. I wasn't thinking about useProvider. That could be used as well.

Closing as there are no plans to implement this.

I really dislike the nesting Consumer creates. I remember that not having a Consumer widget was largely celebrated by everyone when extensions landed in Dart and context.watch was introduced to Provider. That's why I don't understand that this issue got dismissed, without a proper technical explanation.

This would both significantly reduce the performances and remove some features (like ".autoDispose")

Can you elaborate on that? If performance isn't critical to me and I don't use .autoDispose at all, why can't I get context.watch?

ConsumerWidget exists for this purpose.
It removes the nesting but doesn't rely on InheritedWidget's subscription mechanism.

Can you elaborate on that? If performance isn't critical to me and I don't use .autoDispose at all, why can't I get context.watch?

InheritedWidgets work by keeping track of all the dependent widgets bound to one InheritedWidget.
They need to be inserted in the widget tree beforehand for consumers to depend on it, and we typically have one inherited widget per state.
Riverpod is different. The providers are not present in the widget tree until the moment that we read them. And we don't have one "InheritedWidget" per provider, but rather one for all providers.
So trying to fit providers in the InheritedWidget model requires heavy workarounds.
Everytime we would like a provider to emit an update, we would have to iterate over all providers and all consumers (including those that called context.watch for a different provider). This simply doesn't scale.
By relying on ConsumerWidget, when a provider wants to emit an update, we evaluate only that provider and iterate over only its consumers.
Considering ConsumerWidget solves this problem with pretty much no downside, while also being more future-proof (with autoDispose and more to come), I don't see a reason to maintain two different approaches.

autoDispose is also not the only feature that requires ConsumerWidget.
There are more to come. Such as:

the ability to have "gapless playback" for AsyncValue (as in, after the first data, going back to loading keeps showing the data).
enabling the devtool to list all the consumers associated with a provider
supporting an "onChange" that doesn't require nesting:Widget build(context, ref) {
  ref.listen<Value>(provider, (value) => showDialog(...));
}

Using ConsumerWidget also fixes bugs in InheritedWidgets. Such as https://github.com/flutter/flutter/issues/62753 and https://github.com/flutter/flutter/issues/62861

Riverpod is different. The providers are not present in the widget tree until the moment that we read them. And we don't have one "InheritedWidget" per provider, but rather one for all providers.
Everytime we would like a provider to emit an update, we would have to iterate over all providers and all consumers (including those that called context.watch for a different provider). This simply doesn't scale.

That was the missing link, thanks!

Could you give examples of those situations? One long-term plan is to make a script to automatically convert from provider to riverpod, so that would be really helpful

Hey @rrousselGit, do we still have this in our roadmap?

Yes, I still want to make a script to convert from provider to Riverpod.
Although it's a lot of work and there are more valuable things to work on atm (docs, devtools)


stop disabling firewalld

a horrible practice to show customers, even for demos

I believe this has been fixed and can be closed.


Refactor - remove EbpfError::UserError, use Box<dyn std::error::Error> instead

In the monorepo we wrap InstructionError in SyscallError and that in turn in an UserError. Through the stack of VM nesting these get unpacked and repacked multiple times. All of this can be avoided by using Box<dyn std::error::Error> instead.
Also, this change is a preparation to unify Errors, (Program)Results and the function signatures of syscalls and built-in programs.

Codecov Report

Merging #449 (e9b0001) into main (98b30ca) will increase coverage by 0.01%.
The diff coverage is 95.34%.

@@            Coverage Diff             @@
##             main     #449      +/-   ##
==========================================
+ Coverage   90.47%   90.49%   +0.01%
==========================================
  Files          23       23
  Lines        9921     9938      +17
==========================================
+ Hits         8976     8993      +17
  Misses        945      945

Impacted Files
Coverage Δ

src/error.rs
100.00% <ø> (ø)

src/interpreter.rs
97.88% <ø> (ø)

src/memory_region.rs
94.05% <77.77%> (ø)

src/jit.rs
94.16% <100.00%> (+0.07%)
:arrow_up:

src/vm.rs
76.49% <100.00%> (+0.08%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Implement Debug for Res and ResMut

This commit adds blanket implementations of Debug for Res and ResMut, as discussed in https://github.com/bevyengine/bevy/issues/2048.

As I mentioned in #2048, could you quickly add this for NonSend, NonSendMut and Local as well? Should be identical as I understand it.

Done. There was already an implementation for NonSendMut, but I updated it to show the wrapper, like the others.

bors r+


Retrieve detailed info about a user without specifying a username

Hi jwiegley,
This PR simply adds the ability to retrieve DetailedUser information about a user by relying on the GithubAuth instead of specifying a username. I need this for example, when authenticating a new user with OAuth2. All I get back is the access_token. So I need to query /user in order to obtain user info.
Thanks!

Currently the tests are not passing after merging in several PRs, so we are waiting on that and some additional testing before uploading to Hackage.  Is that anything you'd be willing to help with?

Definitely.
I had to test my modifications with that NFData PR in order to get the version on github to work. I'm currently rebuilding it with master so I can test it out. I'm running GHC-7.10.2, Let me know how I can help.
pc!

Here's the errors I encounter when building master. In order to get it to build, I apply the changes in src/Github/Data/Definitions.hs in the current NFData PR: https://github.com/jwiegley/github/pull/115
Github/Repos/Webhooks/Validate.hs:10:1: Warning:
    The import of ‘Control.Applicative’ is redundant
      except perhaps to import instances from ‘Control.Applicative’
    To import instances alone, use: import Control.Applicative()
[ 2 of 32] Compiling Github.Data.Definitions ( Github/Data/Definitions.hs, dist/dist-sandbox-ddf36791/build/Github/Data/Definitions.o )

Github/Data/Definitions.hs:592:17:
    Not in scope: type constructor or class ‘ContentData’
Updating documentation index
/github/jwiegley/github/.cabal-sandbox/share/doc/x86_64-osx-ghc-7.10.2/index.html
cabal: Error: some packages failed to install:
github-0.14.0 failed during the building phase. The exception was:


Add 129 in c language

File(s) Modified: 129-sum-root-to-leaf.c
Language(s) Used: c
Submission URL: _https://leetcode.com/submissions/detail/832121861/_

Thanks, @julienChemillier!


Add support for AMPE token to batching server

We want to have the latest price of the AMPE token on-chain. So far, we are feeding the AMPE price manually once per week and we set it to a fixed price of 1$. However, we should add support for the AMPE token to the batching server so that the latest price of the AMPE token is updated on every block and we don't need to feed it manually anymore.
TODO
We need to change the logic of the batching server to:

should work for a new asset with blockchain = Amplitude and symbol = AMPE. Capitalization should not matter here.
the price for this asset should be fetched from our Amplitude squid (https://squid.subsquid.io/amplitude-squid/graphql) with query the following query. The value of ethPrice represents the price of AMPE in USD.

query MyQuery {
  bundleById(id: "1") {
    id
    ethPrice
  }
}

be configured to use the AMPE price. Might be best to add AMPLITUDE:AMPE to the list of default values here.

GraphQL
For the graphql queries, we should use some crate like graphql-client. The schema can be found here.

Hey team! Please add your planning poker estimate with Zenhub @adelarja @b-yap @ebma @TorstenStueber

@ebma what is the batching server?
What is the flow of data here: the batching server should request the price from the squid, which gets the price from the chain and then feed it into the chain? Isn't that circular?

The batching server is the service that provides prices from the DIA price feed to the off-chain worker. So on every block the offchain worker tries to get the latest prices of the designated assets from the batching server.
The difficulty here is how we derive the AMPE price. AMPE is not listed on a CEX so DIA has no data for it. What we decided to do instead is to base the AMPE price on the ratio of the liquidity in the AMPE/KSM pool. So we check how many AMPE are 'equal' to 1 KSM according to the current pool ratio and then use the latest dollar price of KSM to define the AMPE price.
The ticket describes that the batching server should query the price from the indexer because the indexer does this price derivation on each block. However, we can also change the batching server to connect to Amplitude and query the on-chain ratio of the AMPE/KSM pool and then do the price derivation locally. IMO querying it from the indexer is easier and faster.

Makes perfect sense now and I also do remember what the batching server is now. Thanks for the explanation.

@pendulum-chain/product I realized that implementing this ticket has an implication for Spacewalk issue requests on Amplitude. At the moment we manually feed a price for AMPE of $0.1 to Amplitude. Once we support the 'actual price' represented by the Zenlink pool, the value of the AMPE token will be around $0.0042. Thus, users will have to lock twice as many AMPE as griefing collateral for issue requests than they have to lock right now. Just FYI. This is not a flaw but important for the overall bridge security.

Makes sense @ebma and how it is expected to work.

@b-yap does this require any runtime upgrade or root transaction?

No it does not.


⚠ PrivateHD has degraded performance

In 0fda9b1, PrivateHD ($PHD) experienced degraded performance:

HTTP code: 200
Response time: 1720 ms

Resolved: PrivateHD performance has improved in 07505fc.


Support Youtube/Google one-touch authentication with Android

When using 2FA to login to Youtube, there is either HTTP Error 500: Internal Server Error or HTTP Error 402.
I suspect it is because Google accounts with an active Android device allow the Android device to authorize sign in request, and 2FA is not required. This is what appears on my Android device:

Could youtube-dl support this one-click sign on for Android when available, and fall back to TOTP/2FA only when it is not available?
Make sure you are using the latest version: run youtube-dl --version and ensure your version is

[x] I've verified and I assure that I'm running youtube-dl 2017.07.23

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[x] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
$ youtube-dl -v 'https://www.youtube.com/watch?v=TAgVVc5hAiE'
[debug] System config: []
[debug] User config: ['--proxy', 'localhost:8888'<EMAIL_ADDRESS>'--mark-watched']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=TAgVVc5hAiE']
Type account password and press [Return]:
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.07.23
[debug] Python version 3.6.1 - Darwin-14.5.0-x86_64-i386-64bit
[debug] exe versions: avconv 12, avprobe 12, ffmpeg 3.3.2, ffprobe 3.3.2, rtmpdump 2.4
[debug] Proxy map: {'http': 'localhost:8888', 'https': 'localhost:8888'}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
Type 2-step verification code and press [Return]:
[youtube] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 500: Internal Server Error
WARNING: Unable to extract CheckCookie URL
[youtube] TAgVVc5hAiE: Downloading webpage

2FA support seems to be broken altogether, it doesn't work whether I use one-touch authorization or enter the TOTP code... The only way it works right now is by using --cookies.

Is 2FA still broken? You can stop google sending the prompt to your phone by disconnecting it from the internet. It will fall back to the normal authenticator codes but they don't seem to be working either.

No, 2FA is still not working with the latest version of youtube-dl as of time of this response, 2018.05.18
$ proxychains4 youtube-dl --ignore-config -2 --mark-watched -u<EMAIL_ADDRESS>-v 'https://www.youtube.com/watch?v=IYfgvS0FA7U'
[proxychains] config file found: /Users/hanxue/.proxychains/proxychains.conf
[proxychains] preloading /usr/local/Cellar/proxychains-ng/4.12_1/lib/libproxychains4.dylib
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--ignore-config', u'-2', u'--mark-watched', u'-u', u'PRIVATE', u'-v', u'https://www.youtube.com/watch?v=IYfgvS0FA7U']
Type account password and press [Return]:
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.05.18
[debug] Python version 2.7.15 (CPython) - Darwin-14.5.0-x86_64-i386-64bit
[debug] exe versions: avconv 12.3, avprobe 12.3, ffmpeg 3.4.2, ffprobe 3.4.2, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] Downloading login page
[youtube] Looking up account info
[youtube] Logging in
[youtube] Submitting TFA code
WARNING: Unable to submit TFA code: HTTP Error 400: Bad Request
[youtube] IYfgvS0FA7U: Downloading webpage
[youtube] IYfgvS0FA7U: Downloading video info webpage
[youtube] IYfgvS0FA7U: Extracting video information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://r3---sn-npoeen7r.googlevideo.com/videoplayback?ipbits=0&fvip=3&dur=1172.704&expire=1526813590&gcr=sg&id=o-AI0GyqxD9yVgFGl3RaEPQ5EeqxqNOEJ1KLfruvFmMHF1&pl=63&mm=31%2C26&mn=sn-npoeen7r%2Csn-i3belne6&ip=2400%3A6180%3A0%3Ad0%3A%3Ae73%3A6001&requiressl=yes&ms=au%2Conr&mt=1526791855&mv=m&sparams=aitags%2Cclen%2Cdur%2Cei%2Cgcr%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&keepalive=yes&ei=Nv8AW6eOIN2cowP9iYaIDw&c=WEB&lmt=1526290576056697&source=youtube&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278&initcwndbps=180000&clen=284912645&itag=137&gir=yes&mime=video%2Fmp4&key=yt6&signature=C482C5FC9A4886E5F1A082EB1B5731C8D9CB4111.AA653F50927E2F151F7A7B217B8F74D303C69509&ratebypass=yes'

Note the HTTP error
WARNING: Unable to submit TFA code: HTTP Error 400: Bad Request

Download still works though, just that I think authentication did not go through, and --mark-watched does not work. I check my YouTube history and that video does not appear.

I have  uploaded the dump files for YouTube / Google sign in at https://gist.github.com/hanxue/f5b62e6e940ab0e03a3cc60e5f130dd5

Using a browser, going to https://accounts.google.com/ServiceLogin, which is the url used by YoutubeBaseInfoExtractor, redirects me to https://accounts.google.com/signin/v2/identifier?flowName=GlifWebSignIn&flowEntry=ServiceLogin.
Perhaps youtube-dl is using v1 signin which doesn't support 2FA anymore? I don't know how to test this as I can't access the signin page that youtube-dl seems to use through a browser.
Another thing: this problem seems to have moved away from the original subject of this issue. I think there are a few issues open that are related but perhaps we should make a new one purely about 2FA not working?

The Android one-touch authentication is one of the few forms of 2FA supported by Google. If we are talking about 2FA, it could be OTP authenticator code, or code via phone/SMS. I am happy to expand the scope of this issue to cover all Google 2FA, not just Android one-touch authentication

From the source:
https://github.com/rg3/youtube-dl/blob/504f20dd302189db2cfe1cb5ee9a622c39ee693c/youtube_dl/extractor/youtube.py#L202
It seems that the authenticator app codes are supposed to be supported while other forms of 2FA (such as the one-touch authentication) aren't yet.
This is why I'm suggesting splitting the issue into one for the bug preventing the existing 2FA support working and one for the request for one-touch support


[TF2.0] Can't use keras validation data with distribution strategy

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
TensorFlow installed from (source or binary): pip tf-nightly-gpu-2.0-preview
TensorFlow version (use command below): 2.0.0.dev20190517
Python version: 3.7.3
CUDA/cuDNN version: 10 / <IP_ADDRESS>
GPU model and memory: 4 x V100

Describe the current behavior
Using keras validation data and tf.distribute.MirroredStrategy() will fail with 'BatchDataset' object is not subscriptable in TF 2.0 nightly.
Describe the expected behavior
Without distribution strategy everything works fine.
Code to reproduce the issue
import tensorflow as tf
import tensorflow_datasets as tfds

mnist_train, mnist_test = tfds.load(name='mnist', split=[tfds.Split.TRAIN, tfds.Split.TEST], as_supervised=True, data_dir="gs://plumerai/data")

strategy = tf.distribute.MirroredStrategy()

def scale(image, label):
  image = tf.cast(image, tf.float32)
  image /= 255

  return image, label

train_dataset = mnist_train.map(scale).shuffle(1000).batch(256)
test_dataset = mnist_test.map(scale).batch(256)

with strategy.scope():
  model = tf.keras.Sequential([
      tf.keras.layers.Conv2D(32, 3, activation='relu', input_shape=(28, 28, 1)),
      tf.keras.layers.MaxPooling2D(),
      tf.keras.layers.Flatten(),
      tf.keras.layers.Dense(64, activation='relu'),
      tf.keras.layers.Dense(10, activation='softmax')
  ])

  model.compile(loss='sparse_categorical_crossentropy',
                optimizer=tf.keras.optimizers.Adam(),
                metrics=['accuracy'])

model.fit(train_dataset, validation_data=test_dataset, epochs=10)

Other info / logs
  File "tf_bug.py", line 30, in <module>
    model.fit(train_dataset, validation_data=test_dataset, epochs=10)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training.py", line 649, in fit
    validation_freq=validation_freq)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training_distributed.py", line 143, in fit_distributed
    steps_name='steps_per_epoch')
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training_arrays.py", line 145, in model_iteration
    _print_train_info(inputs, val_inputs, steps_per_epoch, verbose)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/training_arrays.py", line 450, in _print_train_info
    hasattr(inputs[0], 'shape') and hasattr(val_inputs[0], 'shape')):
TypeError: 'BatchDataset' object is not subscriptable

@lgeiger Able to reproduce the issue with the provided code.
TypeError: 'BatchDataset' object does not support indexing

@lgeiger I don't see any issue when I ran the code with TF2.0 (gist is here as well as TF-nightly-gpu-2.0-preview(gist is here. Thanks!

I'm also unable to repro, @lgeiger could you try the lastest nightly? This may be already fixed

@omalleyt12 I am still able to reproduce the error even with the latest nightly (2.0.0.dev20190523)

@jvishnuvardhan @omalleyt12 Have run this code on multiple GPUs?

Updating to the latest nightly (2.0.0.dev20190528), I receive a different error message:
  File "train_multi.py", line 60, in <module>
    verbose=verbose)
  File "~/.virtualenvs/tf_v2_nightly/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 644, in fit
    use_multiprocessing=use_multiprocessing)
  File "~/.virtualenvs/tf_v2_nightly/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_distributed.py", line 899, in fit
    validation_freq=validation_freq)
  File "~/.virtualenvs/tf_v2_nightly/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_distributed.py", line 149, in fit_distributed
    steps_name='steps_per_epoch')
  File "~/.virtualenvs/tf_v2_nightly/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 415, in model_iteration
    steps_name='validation_steps')
  File "~/.virtualenvs/tf_v2_nightly/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 178, in model_iteration
    steps_per_epoch)
  File "~/.virtualenvs/tf_v2_nightly/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 469, in _get_num_samples_or_steps
    'steps_per_epoch')
  File "~/.virtualenvs/tf_v2_nightly/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_utils.py", line 256, in check_num_samples
    if hasattr(ins[0], 'shape'):
TypeError: 'function' object is not subscriptable

I was able to reproduce this new error, looking into a fix now. In the meantime a workaround should be to pass the validation_steps arg to fit

Similar error here: https://github.com/tensorflow/tensorflow/issues/30843#issuecomment-514534349

This is fixed in the latest 2.0 nightly


[CI] CemuStub build task is returning false positive results

Recently, some pull requests made it through CI that broke the CemuStub build.

Project "D:\a\RTCV\RTCV\CemuStub-Vanguard\CemuStub-Vanguard.sln" on node 1 (default targets).

12ValidateSolutionConfiguration:

13 Building solution configuration "Debug|Any CPU".

14ValidateProjects:

15 The project "CemuStub" is not selected for building in solution configuration "Debug|Any CPU".

16Project "D:\a\RTCV\RTCV\CemuStub-Vanguard\CemuStub-Vanguard.sln" (1) is building "D:\a\RTCV\RTCV\RTCV\Source\Frontend\StandaloneRTC\StandaloneRTC.csproj" (2) on node 1 (default targets).

Per the Discord chat, there are two solutions:

Use a different configuration in the RTCV CI (we'll likely want to fix CemuStub-Vanguard's CI too)
Modify CemuStub-Vanguard so that the "Debug|Any CPU" configuration is valid

I'm leaning on #2 as for all our projects, debug should build fine. I'm going to take a look at it just in case CemuStub was set to x64 for some reason (we hook the cemu window but that should still be fine from AnyCpu on a 64 bit system)

Resolved via https://github.com/redscientistlabs/CemuStub-Vanguard/commit/e64b1a59c3cd85d3a5838df3050d3d33753dabe0


Lua implementation note

Hello
There is a new pure-Lua implementation of MQTT 3.1.1 protocol here: https://github.com/xHasKx/luamqtt
The MQTT protocol packets parsing/creating may be taken from mqtt/protocol.lua module in that repo.
This module may be used in your MQTT/UDP implementation (almost) without changes because of (almost) zero dependencies.
Sorry, I don't have time resources to PR to your project, but maybe you may find something useful in my project.
Thanks for your attention, and good luck!

Thanx!

Thank you for bit proxy/implementation!