Assignment from/to MultiIndexed columns should probably be symmetric

This was counter-intuitive for me.
    N = 5
    columns = pd.MultiIndex.from_tuples([('a', 0), ('a', 1), ('b', 0), ('b', 1)])
    df1 = pd.DataFrame(np.random.rand(N, 4), columns=columns)
    df2 = pd.DataFrame(np.random.rand(N), columns=['c'])
    df1_copy = df1.copy()
    df2_copy = df2.copy()
    df1_copy[df2.columns] = df2
    df2_copy[df1.columns] = df1
    print "===== A ====="
    print df1_copy
    print
    print "===== B ====="
    print df2_copy

gives
===== A =====
          a                   b                   c
          0         1         0         1
0  0.540405  0.814476  0.923961  0.935421  0.909791
1  0.679014  0.220417  0.511039  0.117898  0.282063
2  0.428563  0.915760  0.518118  0.780704  0.766191
3  0.928167  0.616540  0.151223  0.664102  0.037299
4  0.717449  0.453904  0.384850  0.953102  0.757288

===== B =====
          c    (a, 0)    (a, 1)    (b, 0)    (b, 1)
0  0.909791  0.540405  0.814476  0.923961  0.935421
1  0.282063  0.679014  0.220417  0.511039  0.117898
2  0.766191  0.428563  0.915760  0.518118  0.780704
3  0.037299  0.928167  0.616540  0.151223  0.664102
4  0.757288  0.717449  0.453904  0.384850  0.953102

I don't care about the ordering of the columns here; most of the time we are completely agnostic to that with pandas. What I do care about is that the MultiIndex gets "flattened" in B.
Seems like a more desirable behavior would be to "promote" the target's column structure so that you always get A. That presents a tricky question about how to deal with complicated MultiIndex schemes on either side of the assignment, but at least for this basic case the current behavior feels wonky.

@brianthelion
you are doing non-symmetric things, so you get a non-symmetric answer.
pandas will align the rhs of an assignment to the lhs. So when assigning a single level index to a multi-index it can easily just put it on level 0 (and fill the rest of the levels with ''), this is your df1_copy case.
however for df2_copy you are assigning a multi-index to a single level index. so the multi-level index is aligned to a single level. The only reasonable thing to do is to flatten it, making it into a single level index (which happens to be tuples). Not pretty, and one could argue that this should actually raise, but its rationale.

@jreback

The only reasonable thing to do is to flatten it, making it into a single level index (which happens to be tuples). Not pretty, and one could argue that this should actually raise, but its rationale.

I guess I am going to change my tack here and argue that flattening is not a reasonable thing to do. Why? My feeling is that flattening the assignor breaks a standard Python idiom, namely that __setitem__ assignment shouldn't fundamentally alter the assignor. You might argue that Python idioms are imprecise beasts and that they don't exist until Guido says they do -- all of which are totally valid arguments. Nonetheless, I don't think you see too many instances in Pydom where that fail to uphold immutability across __setitem__.
In other words, we should never have a situation in which:
>>> df1_copy[df2.columns] = df2
>>> assert df1_copy[df2.columns] == df2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/br/.virtualenvs/foo/local/lib/python2.7/site-packages/pandas/core/ops.py", line 912, in f
    return self._compare_frame(other, func, str_rep)
  File "/home/br/.virtualenvs/foo/local/lib/python2.7/site-packages/pandas/core/frame.py", line 3136, in _compare_frame
    raise ValueError('Can only compare identically-labeled '
ValueError: Can only compare identically-labeled DataFrame objects

I think maybe that's the most succinct way of phrasing it; apologies for being so circumspect previously. I would encourage the devs to consider changing the behavior to reflect something more idiomatic. I think there are a couple of good options available, and that a raise is probably not necessary.

What exactly are you proposing? This is a fundamentally non-symmetric operation.

IMHO, the most important thing is keeping the __setitem__ interface consistent. If you assign something via df['A'] = B then you should be able to access it -- and not some fundamentally altered version of it -- via df['A']. Note that I'm not talking about strict immutability (df['A'] is B) here; just functional equivalence (df['A'] == B).
I don't disagree that something fundamentally non-symmetric is happening. I just can't think of any situations with __setitem__ in which the expectation for interface adjustment isn't on the container object. In the present case specifically, I think it's much more natural to "upgrade" the assignee's index to a MultiIndex than to flatten the assignor's MultiIndex.
I am completely clueless about what an implementation of the above might look like. Naively, it seems like the most apt formulation of panda's indexing in general is as a set of operations on tree structures where leaf nodes reference individual columns (or rows) of data. I would be very surprised if this wasn't how it was already implemented, actually. From the tree-based perspective, combining any two indexes -- single-level, multi-level; it doesn't matter -- is just a question of building an edge at the right spot between two trees. I can imagine an API for moving entire subtrees up and down index levels. But again, that's just me speculating about something I haven't actually tried to understand in detail.
TL;DR - Immutability across __setitem__ is the most important thing here. A close second is that the containing data structure's interface should be the one that changes, if any. In this specific case, the container's "single" index should be upgraded to a MultiIndex.

@brianthelion your model of how a MultiIndex works is not correct. Pls have a look at core/index.py for more information. A MI is basically a fancy list of equal length tuples.

@jreback, please don't let my poor understanding of the implementation details detract from concerns raised about the __setitem__ behavior. I think this issue is a point worthy of consideration by the devs insofar as it seems to violate a basic Python idiom. In the meantime I'll do my best to understand the current implementation and come back with something constructive.

I actually agree with @brianthelion here -- the lack of symmetry here is surprising.
From an implementation perspective, I believe this would be a matter of updating Index.reindex with a special case so that it returns a MultiIndex if it has a MultiIndex target: https://github.com/pydata/pandas/blob/35d0893edfc01b2257826480e17cbe0632a24de3/pandas/core/index.py#L1707
Note that:

Reindex is not a particularly efficient operation ever (all the values need to be copied).
Reindex already does not always preserve Index subtype (e.g., try inserting string column names into a frame with only integer column names).

So on the whole, I don't see much of an issue with returning a new MultiIndex rather than an Index from Index.reindex.

well, this is NOT a reindex operation at all. rather it is an align type of operation.
In [108]: df2.align(df1,level=0)
Out[108]:
(    a       b
     0   1   0   1
 0 NaN NaN NaN NaN
 1 NaN NaN NaN NaN
 2 NaN NaN NaN NaN
 3 NaN NaN NaN NaN
 4 NaN NaN NaN NaN,           a                   b
           0         1         0         1
 0  0.449161  0.977953  0.182016  0.215457
 1  0.050702  0.030895  0.233465  0.170092
 2  0.707819  0.455248  0.050476  0.953795
 3  0.708435  0.217373  0.508681  0.541064
 4  0.729345  0.935472  0.897516  0.776996)

However, it differs from this for a very important perspective, and the reasons this doesn't work like you think (and prob shoujd just raise instead).

there is not control on how to align, e.g. which level, so doing anything is ambiguous and surprising here, e.g. no level argument can be specified
changing the lhs of an expression from a less specialized object (Index) to a more specialized object (MultiIndex) is completely non-intuitive, and violates a host of principles
changing the left hand index in a way that would so easily break any further operations would be so surpising (e.g. now you have to index via tuples, rather than via a string)
pandas fundamentally tries to align the objects. when you are returning a NEW object, this is no problem. This is an inplace method and as such HAS to much more restricted on what it can change.
imagine that this was allowed to morph to a MultiIndex on the lhs. So now I have to go inspect every set to be sure that I am NOT assigning a MultiIndex? seems odd to me
note that this is similarly the case for say mixing a DatetimeIndex and an Object index (of say date strings). You don't get inference, merely concat.

so to me, this is by far the least surpising and practical thing to do. A MultiIndex is really just a tuple indexer. That is the most compatible object with an Index.

Thanks for explaining your thoughts here @jreback. I'm not 100% sure I agree with you -- in particular, I think we try to make all indexing operations with .loc that work on a tuple index (e.g., the original example A) also work with a MultiIndex in the same way (example B), though I'm sure there are edge cases that means this is not the case in practice. Also, from a certain perspective, you can actually think of a MultiIndex as a more general type of Index, not a more specialized subtype. That said... this sort of operation is inherently quite messy, so this is quite far down my list of consistency fixes for pandas.


Support returning dataloader from suspend function

Rudimentary implementation to support returning a CompletableFuture from a suspend fun.
CoroutinesUtils.invokeSuspendingFunction wraps the return value of the function in a Mono (or Flux). But it also does this when a CompletableFuture is returned, and thus results in a Mono<CompletableFuture<?>>, which isn't captured by graphql-java, and thus the dataloader is never dispatched.
By unwrapping the future, and converting it to a mono (as opposed to wrapping), the dataloader seems to be dispatched correctly.
Haven't added any tests yet, but if you also think this would be the right approach I can work on adding some tests as well.
Related issue: https://github.com/spring-projects/spring-graphql/issues/343

@rstoyanchev did you or anyone on the team had a chance to look at this?

Check https://github.com/spring-projects/spring-graphql/issues/393

Thanks for the ping and apologies for the delay.
Forgive the naive question, but just wondering why declare such a method that uses a DataLoader as a coroutine? It should already be non-blocking.
The implementation looks straight forward otherwise.

When first calling other suspending functions before loading data using the dataloader, e.g.;
We currently use this:
    @SchemaMapping
    fun transactions(
        model: Order,
        orderDataLoader: OrderDataLoader,
        paymentAccountDataLoader: PaymentAccountDataLoader,
        principal: Principal,
    ): CompletableFuture<List<OrderTransaction>> {
        val order = orderDataLoader.getFromCache(model.id)
        permissionValidator.validate(checkNotNull(order), principal, "admin-write")

        return CoroutineScope(Dispatchers.IO).future {
            orderPaymentService.findAllByOrderId(model.id)
        }.thenCompose { orderPayments ->
            val nonManualPayments = orderPayments.filterNot { it.isManualPayment }

            paymentAccountDataLoader
                .loadMany(nonManualPayments.mapNotNull { it.paymentAccountId })
                .also { paymentAccountDataLoader.dispatch() }

But when we can return a dataloader from the function it would look more like:
    @SchemaMapping
    suspend fun transactions(
        model: Order,
        orderDataLoader: OrderDataLoader,
        paymentAccountDataLoader: PaymentAccountDataLoader,
        principal: Principal,
    ): CompletableFuture<List<OrderTransaction>> {
        val order = orderDataLoader.getFromCache(model.id)
        permissionValidator.validate(checkNotNull(order), principal, "admin-write")

        val orderPayments = orderPaymentService.findAllByOrderId(model.id)
        val nonManualPayments = orderPayments.filterNot { it.isManualPayment }

        return paymentAccountDataLoader
            .loadMany(nonManualPayments.mapNotNull { it.paymentAccountId })
            // Not sure if manual dispatch would still be necessary here.
            .also { paymentAccountDataLoader.dispatch() }
    }

Thanks for clarifying.

@koenpunt Add kotlin-coroutines-jdk8 into your deps, and use future builder to convert it to a CompleteableFuture.

fun test() : CompletableFuture<List<OrderTransaction>>  = future{
}

That doesn't work, because future is a method on the coroutine scope.
This is valid code, but doesn't work because then we're back at the original problem:
suspend fun test(): CompletableFuture<List<OrderTransaction>> = withContext(Dispatchers.IO) {
   future {
      // ...
   }
}


GWindows.Application: Explorer_Path_At_Location does not work on Windows 11

The hack allowing to get the directory path from an Explorer window at an arbitrary (X, Y) location doesn't work anymore.
You can notice by running the sample (project: gwindows_samples.gpr) cap_test.adb.
Some keywords to explore a solution:
SHDocVw
ShellWindowsClass
path
NB: Ada generated code exists for SHDocVw, see tutorials\msie\ie.ads
Additionally the recommended method seen on Stack Overflow is to enumerate objects in the SHDocVw.ShellWindows collection; for each ie in that collection, match those with ie.FullName = "explorer" (after some treatement); grab the path via ie.LocationURL.

Solved in commit 9b95c6b6c399e33e268789cebd3873b6822be6c9


[fuchsia] JIT --no-link-platform causes crashes in flutter apps

This was uncovered by https://github.com/flutter/flutter/pull/44920#discussion_r346462657
More context at: https://github.com/flutter/flutter/issues/44724
This needs to be fixed ASAP.

Putting in December milestone since it says "needs to be fixed ASAP" in the description.

Merged in #47242


Unable to Buy when I change the Coin Pair

Whenever I change the Coin Pair from the Config, the amount doesn't change from BTC to the other Pair. I'm unable to pay from the USDT balance when I select USDT.

In line 80 you need to change the "asset" parameter from 'BTC' to coinPair (without ' ').

I'll fix this now. Thanks for spotting this.

Should be fixed now!


Fix accessible-name computation for table, fieldset, image input elements, and for “encapsulation” cases

This PR branch contains three commits:

e85812a64fe018f300611197536fdfdc6a7b1255 LibWeb: Fix accessible-name computation for “encapsulation” cases
This change makes Ladybird correctly handle all “encapsulation” tests in the https://wpt.fyi/results/accname/name/comp_host_language_label.html set of tests in WPT.
Those all test the requirement that when computing the accessible name for a <label>-ed form control, then any content (text content or attribute values) from the control itself that would otherwise be included in the accessible-name computation for it ancestor <label> must instead be skipped and not included.
The HTML-AAM spec seems to try to achieve that result by expressing specific steps for each particular type of form control. But what all that reduces/optimizes/simplifies down to is just, “skip over self”.
Otherwise, without this change, Ladybird includes that “self” content from those “encapsulated” elements when doing accessible-name computation for the elements — which results in AT users hearing unexpected extra content in the accessible names for those elements.
This change causes Ladybird to pass an additional 6 subtests in the comp_host_language_label.html set of tests at https://wpt.fyi/results/accname/name?product=ladybird.

20375dd3bb1e32d96b011d640e6cae049eb39180 LibWeb: Fix input@type=button|submit|reset accessible-name computation
This change makes Ladybird conform to the requirements in the HTML-AAM spec at https://w3c.github.io/html-aam/#accname-computation for the cases of HTML input@type=button, input@type=submit, and input@type=reset elements. Otherwise, without this change, Ladybird fails to expose the expected accessible names for those cases.
This change causes Ladybird to pass an additional 3 subtests in the comp_host_language_label.html set of tests at https://wpt.fyi/results/accname/name?product=ladybird.

48c639b0f4fd9c127305fac8fa31ff64e1bf6ef9 LibWeb: Fix accessible-name computation for table, fieldset, image input
This change makes Ladybird conform to the requirements in the HTML-AAM spec at https://w3c.github.io/html-aam/#accname-computation for the cases of HTML <table>, <fieldset>, and <input type=image> elements.  Otherwise, without this change, Ladybird fails to expose the expected accessible names for those cases.
This change causes Ladybird to pass an additional 4 subtests in the comp_host_language_label.html set of tests at https://wpt.fyi/results/accname/name?product=ladybird.

So these two changes together cause Ladybird to pass all 78 tests in the comp_host_language_label.html set of tests at https://wpt.fyi/results/accname/name?product=ladybird.

This has conflicts.

This has conflicts.

Thanks for the heads-up — fixed now ✅


feat(#2): add messy implementation for futures ws

Excuse the mess as this is just a very rough MVP/POC to get the futures WS hooked up. Not sure I'll manage to clean it up properly, but hopefully it's helpful should you wish to adopt it in a cleaner way as the pieces are all mostly functional.
Summary:

ping/pong parsing/messaging needed changing, as the format is different for this ws
request keys needed changing as they're not camel case
the signature mechanism needed changing as it's slightly different
the parsers need changing as they're not returning the same data format

Feel free to close this if you don't need this PR, I just wanted it noted down if it's a time saver in future! Thanks for the guidance you gave!

Hi @tiagosiebler, thanks for the PR. It will definitely save me some time in the future! I really appreciate it


Taught deabstraction and function inline infra to dump under debug log CSV-formatted events for code size measurement.

The initial code size metric of a function is defined by the number of SIL
instructions in the function.
Under a newly added compiler flag -Xllvm -tf-log-deabstraction-stats, we dump
CSV-formatted events for 2 tables.
Table schemas
Table 1: Function size.
The table schema is: "S4TF FuncSize", FuncName, Size, StageName
Here StageName can be one of {"BeforeInline", "AfterInline", "AfterDA"}
This gives us the code size impact of doing inlining and running the
deabstraction pass over SIL functions.
Table 2: Call graph information used by function inliner
Table schema is: "S4TF CallEdge", ParentName, ParentSizeBefore, ParentSizeAfter, ChildName, ChildSize
Each time when a function child() gets inlined into another function
parent(), we write such a record, where we log the parent size before and
after the inlining, and the child size.
Analysis workflow
Based on the above tables, we can do SQL based analysis, similar to
https://github.com/apple/swift/pull/20558.
For example, to analyze table #2 above, first dump the CSV log via a command like:
$ time ../build/$rdir/swift-linux-x86_64/bin/swiftc -frontend -emit-sil -O -Xllvm -tf-dump-intermediates -Xllvm -tf-log-deabstraction-stats ResNet/main_perf.swift >/tmp/raw_log3 2>&1 && grep 'S4TF CallEdge' /tmp/raw_log3 > /tmp/call.log3

Next, run sqlite3 profile.db. In the sqlite> prompt, do the following:
drop table c;

CREATE TABLE c(
  "S4TF" TEXT,
  "ParentName" TEXT,
  "ParentSizeBefore" INTEGER,
  "ParentSizeAfter" INTEGER,
  "ChildName" TEXT,
  "ChildSize" INTEGER,
  "CallDepth" INTEGER
);

.separator ","
-- call.log stores all CSV events labelled "S4TF CallEdge"
.import /tmp/call.log c

select count(*) from c;

Some example SQL queries for analysis:
-- Which child functions increase code size most after being inlined.
-- Here we filter out functions that are online inlined once, since such
-- inlining would not increase code size, as long as the child function gets
-- deleted after inlining.
SELECT ChildName, COUNT(*) cnt, SUM(ChildSize)
FROM c
GROUP BY 1
HAVING cnt > 1
ORDER BY 3 DESC
LIMIT 10;

-- Find the top K largest children that get inlined into a particular parent
SELECT MIN(ParentSizeBefore), MAX(ParentSizeAfter), ChildName, COUNT(*), SUM(ChildSize)
FROM c
WHERE ParentName = '$s9main_perf3run33_01E2ABC548C103A0BD150EB87B3A1DADLLyyF'
GROUP BY 3
ORDER BY 5 DESC;

Use case study
Based on the above analysis, we are able to figure out which functions in a
resnet50 model contributed most to code size bloat (and compile time increase)
because of them getting inlined. After marking them with @inline(never), the
compilation cost dropped from 2 minutes to 3 seconds.

@swift-ci please test tensorflow

This is another candidate to upstream later.

Thanks for the review!


Feature request: Doubletap in notes mode to toggle gates

Performer + launchpad is such a killer combo. But there is one thing thing that I would really wish for: The possibility to toggling gates on/off in notes mode via a double press (or a long press) on a pad.
When editing a melodic sequence it is super convenient that one can pick 8 notes in scale or semitones. However, it is quite cumbersome to have to switch to gate mode, toggle gate status, return to note mode and set a note in order to make a small change to the rythm. It would be so much more intuitive to simply double tapping a note to change the gate value on/off.
It is worth considering if long pressing one or more notes simultaneously could be used to make a tie or something. But adding some flexibility in how notes and gates can be edited on the notes page would make editing significantly faster and more intuitive.
Note: I also left this comment in the thread about follow mode, but toggling gates on/off in note more is really a separate request, so I thought I would give it its own thread as well.

Double press for toggling gates will be part of 0.1.38 release.


fixed Remediate package vulnerability in grc-ui-api

Signed-off-by: Chunxi Luo<EMAIL_ADDRESS>

open-cluster-management/backlog#11541

/lgtm


⚠ Quirk Club Argentina (API endpoint) has degraded performance

In f684b41, Quirk Club Argentina (API endpoint) (https://us-central1-quirkclub-dev.cloudfunctions.net/api/check/api) experienced degraded performance:

HTTP code: 200
Response time: 8889 ms

Resolved: Quirk Club Argentina (API endpoint) performance has improved in 0cbb011 after 19 minutes.


Grover/Gallium Nine Support?

Considering the fact that there is a driver that would add native glide support for mesa, wouldn't it be purely better to use that as well? I mean, the code is done and useable. Just needed to be added to mesa, but was unfortunately rejected.

Yes, that would be really an amazing thing to have native Glide support in SoftGPU via Mesa.
The same is true for Gallium Nine aka the native Direct3D9 API implementation in Mesa. :wink: As far I known QEMU has support for it. In comparison to WineD3D it is definitely faster, more accurate and even more stable. And unlike the Grover Project Gallium Nine is already part of Mesa since several years. :+1:
And by the way, the Grover merge request in Mesa was not rejected. It looks that its original author has simply no longer any time free to work on it. Ergo it is still in draft stage. More information about that project can be found at the original phoronix article.

Yes, that would be really an amazing thing to have native Glide support in SoftGPU via Mesa.
The same is true for Gallium Nine aka the native Direct3D9 API implementation in Mesa. 😉 As far I known QEMU has support for it. In comparison to WineD3D it is definitely faster, more accurate and even more stable. And unlike the Grover Project Gallium Nine is already part of Mesa since several years. 👍
And by the way, the Grover merge request in Mesa was not rejected. It looks that its original author has simply no longer any time free to work on it. Ergo it is still in draft stage. More information about that project can be found at the original phoronix article.

this is quite interesting to learn. In this case then, it appears that using gallium nine and grover would be far better than using wine libraries. Especially since it would both be native support in those cases.

Hi, Glover doesn’t seem to me to be in some complete state – but I currently using mainline version of Mesa, so it can be merged relatively easily (question is Rust runtime on Windows 9x – this could be quite nightmare).
Nine is something different – I already created skeleton to implementation. In last months I heavily worked on GPU gen. 10 implementation and now I can focus to Nine. Plan is also route DDI calls directly to Nine – result will be native DirectX/DirectDraw implementation without wrappers. Driver side skeleton is already here. But I don't want to promise anything yet, it's possible that I'll “hit the wall” somewhere.

Great news and nice to know that native Direct3D support will be available at some point! :tada:
By the way regarding the Grover project it looks that I was wrong and @michaeljhkim was right. Current state is that it got in some way a NAK which means that it will not land. :disappointed:  I have asked regarding the actual situation and one of the Mesa devs gave an informative answer what Mesa can support and what not, I quote Karol Herbst from here:

there are three parts of to that:
usefulness: is having an upstream glide frontend actually useful or is it only done for the sake of it? Like does it allow any user to do something which they couldn't do otherwise? Is it giving any performance advantages? tldr: is it critical to have this inside mesa the way it is?
cost of maintenance: Are the potential benefits a good trade of for the increased maintenance burden as of today? Does it makes sense to add Rust abstraction layers just for this frontend?
viable alternatives: Does it really have to be a real gallium frontend or could it also be an external library simply layered on top of OpenGL/Vulkan?
The Nine frontend was a product of its time to bring up performant gaming to Linux, there was no viable alternative as the OpenGL translation layer in wine was generally significantly slower. d3d10 support in wine was very poor. These days we have d3d9 layering on vulkan being equally fast if not faster, so if anybody would propose a new frontend like Nine it might also get rejected due to the diminishing benefits. The only real reason to keep nine is that it does run d3d9 era games on d3d9 era GPUs not supporting Vulkan. But the question remains for how long it still makes sense to keep maintaining it.

So instead of closing this topic here it would perhaps make sense to rename it from Grover to Gallium Nine support. :thinking:

Great news and nice to know that native Direct3D support will be available at some point! 🎉
By the way regarding the Grover project it looks that I was wrong and @michaeljhkim was right. Current state is that it got in some way a NAK which means that it will not land. 😞 I have asked regarding the actual situation and one of the Mesa devs gave an informative answer what Mesa can support and what not, I quote Karol Herbst from here:

there are three parts of to that:
usefulness: is having an upstream glide frontend actually useful or is it only done for the sake of it? Like does it allow any user to do something which they couldn't do otherwise? Is it giving any performance advantages? tldr: is it critical to have this inside mesa the way it is?
cost of maintenance: Are the potential benefits a good trade of for the increased maintenance burden as of today? Does it makes sense to add Rust abstraction layers just for this frontend?
viable alternatives: Does it really have to be a real gallium frontend or could it also be an external library simply layered on top of OpenGL/Vulkan?
The Nine frontend was a product of its time to bring up performant gaming to Linux, there was no viable alternative as the OpenGL translation layer in wine was generally significantly slower. d3d10 support in wine was very poor. These days we have d3d9 layering on vulkan being equally fast if not faster, so if anybody would propose a new frontend like Nine it might also get rejected due to the diminishing benefits. The only real reason to keep nine is that it does run d3d9 era games on d3d9 era GPUs not supporting Vulkan. But the question remains for how long it still makes sense to keep maintaining it.

So instead of closing this topic here it would perhaps make sense to rename it from Grover to Gallium Nine support. 🤔
Edit: The discussion on Mesa around Grover made an interesting development. The mentioned NAK was not a general one. It was more related to the current implementation in Rust. So it looks that there may exist a chance that a Glide API Mesa implementation would be accepted but it has to be done most likely in C. 👍

I mean, the Grover code still exists for anyone to use freely, so I do not see a major issue in trying to get things working unless it just doesn't work to begin with

Yes, a rewrite of Grover to C is most likely a huge amount of work. But I suppose that for this project here that would be a clear win because the implementation in SoftGPU would be far easier when it's C based instead of Rust.

Yes, a rewrite of Grover to C is most likely a huge amount of work. But I suppose that for this project here that would be a clear win because the implementation in SoftGPU would be far easier when it's C based instead of Rust.

the only problem now is to figure out who is going to do it? because it would be unfair to give another work load onto the author. Gah, if only there were more people interested in this project and had skills.

This is true, - it looks that one main goal of the Grover MR was the idea to use Rust. So most likely a rewrite to C would be a project of its own. And when I compare it to Gallium Nine, then the whole context seems to be a different one because Gallium Nine is centrally focused on Gallium although its uses the old TGSI and not the newer NIR technique as base.
Whatever, one positive aspect is probably that the Glide API is far smaller than the Direct3D9 one. So the total amount of that work should be less. And, finally the Glide API is open-source and the way how it works is therefore better documented than Direct3D9 (which needed back quite some reverse-engineering). There exist still the original SourceForge page as well as some minor updated forks at GitHub.
So yeah, this should be done but the question is who has the motivation for it. Maybe someone will come along the road and try this a second time but in a way that it can be accepted by the Mesa devs. :wink:
I assume that @linkmauve is no longer interested in that.

@Torinde: Another DDraw/DX6/DX7 wrapper in not plan now, but I'm planning to add native D3D HAL support, but is long way to go...

https://www.phoronix.com/news/Mesa-Gallium3D-Nine-Sunset


Adding release scripts

Adding release scripts and automatic changelog generation.

Would you like to add the ./scripts/release script to this branch or in a separate one?

@chrissrogers the scripts/release script was already present in this repository.


Add 3.10.0a1

See https://pythoninsider.blogspot.com/2020/10/python-390-is-now-available-and-you-can.html

GitHub Actions is having a bit of a flake (https://www.githubstatus.com/incidents/t34640ccrmfs), but the 3.10 builds are all green :innocent:


npm shrinkwrap causes issues on setup

Every time I setup react_on_rails on my machine with the generators, the npm-shrinkwrap file breaks my setup.
I cd into client, run npm install successfully. Then I cd up to the root and run foreman start -f Procfile.dev. Then it gives me an error involving the 'body-parser' module.
I solve this by deleting the npm-shrinkwrap file and re running npm install in the client directory.

Most likely the issue is using the same version of npm and node that generated the shrinkwrap file.
@robwise We should make sure the generator installs the package.json, but runs the command to shrinkwrap.

Sounds good.


[WIP] Breaking invoice payment

Developer(s): @alex-galych

Please confirm

[ ] No new OS components - or they have been approved by legal department

Documentation

[ ] Functional documentation provided or in progress?
[ ] Integration guide for projects provided or not needed?
[ ] Migration guides for all contained majors provided or not needed?

Release Table

Module
Release Type
Constraint Updates

Heidelpay
patch

Release Notes
{skip}

Module Heidelpay
Change log
Bugfixes

Added adapterRequestFromOrderBuilderCollection to the createFinalizeTransactionHandler in order to handle different request payload for Invoice and EasyCredit payment methods.

Tested with only invoiceSecured payment method. Finalize process works fine.

Could you please also check that Easycredit not broken by this fix.

Tested module with only EasyCredit payment method enabled. Works fine.


fix new line insertion introduced in #73

The change introduced in #73 produces an invalid output (at least on Ubuntu 16.04 LTS) :

This happens because echo doesn't interpret backslash escapes by default on some distros, it only does so with the -e flags, which doesn't have a consistent behavior.
Using printf solves that and makes sure that the newline characters are outputed correctly.
ref: http://stackoverflow.com/questions/8467424/echo-newline-in-bash-prints-literal-n/8467449

Per usual, @MrMitch to the rescue. Thanks!

I actually caught this one on pure luck because i was installing desk on a fresh Ubuntu 16.04 ;)


Change README.md to correctly show --xdelta options

The current README.md uses --xpatch instead of --xdelta, this is in error.

Good catch, thanks!


os: IsDir doesn't work with Windows OneDrive

I think there is an issue with FileInfo.IsDir in Windows 10 using the new Files On-Demand feature of OneDrive. (This occurs only after the Fall Creator Update that introduce this feature).
If I create a folder in OneDrive (with the Files-On-Demand feature on) IsDir return false. Note that just after the creation it return true but once OneDrive finish its sync it then return false. Also if I deactivate the Files-On-Demand feature I no longer repro (for newly created folder, the old one still have the issue). Using the "Always keep on the device option" or the "Free up space" doesn't change the repro.
Also I may miss something obvious, I never used Go before but I found this issue while debugging and issue with Hugo (a site generator in Go).
What version of Go are you using (go version)?
go version go1.9.2 windows/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\theo\go
set GORACE=
set GOROOT=C:\Go
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config

My OneDrive version is: Version 2017 (Build 17.3.7105.1024)
What did you do?
package main

import (
	"fmt"
	"os"
)

func main() {
	src := "C:\\Users\\theo\\folder"
	srcO := "C:\\Users\\theo\\OneDrive\\folder"
	fi, err := os.Stat(src)
	fmt.Println(fi.IsDir())
	fmt.Println(err)

	fiO, errO := os.Stat(srcO)
	fmt.Println(fiO.IsDir())
	fmt.Println(errO)
}

What did you expect to see?
true
<nil>
true
<nil>

What did you see instead?
true
<nil>
**false**
<nil>

What does fi0.Mode().String() print?

/cc @alexbrainman

Thanks for the quick answers.
fmt.Println(fiO.Mode().String()) returns Lrw-rw-rw-
It seems that the issue is here: https://github.com/golang/go/blob/936b977c174dd66348bacd1392f761ae518ca7c1/src/os/types_windows.go#L41
I'm not sure why OneDrive folders became symlinks so maybe that's not a bug. But this whole behavior breaks assumptions in other programs so that's kind of an issue.

Or this condition shouldn't be a return but just a m |= ModeSymlink.
But in that case that contradict this old fix (https://github.com/golang/go/commit/1989921aef60c83e6f9127a8448fb5ede10e9acc#diff-befffd222e4e39277f81cd4cb7116a0b).
The fact that OneDrive transform stuff in symlinks while probably cause lots of weird issues.

IMHO the ideal situation is that the standard library treats symlinks as transparently as possible, because otherwise many apps doing IO will have to write special code in order to work correctly with symlinks (but since this is subtle, it would probably mean that in practice apps written in Go wouldn't work correctly with symlinks).
While the change introducing this issue was in response to some valid bugs, it would seem preferable that the specific cases where symlinks should not be handled transparently should explicitly check for it (e.g. when doing recursive traversal).

Users of the Arduino IDE are also reporting problems with OneDrive files not being read by the arduino-builder tool, which is written in go. We're tracking the issue at https://github.com/arduino/arduino-builder/issues/254 The problem there is ReadLink failing, which is (probably) different from this issue, but I think the underlying cause might be the same.

I'm not sure why OneDrive folders became symlinks so maybe that's not a bug.

I believe the OneDrive files/folders are not symlinks, but they are "reparse points" (and the most common types of reparse points seem to be symlinks and junctions, both kinds of links that ReadLink now supports explicitely). This  post suggests they are "Cloud reparse points", though I'm not sure what that means exactly.
Perhaps this provides some starting points for fixing this (and perhaps also ReadLink)?

Interesting.. If they are not true symlinks, then it would seem that would strengthen the case that the IO library should treat them as files transparently. Especially if there are guarantees that "cloud reparse points" will always behave just as files do (in the sense that they're not going to cause loops while listing directory contents, etc.).

Users of the Arduino IDE are also reporting problems with OneDrive files not being read by the arduino-builder tool, which is written in go. We're tracking the issue at arduino/arduino-builder#254 The problem there is ReadLink failing, which is (probably) different from this issue, but I think the underlying cause might be the same.

@matthijskooijman
Looking at your bug arduino/arduino-builder#254

readlink C:\Users\HaMac\OneDrive\Documents\Ardunio\BlinkTwoLEDs\BlinkTwoLEDs.ino: The system cannot find the file specified.

I would agree that it, probably, is different problem. This current issue is about os IsDir. While yours is os.Readlink. They might be related, but they different problems.
I suggest you file a new bug with small reproducible example. Mind you, os.Readlink is broken on Windows - there is no way to implement it properly on Windows. I suggest you use filepath.EvalSymlinks instead. Would that work for you?
Alex
PS: I still do not have Windows with "Fall Creator Update" to debug this.

I suggest you file a new bug with small reproducible example.

Yeah, I was hoping to do that, but no time at all right now. I was mostly thinking that the info would be useful for this bug as well.

Mind you, os.Readlink is broken on Windows - there is no way to implement it properly on Windows. I suggest you use filepath.EvalSymlinks instead. Would that work for you?

IIRC we're using Filepath.Walk which apparently uses ReadLink internally. I'm not 100% sure yet if thats's also the case here, but it was in a previous bug with the same symptom.

IIRC we're using Filepath.Walk which apparently uses ReadLink internally.

I don't see that filepath.Walk uses os.Readlink. But I have been wrong before. That is why I suggest you create a new issue with small reproducible example, if you have one.
Alex

I don't see that filepath.Walk uses os.Readlink.

I've also looked, and couldn't find this either (perhaps it was a wrong conclusion before). I couldn't actually find any path to os.Readlink, so I'm not quite sure what happens yet exactly, so haven't been able to create a small example yet.

@ttrunck I finally managed to update my Windows 10 to correct version, and I can reproduce something similar to your bug.
I suspect the problem is that we don't follow advise from https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/ to the letter. In particular, it says: "... If you use the FindFirstFile function, you can tell that you have a symbolic link because the file attributes will have the FILE_ATTRIBUTES_REPARSE_POINT flag set, and the dwReserved0 member will contain the special value IO_REPARSE_TAG_SYMLINK. ...". We do check the FILE_ATTRIBUTES_REPARSE_POINT flag set, but we do not check dwReserved0. We should rectify this problem, but only once go1.10 is released, because I suspect the change might be involved. We need to fix everything: os.Stat and os.File.Stat and os.Lstat and os.Readdir.
Alex

Thanks, so it's more involved than just not breaking early in types_windows.Mode
So this should be fixed in go1.11 and the timeline is mid 2018 right? For now my workaround is to not use that OneDrive feature or move my projects outside of OneDrive. I also think Docker is hitting this issue but I don't know how I can easily confirm that it's indeed the same issue.
Thanks for looking at it!

I punted this to Go 1.11 because old bugs (ones that aren't recent regressions) always get punted. But if this is suddenly more prominent because of OneDrive, we could consider a fix for Go 1.10, if somebody could prepare one soon that checks dwReserved0.

@alexbrainman rather than check for specific reparse tags, I believe the correct method is to test for the presense of the reparse tag surrogate bit. The surrogate bit in the reparse tag specifies if "the file or directory represents another named entity in the system". I have manually verified that the surrogate bit is set only for reparse points that have symlink-like semantics:

PATH
REPARSE TAG
IsReparseTagNameSurrogate

D:\symlink
10100000000000000000000000001100
true

D:\symlink_dir
10100000000000000000000000001100
true

D:\junction
10100000000000000000000000000011
true

D:\mountpoint
10100000000000000000000000000011
true

D:\OneDrive
10010000000000000111000000011010
false

D:\hardlink
hard links do not use reparse points
and they don't act like symlinks

Additionally, Microsoft has indicated that the IsReparseTagNameSurrogate macro is the correct way to determine if a reparse point behaves like a symlink.
(I copied this information over from #23684 since that issue was essentially a duplicate of this one)

@mattico I suspect this change https://golang.org/cl/86556 will fix your problem. Can you, please, check?
But I agree, perhaps, we should use IsReparseTagNameSurrogate to decide if file is symlink or not. CL 86556 checks for IO_REPARSE_TAG_SYMLINK and IO_REPARSE_TAG_MOUNT_POINT. This might be helpful to solve problem listed in #23512
Alex
For my own record:
winnt.h:1760:#define IsReparseTagNameSurrogate(x) ((x)&0x20000000)

But I agree, perhaps, we should use IsReparseTagNameSurrogate to decide if file is symlink or not.

@alexbrainman So, basically, something like this:
diff --git a/src/os/types_windows.go b/src/os/types_windows.go
index f3297c0338..6668d911fc 100644
--- a/src/os/types_windows.go
+++ b/src/os/types_windows.go
@@ -140,14 +140,14 @@ func (fs *fileStat) updatePathAndName(name string) error {
 }

 func (fs *fileStat) isSymlink() bool {
+       const reparsePointNameSurrogate = 0x20000000
        // Use instructions described at
        // https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
        // to recognize whether it's a symlink.
        if fs.FileAttributes&syscall.FILE_ATTRIBUTE_REPARSE_POINT == 0 {
                return false
        }
-       return fs.Reserved0 == syscall.IO_REPARSE_TAG_SYMLINK ||
-               fs.Reserved0 == windows.IO_REPARSE_TAG_MOUNT_POINT
+       return fs.Reserved0&reparsePointNameSurrogate != 0
 }

 func (fs *fileStat) Size() int64 {

would be good to have.

@alexbrainman So, basically, something like this:

Yes. Perhaps more comments are needed to explain the 0x20000000.

would be good to have.

I don't see anything wrong with existing code - the code has comment explaining why it does what it does. What do you think the code needs updating? Do you have an issue that is affected by this code?
Alex

What do you think the code needs updating?

@alexbrainman per your own earlier comment: https://github.com/golang/go/issues/22579#issuecomment-362870929 (also see comments above that one).

Do you have an issue that is affected by this code?

We had an issue with Docker on Windows with deduplication feature turned on (https://github.com/moby/moby/issues/37026) which was solved (in https://github.com/moby/moby/pull/37769) in a way very similar to your commit e83601b4356f92f2, but with the final check for is IsReparseTagNameSurrogate (i.e. same as in my example patch above).
Perhaps both ways to check are equally correct for now, but it seems that the check for IsReparseTagNameSurrogate might be more future-proof.

We had an issue with Docker on Windows with deduplication feature turned on (moby/moby#37026) which was solved (in moby/moby#37769) in a way very similar to your commit e83601b, but with the final check for is IsReparseTagNameSurrogate (i.e. same as in my example patch above).

Sounds reasonable. But, please, create new issue first at https://golang.org/issues/new with simple repro so we have some history about why we made the change. Please, also describe the environment needed to be able to reproduce this issue.
Thank you.
Alex

@kolyshkin also I recently fixed os.SameFile - it was completely broken when os.SameFile parameters were symlinks - 1c95d9728aa . Maybe it will fix your issue.
Alex

@alexbrainman it looks like you oversimplified things too much (maybe because you assumed that FILE_ATTRIBUTE_REPARSE_POINT can be set only for either symlinks or directories -- in addition to these two cases I know it can also be set on files residing on volumes with data deduplication turned on, see https://github.com/moby/moby/pull/37769 and also "steps to reproduce" from https://github.com/moby/moby/issues/37026). With this in mind, the check

func (fs *fileStat) isSymlink() bool {
return fs.FileAttributes&syscall.FILE_ATTRIBUTE_REPARSE_POINT != 0

is not sufficient, you'll get false positives (isSymlink() returning true for file systems objects that are not "symlinks".
The correct check should look something like
func (fs *fileStat) isSymlink() bool {
	const reparsePointNameSurrogate = 0x20000000 // TODO document this
	return fs.FileAttributes&syscall.FILE_ATTRIBUTE_REPARSE_POINT != 0 &&
		fs.Reserved0&reparsePointNameSurrogate != 0
}

Of course, Reserved0 should be populated from syscall.FindFirstFile by newFileStatFromWin32finddata, which your patch removed.
PS all this is theorizing based on the code I've seen; I don't even have a Windows system to try things out.

Both moby/moby#37769 and moby/moby#37026 describe repro for a bug in Docker

@alexbrainman ... which is caused merely by incorrect identification of symlinks on Windows (which is fixed in Go 1.11 by your commit https://github.com/golang/go/commit/e83601b). I'm afraid it's going to be broken again.
You don't have to dig into Docker source code, or to use Docker in order to reproduce. If you can take another look at https://github.com/moby/moby/pull/37769 description, in particular "How to verify" section, you'll find the steps to enable deduplication. Once enabled, the test to perform (without using Docker) is to iterate through the files, doing something like this for each one:
	fi, err := os.Lstat(path)
	if err != nil {
		return err
	}

 	if fi.Mode()&os.ModeSymlink != 0 {
 		link, err := os.Readlink(path)
 		if err != nil {
 			return err
 		}
 	}

As you can see, the code assumes if a file is a symlink, os.Readlink() can be used.
Previously (before commit https://github.com/golang/go/commit/e83601b) the above code failed on deduplicated files with the error like this:
readlink \?\D:\temp\file: The system cannot find the file specified.

From what I see, this bug might come back with the change in https://go-review.googlesource.com/c/go/+/143578. Ideally, of course, there should be a unit test added to check for this specific issue (treating dedup'ed files like symlinks).
I apologize for not making things clear earlier, and hope this explanation makes sense. This is as close as I can get to presenting a simple reproducer without actual access to a Windows machine.

@alexbrainman ... which is caused merely by incorrect identification of symlinks on Windows (which is fixed in Go 1.11 by your commit e83601b). I'm afraid it's going to be broken again.

Maybe yes, maybe not. We won't know that until we apply CL 143578 to your test case. If you give me your test case, I will try it, if I can. If you cannot provide me with a test case, I cannot even try. Feel free to try CL 143578 yourself, and report results back here. I will happy to help with instructions, if you need any.

iterate through the files, doing something like this for each one:

This is to vague for a test. Please use https://github.com/golang/go/issues/new and answer all questions there.

os.Readlink() can be used.

os.Readlink is broken on Windows in many situations. You should not be using it in your code.

Ideally, of course, there should be a unit test added to check for this specific issue (treating dedup'ed files like symlinks).

I agree, it might be difficult to have permanent test. But I would like to try and reproduce it myself at least somewhere.

This is as close as I can get to presenting a simple reproducer without actual access to a Windows machine.

Unfortunately this is not enough for me for a repro.
Maybe you should try and explain what your are trying to do? Perhaps I might suggest the code that will work for you.
Alex

Maybe you should try and explain what your are trying to do? Perhaps I might suggest the code that will work for you.

The code in question adds a file to a tar; the relevant part is creating a tar entry header. This is the code (
https://github.com/moby/moby/blob/bc4c1c238b551ce58fc26690523354d1571bab1d/pkg/archive/archive.go#L456-L475):
// addTarFile adds to the tar archive a file from `path` as `name`
func (ta *tarAppender) addTarFile(path, name string) error {
	fi, err := os.Lstat(path)
	if err != nil {
		return err
	}

	var link string
	if fi.Mode()&os.ModeSymlink != 0 {
		var err error
		link, err = os.Readlink(path)
		if err != nil {
			return err
		}
	}

	hdr, err := FileInfoHeader(name, fi, link)
	if err != nil {
		return err
	}

On a system with deduplication turned on and before commit https://github.com/golang/go/commit/e83601b (and I suspect after CL 143578), this leads to
readlink \\?\D:\temp\dedupme\CbsApi.dll: The system cannot find the file specified.

As you can see, the code assumes that if os.ModeSymlink bit is set in s.Mode() returned by os.Lstat(), then os.Readlink() can be used. Is this assumption wrong?
You say that

os.Readlink is broken on Windows in many situations. You should not be using it in your code.

Can you please elaborate (and/or propose a replacement for the above code)?

@kolyshkin I played with files on deduped volume. I am happy to report that, you are correct - all files on deduped volume looks like files now, because Windows reports then with both FILE_ATTRIBUTE_REPARSE_POINT and IO_REPARSE_TAG_DEDUP set. So after my CL 143578 - which only checks for FILE_ATTRIBUTE_REPARSE_POINT - they will be interpreted as symlinks. I suppose you are saying they should be plain files, not symlinks, and I cannot go ahead with CL 143578 unless I find way to check for IO_REPARSE_TAG*. Fair enough.
Alex

@alexbrainman glad to be of help.
Checking IO_REPARSE_TAG_* is not a good idea though, as it's not future-proof (say MS adds some other fancy feature and a flag, and then your code needs to be updated yet again to account for this new flag).
It still seems to me the correct check is the one described in https://github.com/golang/go/issues/22579#issuecomment-432015264, ie the check for reparsePointNameSurrogate (0x20000000) in Reserved0 field, populated from syscall.FindFirstFile by newFileStatFromWin32finddata. This is very similar to what the code in Go 1.11 does. I could create a CL but need to go through the horrors of setting up Windows development environment first

It still seems to me the correct check is the one described in #22579 (comment), ie the check for reparsePointNameSurrogate (0x20000000) in Reserved0 field,

One thing at a time. I am trying to fix #27225 and #27515 at this moment.
Alex

@kolyshkin
I adjusted CL 143578 for my yesterday's findings. I hope after CL 143578 deduped volumes will work as good / bad as they do now.
Alex

@alexbrainman thank you Alex, it looks like CL 143578 (patch set 3) is good, I have left my review comments right in there (should have done it in the first place, sorry for abusing github).


refactor(es): convert requires to imports

Converted requires to imports for modules updated in ngageoint/opensphere/pull/1363. This was fully scripted.
The build for this will fail until the OpenSphere PR is merged.

:tada: This PR is included in version 1.7.9 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


WindowOperatorQueryFrameProcessor: Fix frame writer capacity issues + adhere to FrameProcessor's contract

Description
This PR handles 2 things:
1. WindowOperatorQueryFrameProcessor bug causing incomplete output data
Currently, we had a bug in the WindowOperatorQueryFrameProcessor where the frame writer's capacity could get reached for larger queries, causing it to not output all of the result rows.
For example, the following query gives incomplete output rows when we use maxNumTasks=2, compared to when we use more workers.
select trip_id, row_number() over(partition by trip_id) as c1 from "trips_xaa" where __time < TIMESTAMP '2013-09-20 11:15:25' group by trip_id
-- This gives 20479 rows with maxNumTasks=2
-- This gives 30341 rows with maxNumTasks=5
-- This gives 30341 rows with maxNumTasks=11

This PR fixes the above issue by maintaining the state of last rowId flushed to output channel, and triggering another iteration of runIncrementally() method if frame writer has rows pending flush to the output channel.
The above is done keeping in mind FrameProcessor's contract which enforces that we should write only a single frame to each output channel in any given iteration of runIncrementally().
For manual testing, I've been verifying the behavior with queries like the following:
select trip_id, row_number() over(partition by trip_id) as c1 from "trips_xaa" where __time < TIMESTAMP '2016-10-20 11:15:25' group by trip_id
-- 49795247 rows inputted, 49795247 rows outputted by window stage (with maxNumTasks=2 and maxNumTasks=11)

select c1, count(c1) from (select trip_id, row_number() over(partition by trip_id) as c1 from "trips_xaa" where __time < TIMESTAMP '2016-10-20 11:15:25' group by trip_id) group by c1
-- [1, 49795247] with maxNumTasks=2 and maxNumTasks=11

Additionally, I have added a test WindowOperatorQueryFrameProcessorTest#testFrameWriterReachingCapacity() which was previously writing less number of rows to the output channel, but is writing the complete set of rows.
2. Optimization to avoid unnecessary re-runs of runIncrementally()
This PR also optimizes the scenarios where we had all window functions with some partition by clause. Currently, we were triggering another run of runIncrementally() on every partition by change. But this is unnecessary overhead. This PR changes the behavior to only trigger a re-run when we write a frame to the output channel.
For example, the below query that processes almost 50M rows was run with maxNumTasks=2 against the current master code vs this PR's code.
select c1, count(c1) from (select trip_id, row_number() over(partition by trip_id) as c1 from "trips_xaa" where __time < TIMESTAMP '2016-10-20 11:15:25' group by trip_id) group by c1

We can see in the following screenshots that the window stage took 5:37 minutes in the current master code compared to 35 seconds with this PR's code, making the overall query time to go down from 23 minutes to 18 minutes.

Current master code
This PR's code

This PR has:

[x] been self-reviewed.
[ ] added documentation for new or modified features or behaviors.
[ ] a release note entry in the PR description.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[ ] been tested in a test Druid cluster.

this seems like a bugfix - please remove the optimization; that should be reviewed separetly

this seems like a bugfix - please remove the optimization; that should be reviewed separetly

@kgyrtkirk I think we anyway should get the optimization also merged for Druid 31 (since it's significant), hence bundled it together with this patch because it was already updating a lot of the frame writer logic.
If it helps, this part of the diff corresponds to the optimization:

remove the performance enhancement ; and place it in a separate PR - also add a benchmark to help keep track of performance improvements

remove the performance enhancement ; and place it in a separate PR - also add a benchmark to help keep track of performance improvements

@kgyrtkirk I was hoping to keep all changes related to frame writer together, to ensure that everything works fine with both of them together. Instead of a bug fix PR, I'm looking at this PR as more of a "revamp frame writer logic in WindowOperatorQueryFrameProcessor" PR.
Splitting it into multiple PRs is certainly doable, but I'd still have to continue testing them coupled together locally, which is unnecessary and would cause unnecessary delays (we are hoping to get this in for Druid 31 release).
Regarding the benchmark - I agree that it would be great to have it. It's on my list of todos, but it's not a trivial task and would require some time. So until then, benchmark shouldn't block any performance improvements / logic changes.
Thoughts? cc: @cryptoe

what's happening in this PR right now is pretty convoluted - that's why I'm asking to remove any performance enhancements ; because I don't think the fix for the bug is right

@kgyrtkirk Fair enough, working on splitting up the PR.

@kgyrtkirk Have removed the optimization diff from this PR. Have opened a separate PR for it: https://github.com/apache/druid/pull/17211, appreciate your reviews on that PR as well. Thanks!


layer2 protocol on k8s 1.10 gives warning

Is this a bug report or a feature request?:
Bug
What happened:
Everything works (I'm able to reach the services via the EXTERNAL-IP but the Service object contains a warning:
$ kubectl describe svc hello-service -n test-namespace
Name:                     hello-service
Namespace:                test-namespace
Labels:                   <none>
Annotations:              kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"hello-service","namespace":"test-namespace"},"spec":{"ports":[{"name":"http","port":80,"...
Selector:                 app=hello
Type:                     LoadBalancer
IP:                       <IP_ADDRESS>
LoadBalancer Ingress:     <IP_ADDRESS>
Port:                     http  80/TCP
TargetPort:               8080/TCP
NodePort:                 http  32707/TCP
Endpoints:                <IP_ADDRESS>:8080,<IP_ADDRESS>:8080
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type     Reason                      Age               From                Message
  ----     ------                      ----              ----                -------
  Normal   IPAllocated                 13m               metallb-controller  Assigned IP "<IP_ADDRESS>"
  Normal   EnsuringLoadBalancer        2m (x8 over 13m)  service-controller  Ensuring load balancer
  Warning  CreatingLoadBalancerFailed  2m (x8 over 13m)  service-controller  Error creating load balancer (will retry): failed to ensure load balancer for service test-namespace/hello-service: error creating loadbalancer af4ec85ca799311e8b5b3fa163ea6e5a: error creating loadbalancer {af4ec85ca799311e8b5b3fa163ea6e5a Kubernetes external service af4ec85ca799311e8b5b3fa163ea6e5a 29d8a4d4-81ee-47fc-9043-408b5448cad8   <nil>  }: Resource not found

What you expected to happen:
The same without the warning.
How to reproduce it (as minimally and precisely as possible):
k8s 1.10
Anything else we need to know?:
k8s is running on OpenStack and security groups are set accordingly (UPDATE: it's worth mentioning Openstack LBaaS is not available in the environment)
Using protocol: layer2
Environment:

MetalLB version: 0.6.2
Kubernetes version:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:13:03Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:00:59Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

BGP router type/version: none
OS (e.g. from /etc/os-release):

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial

Kernel (e.g. uname -a): Linux node01 4.4.0-28-generic #47-Ubuntu SMP Fri Jun 24 10:09:13 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Thanks for the report.
That warning is not coming from MetalLB, it's coming from the Kubernetes controller-manager, a core component of k8s. This means that your cluster is configured with a cloud provider integration (probably OpenStack integration), and it's also trying to sync the LB state.
You can resolve this by disabling the OpenStack integration if you don't need it at all - however, it's possible you need that integration for other things like volume management. If you do, you have to either live with these warnings, or find a way to disable only the LBaaS integration in kube-controller-manager. That would be a bug/feature request for the OpenStack folks, and/or upstream Kubernetes.
Closing as there's nothing I can do about this in MetalLB, we can't control what the core control plane does with cloud platform integrations.


Introduce HashedStorageProvider

Overview
This PR introduces the HashedStorageProvider which allows us to instantiate HashedStorage using the KeyHasher from the StateCommitment type. This should be reviewed after https://github.com/paradigmxyz/reth/pull/12607.

False negative on failing tests due to https://github.com/paradigmxyz/reth/pull/12893

False negative on failing tests due to https://github.com/paradigmxyz/reth/pull/12893

overall approach makes sense though I don't like that we are losing type safety by passing hashers to the functions directly

I agree with the sentiment here.

I am wondering if we could make the HashedStorage/HashedPostState generic over the key hasher in a similar way to how HashMap operates? that way we could enforce that provider and StateCommitment impl only receive the states hashed with expected hasher

Yes, I think is a good approach. We would need to make the fields within HashedStorage and HashedPostState private to prevent direct modification as in some parts of the codebase hashing is done externally. Instead we would extend the interface to include methods for adding new entries to the objects. This may also mitigate the requirement of the KeyHasherProvider https://github.com/paradigmxyz/reth/pull/12895/files, once again providing better type safety.


Added auto_update_facets form parameter for /custom_list edit API

Description
Accepts a JSON of attributes that would generally go into a Facet object
Eg.

order
language
media
author

Added open API documentation for the modified endpoint which can be rendered using the index.html

Motivation and Context
The auto update feature for custom lists was missing the facets which are a part of the search criteria.
This is an important set of attributes which affect the results of the search and hence what may or may not be part of a list.
Notion

How Has This Been Tested?
API and facets have been unit tested

Checklist

[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.

Ready to merge.


I have given path and gem_path but it's still not working. I'm on OSX 10

{
"indent": 2,
"dasherize": false,
"old": false,
"path": "/usr/local/share/npm/bin /Users/jitendravyas/.nvm/v0.10.33/bin /Users/jitendravyas/.rvm/gems/ruby-2.1.2/bin /Users/jitendravyas/.rvm/gems/ruby-2.1.2@global/bin /Users/jitendravyas/.rvm/rubies/ruby-2.1.2/bin /usr/local/bin /usr/local/bin /usr/local/sbin /usr/bin /bin /usr/sbin /sbin /Users/jitendravyas/.rvm/bin /Users/jitendravyas/.rvm/bin",
"gemPath": "/Users/jitendravyas/.rvm/gems/ruby-2.1.2:/Users/jitendravyas/.rvm/gems/ruby-2.1.2@global",
"beautifyOnSave": true,
"inlineComments": false,
"newlineBetweenSelectors": true
}

Closing due to lack of feedback.


fix: correct documentation URLs for rules created by ESLintUtils.RuleCreator

Overview

What is it?

[ ] Feature / enhancement
[x] Bug
[x] Docs / tests / types / typos

Description

The documentation url for eslint rule should be https://qwik.builder.io/docs/advanced/eslint/#{ruleName} instead of https://qwik.builder.io/docs/advanced/dollar/.
{
  // ...
  meta: {
    docs: {
      description: ""
      recommended: "warn",
      url: "https://example.com/rule/my-rule" // ❌ This is omitted in the `NamedCreateRuleMetaDocs`
    },
}

Ref: https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/utils/src/eslint-utils/RuleCreator.ts#L11
Use cases and why

Write an img tag without giving it specific width and height and find it caught by the qwik/jsx-img lint rule. Click the eslint(qwik/jsx-img) in the end of the error message should open the correct documentation, which is https://qwik.builder.io/docs/advanced/eslint/#jsx-img in this case.

Checklist:

[x] My code follows the developer guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have made corresponding changes to the documentation
[ ] Added new tests to cover the fix / functionality

Love it!


Allow --build with Intermediate Errors - shouldn't happen on tsconfig errors

🔎 Search Terms
intermediate error
🕗 Version & Regression Information
This is about new feature in 5.6
⏯ Playground Link

💻 Code
tsconfig.json
{
    "extends": "../wrong/path/to/some/tsconfig.json",
    "compilerOptions": {
        // nothing, expects to inherit things
    }
}

🙁 Actual behavior
When compiling such project, tsc does complain that it cannot find the base config, however it continues building. This is a big problem when the base config contains important things like "outDir": "${configDir}/lib" - TS will happily build using the defaults, writing a ton of .js and .d.ts files next to the source files.
🙂 Expected behavior
For tsconfig.json file errors, do not continue building the project.
Additional information about the issue
No response

i encountered this during very large refactoring (moved folders for 100 packages in a monorepo). so i am updating paths left and right, fixing some other issues that pop up. unfortunately, some fixes require me to add new files, and in some folders we do have .js files, so they aren't ignored. and .d.ts files definitely aren't ignored.
thus when one project suddenly wrote all those .js and .d.ts files it was a very unexpected problem, git clean was the solution but as i found out later, when cleaning i did delete some files i had added earlier, but not staged/committed.
though there are many ways of running into this with malformed tsconfig files, in my case syntactically everything was correct, i was actually building a dependent project, for which the tsconfig was fixed and everything was green (vscode does show red squigly if the referenced file isn't found). It then unexpectedly pulled in a transient dependency that i hadn't prepared yet fully and that had the old wrong extends value.


Add Dead NFT Society Token

{
"chainId": 101,
"address": "9PzDH3HMwuXWTcKzRGuviPhRzryjqA3VgLqrvJE6fj2H",
"symbol": "DEAD",
"name": "Dead NFT Society",
"decimals": 0,
"logoURI": "https://raw.githubusercontent.com/solana-labs/token-list/main/assets/mainnet/9PzDH3HMwuXWTcKzRGuviPhRzryjqA3VgLqrvJE6fj2H/logo.png",
"tags": [
"social-token",
"community-token",
],
"extensions": {
"website": "https://deadnftsociety.com"
}
}
],
"version": {
"major": 0,
"minor": 3,
"patch": 3
}
}

Please note: This repository is being rebuilt to accept the new volume of token additions and modifications. PR merges will be delayed.

I agree to not ping anybody on Discord/Twitter/email about this pull request. Instead I will inquire by posting a new comment in the pull request if needed.

PRs are reviewed in bulk and and can take up to two weeks to be merged.
This repository is managed using an auto merge action. Please ensure your PR has no deleted lines, and it will be merged.
Please provide the following information for your token.
Please include change to the src/tokens/solana.tokenlist.json file in the PR.
DON'T modify any other token on the list.
At minimum each entry should have

Token Address:
Token Name:
Token Symbol:
Logo: (logo should be uploaded under assets/mainnet//*.<png/svg>)
Link to the official homepage of token:
Coingecko ID if available (https://www.coingecko.com/api/documentations/v3#/coins/get_coins__id_):

Auto merge requirements
Your pull request will be automatically merged if the following conditions are met:

Your pull request only adds new tokens to the list. Any modification to existing
tokens will require manual review to prevent unwanted modifications.

Your pull request does not touch unrelated code. In particular, reformatting changes to unrelated
code will cause the auto merge to reject your PR.

Any asset files added correspond to the token address you are adding. Asset files
must be PNG, JPG or SVG files.

Your change is valid JSON and conforms to the schema. If your change failed validation,
read the error message carefully and update your PR accordingly.

No other tokens shares the same name, symbol or address.

For example, this change would be rejected due to unrelated changes:

The bot runs every 60 minutes and bulk-merges all open pull requests to prevent conflicts.
This means that you need to wait up to 60 minutes for your pull request to be merged or reprocessed.

Website is purchased. Hosting not setup yet. Currently parked at GoDaddy


az storage-blob upload batch fails yet exitcode 0

This is autogenerated. Please review and update as needed.
Describe the bug
As described in #21477
az storage blob upload-batch gives errors when blob already exists and --overwrite is absent.
Yet the exitcode is still 0 indicating success. Someone might not pick up on these errors.
Command Name
az storage blob upload-batch
Errors:

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

az storage blob upload-batch --account-key {} -s {} -d {} --account-name {}
az storage blob upload-batch --account-key {} -s {} -d {} --account-name {}
$LASTEXITCODE

Expected Behavior
exitcode != 0
Environment Summary
Windows-10-10.0.19043-SP0
Python 3.10.4
Installer: MSI

azure-cli 2.37.0

Extensions:
account 0.2.2
application-insights 0.1.16
azure-devops 0.23.0

Dependencies:
msal 1.18.0b1
azure-mgmt-resource 21.1.0b1

Additional Context

storage

I'm facing a similar situation.
The difference is that I'm receiving another error from az blob upload-batch command in a bash script.
It's a networking issue, so nothing related to the --overwrite parameter.
But, the exit code is 0 as well.


Implement info button for mobile #85

testing instructions
Open the render site for this PR on mobile: https://lenape-timetree-dev-pr-158.onrender.com/

[x] when you load the page, an info button should be visible in the header but inactive/disabled
[x] when you close the intro, the info button should become active; tapping it should display the intro again, and make the info button inactive again
[x] if you select a leaf, the info button should become active; tapping it should display the intro and deselect the leaf
[x] if you select a leaf, then select a tag, then tap the info button, you should see the intro; the leaf and tag should be deselected
[x] go to the tag index page and choose a tag, so that you see the tree with the tag selected but no leaf active; the info button should be active; activating the button should deselect the tag and display the intro

check the render site on desktop:

[x] info button should not be visible in the header

Done!

Wow, super speedy! 🚀 Thank you.


Initial Quest

What does your feature do?
The first time someone login on to the server, get an initial quest to understand the system, a lot of my players have no idea how to get on the server.
Its like an option: Initial Quest?
How would people use it?
Because its very usefully to players who dont know the quests system.

Thanks for your suggestion. It's not within this project's goals to force a quest onto a player based on conditions, as there are already countless ways to accomplish this without a native option in Quests. Here are just a few:

Use a plugin like this one to run /questadmin give [player] [quest] on join
Place a Command Block within range which runs the aforementioned command
Create a Citizens 2 NPC at the spawn location that advertises the quest to the player
Put up Command Signs which runs /questadmin give [player] [quest] when clicked
Have someone write a plugin or module which accomplishes your task


[DissasmView]  Add support for the jmp instructions for the x86 code to follow and go to that memory region.

Fixed


(UX) Windows agent startup should hint if ovs* is down when running NetAdapterAdv. commands

Describe the bug
The setup for antrea can fail with a complex error message when the various ovs services are down:
F1110 09:17:32.738719   33300 main.go:58] Error running agent: error initializing agent: Set-NetAdapterAdvancedProperty : No matching MSFT_NetAdapterAdvancedPropertySettingData objects found by CIM query for
instances of the ROOT/StandardCimv2/MSFT_NetAdapterAdvancedPropertySettingData class on the  CIM server: SELECT * FROM
MSFT_NetAdapterAdvancedPropertySettingData  WHERE ((Name LIKE 'br-int')) AND ((RegistryKeyword = 'NetworkAddress')).
Verify query parameters and retry.

It turns out that an easy fix, if installation worked previously, is to ensure all ovs services are running:
Start-Service ovs-*

To Reproduce
On a working antrea cluster, stop one of the ovs services and then restart the agent.
Expected
The agent should report that it couldnt connect to OVS, or at least hint that connection rather then query params may be the culprit
Actual behavior
Agent just forwards the underlying query failure message from the CIM query

proposed fix
maybe  run something like
Get-Service ovs-*

and confirming all are running before proceeding with those CIM queries
then just exit gracefully, if not?
@ruicao93 for thoughts

Thanks @jayunit100 for your report.
I think query the network adapter retrun proper failure message if not exist is good. Will create PR for that, thanks.


Add Github Actions supported wheel-builds for CPU-only jaxlib on Winx64

Context: This PR aims to build CPU-only x64 wheels for Windows platforms. Since the current jaxlib wheel support for Windows is community-oriented with no officially-supported builds, this causes problems for platforms that need a cross-platform way to depend on jax, of which has become a recent issue for us on PennyLane. Our goal to to fully replace autograd (our default AD framework) in PennyLane with jax, and we require officially supported Windows wheels to allow this. This PR follows a recent discussion between Xanadu (@mlxd @trbromley) and the JAX team (@hawkinsp @yashk2810 ) on Xanadu preliminary exploring and adding the support.
Description of the Change: This PR adds a Github Actions workflow to make use of the windows-2019 runner and Visual Studio Enterprise 2019 to build jaxlib wheels, targeting CPU-only execution under Windows. This has been validated on 16GB private instances, though may require additional configuration to ensure memory walls are not hit on the 7GB free instances.
Benefits: Adds native support for CPU-only jaxlib under Windows.
Possible Drawbacks: Additional maintenance may be required for further development and extensions. May require a larger CI runner to compile end-to-end.

You'll need to sign the CLA first.

You'll need to sign the CLA first.

Thanks @hawkinsp . Just checking for internal approval, will sign when confirmed.

Hey @hawkinsp just dropping in to say the CLA is signed. Feel free to reach out if anything else is needed.

Is this really the dream it seems?  CPU-only JAX available which can be added with standard pip requirements/etc?  If there is any chance this is available in the next few months as an official release it would make JAX a feasible alternative for teaching ML to students in fall coursework.

Can someone at Google please merge this?

Address this https://github.com/google/jax/issues/15907, then we will be able build CPU and CUDA natively on Windows.
@jlperla At the moment, you can use https://github.com/cloudhan/jax-windows-builder for old version jaxlib builds.
The idea that use free tier Github Actions to build jaxlib is actually executed poorly in https://github.com/cloudhan/jax-windows-builder. The problem is not disk space and RAM space. The OS SSD is ~120GB and tmp HDD disk is 16GB. You can mount a 32GB swap disk for RAM. The root problem is 2 cores for building without caching.  The CPU jaxlib will take ~5 hours and CUDA jaxlib will take ~6 hours, where 6 hours is the hard limit for a single job in github actions. To enable windows build, some inc, say Google, MS or maybe some other startups that advocate Bazel, need to donate some resource to it.

I'm going to merge this and see how it goes. I believe we may be able to budget for a larger runner, but I'll make that change in a subsequent PR.

Sounds good, thanks @hawkinsp
Feel free to tag me if I can help out in any further way.

Ok, we're making some progress here.
I forked this config to make a new "Windows CI" config (https://github.com/google/jax/actions/workflows/windows_ci.yml
) that is triggered:

nightly
when manually triggered.

This action builds jaxlib and runs the tests for one Python version (3.9, at the moment).
After a bit of iteration, we can build jaxlib and run the tests:
https://github.com/google/jax/actions/runs/5259225418/jobs/9504483477
It took around 57 minutes on a 16 core Windows 2019 VM. Since a 16 core VM costs $0.125 per minute, this means the build cost about $7.25. So it's a bit too expensive and slow to run as a presubmit at the moment.
Most tests pass:
========= 12 failed, 17357 passed, 2138 skipped in 747.98s (0:12:27) ==========
I've actually already fixed a few of those, namely https://github.com/openxla/xla/pull/3544 fixes LaxTest.testRngBitGeneratorReturnedKey and https://github.com/google/jax/pull/16373 fixes MetadataTest.test_source_file_prefix_removal.
PRs to fix any of the other problems welcome!


en.json and cy.json files are synced

39 unused phrases have been removed, backed up in an excel
the 2 json files have been synced to contain all the constants in both files
here are the list of phrases currently required for translation
Translations required Eng to Cy.xlsx
Unused phrases backup.xlsx

@SushreeM-h  is there a ticket assigned for this update? I don't see anything mentioned on the tickets

If you don't have a ticket created Please create one and update it.. Thanks.. just to be able to keep track on the work done.


runtime: config: linux: Edit BlockIO struct

WeightDevice, ThrottleReadBpsDevice, ThrottleWriteBpsDevice, ThrottleReadIOpsDevice, ThrottleWriteIOpsDevice are now slices to well defined structs to allow setting multiple devices in their respective blkio file. By using a string to represents those values it wasn't possible to set correct values when multiple devices were passed in the config (either newline separated or comma separated).
I don't know how this worked before to define multiple devices for these cgroups resources. Writing something like 8:0 500\n8:16 300 isn't working (there's a Docker PR https://github.com/docker/docker/pull/13959 which passes that format to libcontainer and by passing that format to current master runc isn't working as well (neither using comma) because it just writes it to cgroup file which doesn't accept newlines/commas).
8:0 500\n8:16 300 will only set the first device before the \n when written to blkio.weight_device for instance.
I'm not fond of blockIODevice embedded struct and specifying major:minor for this. I think it will suffice specifying just a Path and do the syscall.Stat to retrieve major:minor where it needs to (I'm thinking about libcontainer as it's currently doing for Rlimits for instance).
By just specifing a Path this will help also for example to just receive a string like /dev/sda and do the major:minor convert in the implementation. It's useful to have the plain string like for checking  which scheduler the device is using (/sys/block/sda/queue/scheduler w/o converting back major:minor to path in implementation).
I'll open an issue in runc to track this and update runc/libcontainer if this patch is correct and lands.
Signed-off-by: Antonio Murdaca<EMAIL_ADDRESS>

On Wed, Sep 16, 2015 at 07:51:53AM -0700, Antonio Murdaca wrote:

I'm not fond of blockIODevice embedded struct and specifying
major:minor for this. I think it will suffice specifying just a
Path and do the syscall.Stat to retrieve major:minor where it
needs to (I'm thinking about libcontainer as it's currently doing
for Rlimits for instance).

I'm fine with blockIODevice.  One benefit is that we don't need to
specify ordering between mounting, mknod calls, and these resource
limits, because we can set major:minor limits before the path exists.

On Wed, Sep 16, 2015 at 07:51:53AM -0700, Antonio Murdaca wrote:
I'm not fond of blockIODevice embedded struct and specifying
major:minor for this. I think it will suffice specifying just a
Path and do the syscall.Stat to retrieve major:minor where it
needs to (I'm thinking about libcontainer as it's currently doing
for Rlimits for instance).

I'm fine with blockIODevice.  One benefit is that we don't need to
specify ordering between mounting, mknod calls, and these resource
limits, because we can set major:minor limits before the path exists.

fine, I guess /proc/partitions or something could be used to map device numbers back to path but shouldn't be a problem now

89bed76 looks good to me, although I'm a bit surprised that all we
have to update are the Go types and examples.  Can we add a longer
example linking to 1 and sketching out the intended mapping between
our docs and that interface?  A list of options for each type with
their intended type, the fact that they're required, and a one-line
summary of their interpretation would help me anyway.  For example:

major (int64, required) number for the devices that will be limited.
minor (int64, required) number for the devices that will be limited.
weight (int16, required) in CFQ IO scheduling

where I've tried to use the “whole thing reads like a sentence”
phrasing 2.

89bed76 looks good to me, although I'm a bit surprised that all we
have to update are the Go types and examples.  Can we add a longer
example linking to 1 and sketching out the intended mapping between
our docs and that interface?  A list of options for each type with
their intended type, the fact that they're required, and a one-line
summary of their interpretation would help me anyway.  For example:

major (int64, required) number for the devices that will be limited.
minor (int64, required) number for the devices that will be limited.
weight (int16, required) in CFQ IO scheduling

where I've tried to use the “whole thing reads like a sentence”
phrasing 2.

sounds good to have these docs around, I'll update

On Wed, Sep 16, 2015 at 12:51:13PM -0700, W. Trevor King wrote:

…
[1]: https://www.kernel.org/doc/Documentation/cgroups/blkio-controller.txt

These also talk about leaf_weight.  While we're formalizing these
entries, do we want to expose that too?
Maybe with:

weight (int16, optional) when competing within this cgroup in
CFQ IO scheduling
leafWeight (int16, optional) when competing with cgroup children
in CFQ IO scheduling

You must specify at least one of weight or leafWeight in a given
entry, and can specify both.
Or we can have separate entries (like you're currently doing for
weight and throttle) for handling weight and leaf_weight.  Or we can
punt for now, and come back and look at leaf_weight when someone asks
for it with a particular use-case in mind.

On Wed, Sep 16, 2015 at 12:56:08PM -0700, Antonio Murdaca wrote:

sounds good to have these docs around, I'll update

I had some other cgroups docs stubbing in #99 if you want to borrow
from there (although there hasn't been discussion about explicitly
handling device-access cgroups, so probably leave that section out ;).

Or we can have separate entries (like you're currently doing for
weight and throttle) for handling weight and leaf_weight.

these are separate cause one is dealing with weight and the other with rate

Or we can
punt for now, and come back and look at leaf_weight when someone asks
for it with a particular use-case in mind.

It shoudln't be so much hard to put leaf_weight into account. And I like your proposed struct with both weight and leaf_weight.

I had some other cgroups docs stubbing in #99 if you want to borrow
from there (although there hasn't been discussion about explicitly
handling device-access cgroups, so probably leave that section out ;).

Thanks!

I'm not fond of blockIODevice embedded struct and specifying major:minor for this. I think it will suffice specifying just a Path and do the syscall.Stat to retrieve major:minor where it needs to (I'm thinking about libcontainer as it's currently doing for Rlimits for instance).

Though systemd takes device path instead of major minor number, but for struct, I think is't better to store just major minor numbers.
And +1 for this change.

Though systemd takes device path instead of major minor number, but for struct, I think is't better to store just major minor numbers.

systemd was one of the reason I thought having Path would have been good. But whatever, I agree having device numbers is better.

ping @vbatts @mrunalp @LK4D4 @wking @hqhq
I've updated the structure of the cgroups section here by splitting the example into per resource section and filled the bllkio one as per current PR topic. If you're happy with this path/format I can do a follow up PR updating the rest of the resources (I'd leave this PR as is now w/o going any further with other modifications if you're ok with this)

On Thu, Sep 17, 2015 at 05:55:38AM -0700, Antonio Murdaca wrote:

I've updated the structure of the cgroups section here by splitting
the example into per resource section and filled the bllkio one as
per current PR topic.

Looks good to me :).  I've left a few copy-edit suggestions, but I'm
on board with all the significant changes in a89565b2.  I'd be fine if
it landed as-is, with follow up PRs to accept/reject my copy-edit
suggestions (but I'd rather they got rerolled in or rejected here ;).

LGTM

LGTM

Thanks, I'll create a PR in runc to update libcontainer

@runcom Sounds good!


add request schema to post /wallet/did/create

The request for seed isn't defined in /wallet/did/create. But I wanna generate did with the specific seed so I made some modifications.
Thx!

I didn't add this because the DID can be provisioned on the command line using a seed, and I didn't want to encourage people to pass private keys over the REST interface. What's the use case, supporting multiple identities?

I didn't add this because the DID can be provisioned on the command line using a seed, and I didn't want to encourage people to pass private keys over the REST interface. What's the use case, supporting multiple identities?

That's true - seeds are secrets

I didn't add this because the DID can be provisioned on the command line using a seed, and I didn't want to encourage people to pass private keys over the REST interface. What's the use case, supporting multiple identities?

Oh, I didn't notice this security issue. The use case is when I need to operate the ledger with the STEWARD identity, there is no place to import DID or create the same steward DID. Or there is another way to reach my needs?
Thx!

I agree that this should not be part of the framework because of the danger of misuse.
Our belief on the right pattern for the deployment of an app in production (and also to be used for testing) is that there be a "provision" agent and "operational" agent, as follows:

"provision" agent gets a config list of objects that must exist in the wallet and on the ledger, checks if they exist and if not creates them, and then exits.  This must be a higher security app, as it is creating things from secrets (e.g. seeds), access to DIDs that can write to the ledger, etc. It might also have to create the wallet, and need higher level access database credentials.
"operational" agent gets a config list of objects that should be in the wallet and on the ledger, checks if they exist and if not, terminates with an error.  If the objects do exist, it runs the full agent logic with a "--read-only-ledger" setting to prevent creating anything else on the ledger. It is a lower security app and should not be creating things from secrets (and so need not know the secrets).

To manage this in testing, we recommend a wrapper script that has entry points to invoke either of the agents, and an entry point for testing that invokes both.
In production, with a production ledger, the "provision" script is only run when you need to create new objects on the ledger.  The wallet that is created MUST be properly guarded with an excellent backup and recovery mechanism. And, of course, the ledger is immutable.  No rewrites! :-)

Second comment - we cannot accept this PR as the DCO has not been done on the commits. Please see the contributors guidelines in the root folder.  All Hyperledger repos require all commits be signed for DCO.


Boat rental search including all locations with any boat access

should be looking at boat_rental column in poi_info only.

I think I just messed up the logic on this one. Will check.

This should now be fixed.


Support Renaming Plugin name?

Hello, Please does this support renaming a plugin.
Thank you

Plugins name can be changed by simply updating the package name in pubspec.yaml I think and there is not special bundle name for them.


How to disable local cache in dev mode

🐛 Bug report

Describe the bug
Coding for a while, this url localhost:8787 is not changed by any modified, e.g. wheh I comment belows if condition, it will keep the old result.
export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    const url = new URL(request.url)
    console.log(url.host);
    console.log(url.hostname);

    // if (url.host in regs) {
    // 	url.host = regs[url.host]
    // 	url.port = '80'
    // 	return proxy(url, request)
    // }
    return new Response("Hello World!");
  },
};

Environment and versions
Fill out the following information about your environment.

operating system: Windows 11
output of wrangler -V: wrangler 2.0.23
output of node -v: v16.14.2
content of wrangler.toml:

# wrangler.toml
name = "proxy-github"
main = "src/index.ts"
compatibility_date = "2022-07-31"
workers_dev = false

To help us diagnose this further, would you be able to provide a bit more information about your environment?

Are you using local mode or remote mode? (i.e. wrangler dev or wrangler dev --local)
Roughly how long does it take for changes to stop working?
Could you provide the CLI output of wrangler dev (or wrangler dev --local if you're using local mode), up until the point at which changes to your code stop refreshing?


Creating USB from Windows?

It feels like an oversight that creating the USB to install MacOS, requires MacOS.

this program work under macOS only


skipMerge v.s. javascript imports

The situation is, we have a setup of js files import each other like this and they cannot pass the compile with skipMerge indicator.
//common.js
export function foo {
}

//client.js
import {foo} from "./common.js";

It keep reporting this problem when minfying client.js:
ERROR - [JSC_JS_MODULE_LOAD_WARNING] Failed to load module "./common.js"
I tried to test it with the closure-compiler.jar with myself, finding that it does not naturally support skipMerge. I am now assuming that the skipMerge is implemented simply by calling the command line with --js parameter multiple times (or similar). This could work only when there is no import relationship between js files, but in the reported above situation.
I don't have a clue on how to handle that for now - but this might help for your experts to follow up in upcoming releases. I understand until Google support skipMerge natively this can be hard, though.

For reference, what is the expected output? Do you want to preserve the import / exports as-is, just minify the content?

For reference, skipMerge is basically just a shortcut for minifying multiple files individually. Closing this for now, as there does not seem anything I can do.


Vocabulary generation error

Attempting to generate an RDF::StrictVocabulary from http://rs.tdwg.org/dwc/rdf/dwcterms.rdf using  RDF::VocabularyLoader.
See projecthydra-labs/rdf-vocab#35 for details.

See how that works for you; note that it's on the develop branch, and not yet released to rubygems. You should be able to clone the ruby-gems gem and add a line to pull from develop to test out. I'll likely release an update of the gem next week.
You can also try by cloning the rdf-rdfxml gem and parsing using script/parse using your URL.
cc/ @jhallida @jcoyne @no-reply @barmintor @terrellt


Wealth correction for observed wealth

So is the function used to transform the wealth definition in survey data to the definition in dcegm?

Wealth in survey data to wealth in dcegm definition


magnet uses StringToken

fix #535

I believe the new rules for query/params_ref broke something with magnet.  For instance,
    auto xt = m.exact_topics();
    for (auto h : xt)
        std::cout << "topic: " << h << "\n";

is now creating a url_view to the wrong address, even before removing MutableString, but I still not being able to figure out why.

I believe the new rules for query/params_ref broke something with magnet.

That suspicion was correct. The old interface that returned string views cannot work anymore because params return references to temporary strings that don't exist after the view is dereferenced.

good to go if you think its ready

Merged


Create a linter rule that warns if people add a constructor to StatefulWidgets

From reading through StackOverflow, Flutter users make a common mistake: they write:
class MyStateful extends StatefulWidget {
  const MyStateful({Key key, this.props}): super(key: key);

  final int props;

  MyStatefulState createState() {
    return MyStatefulState(props);
  }
}

class MyStatefulState extends State<MyStateful> {
  MyStatefulState(this.props);

  int props;

  @override
  Widget build(BuildContext context) {
    return Text('$props');
  }
}

when they should instead do:
class MyStateful extends StatefulWidget {
  const MyStateful({Key key, this.props}): super(key: key);

  final int props;

  MyStatefulState createState() => MyStatefulState();
}

class MyStatefulState extends State<MyStateful> {
  Widget build(BuildContext context) {
    return Text('${widget.props}');
  }
}

This causes issues.
For example, this SO question is one of the most popular ones:
https://stackoverflow.com/questions/50818770/passing-data-to-a-stateful-widget/50818870#50818870
It is also regular for people to ask why State doesn't get updated in that situation, like https://stackoverflow.com/questions/59130099/how-does-the-stateful-vs-stateless-distinction-affect-when-widgets-will-be-rebu
A linter rule like no_state_constructor that warns if there's a custom constructor on State should help people solve this issue naturally.

/cc @HansMuller @devoncarew

cc @pq, @a14n

This is now implemented in a new lint no_logic_in_create_state:  https://github.com/dart-lang/linter/pull/1877 and will be available in the next linter release (0.1.106).

NICE

As of https://github.com/dart-lang/sdk/commit/c0dfdb27e77bc7f6ddd35d93f5f4e2104210a69e, this is in the Dart SDK; as of https://github.com/dart-lang/sdk/commit/d9570f907e8bd13166ee9a215c229ad130bb152a, there's a quick-fix to automate corrections.
Once it makes it's way into the Flutter SDK, I'd love any feedback. 👍


Copy Name(s) in Projects Browser

What is the feature you'd like to have?
It would be nice to be able to copy a filename/multiple filenames out of the Project Browser.
Are any alternative solutions acceptable?
No. There is no other way to recover remote filenames other than renaming each one and copying the filename.
Additional Information:
Components UI has this functionality

Added in 4.0.4827-dev