[WIP] OffsetManager Implementation

There are still some TODOs sprinkled throughout, but not all of them necessarily need to be fixed before we ship this (it's already pretty close to the minimum viable implementation). The only critical missing piece is implementing AsyncClose for the PartitionOffsetManager.
Regardless, it's at the state where it can be opened up for comments. This should completely supersede #379.

@Shopify/kafka

What is approximate ETA?

We don't have any direct need for this, so finishing this PR does not have high priority for us. If you're willing to work on this, the following things need to be added before we can merge it:

AsyncClose
unit tests
functional tests

I just added a quick and dirty AsyncClose implementation, so this is theoretically feature-complete in a minimal state, but I can pretty much guarantee it's broken since I've never actually run it.

It's not clear to me if kafka-based offset management is functional in 0.8.2 since v1 of the request requires fields which won't have meaning until the full consumer rewrite lands in 0.9 (https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.9+Consumer+Rewrite+Design)

It's functional; when I was testing this it was successfully storing and retrieving offsets and metadata.

What did you put in those fields then? Zeros?
(Specifically: ConsumerGroupGeneration, and ConsumerID)

Yup, I just didn't set them: https://github.com/Shopify/sarama/pull/379/files

Hey guys, I was looking at the offset manager interface from the end user perspective and I have a couple of suggestions to make:

It would be nice if consumer group was not read from config, but rather accepted as a parameter along with topic and partition. That would allow using one offset manager for many consumer groups.
Why does the user should be concerned with creating a committer for each particular topic/partition? This seems to be unnecessary complication. Why not make the OffsetConsumer take care of that and provide a stupid simple interface. e.g.:

type OffsetResult struct {
    offset int64
    err error
}

type OffsetManger interface {
    FetchOffset(group, topic string, partition int32, replyTo chan<- OffsetResult)
    CommitOffset(group, topic string, partition int32, offset int64, replyTo chan<- error)
    Close() error
}

NewOffsetManagerWithClient(c *Client) (*OffsetManger, error)

I understand that you were modeling the API to align with Consumer/PartitionConsumer, but the OffsetManager seems to be a higher level abstraction that asks for a simpler interface. Just think how much code needs to be written to make use of the OffsetManager implemented in this PR comparing to the usage of the suggested interface.
This is certainly just my subjective opinion. In any case thank you very much, guys, for what you are doing.

allow using one offset manager for many consumer groups

That's actually surprisingly difficult. While you can include multiple topics and partitions in a single offset fetch or commit, you can only include a single consumer group (this is a limitation of the kafka protocol) so having an OffsetManager support multiple consumer groups would require a great deal more work.

Why does the user should be concerned with creating a committer for each particular topic/partition?

Eventually they won't have to create anything at all, it will be baked into the consumer (so creating a PartitionConsumer will automatically create a PartitionOffsetManager behind the scenes that keeps itself up-to-date with the messages you've consumed). I just haven't written that part yet :)

I understand that you were modeling the API to align with Consumer/PartitionConsumer

Yup, see my previous answer.

@eapache makes sense. Thank you for the clarification.

@eapache Could you please consider making consumer group into a parameter of NewOffsetManager, for having it in sarama.Config forces us to use one sarama.Client per offset manager.

We're using Sarama, and looking at our options for offset management.
Is this PR still a low priority for?

@aaronkavlie-wf We are not in need of this feature ourselves (we use Zookeeper to manage our offsets). To get this rolling again, it would be great if you can try out this branch to a) see if it works and b) give feedback on the proposed API.

@eapache @wvanbergen I have tried to use this offset manager in our balanced consumer implementation here are my comments (some of them have already been mentioned above by I decided to list them all here anyway):

It would be nice to have group as a OffsetManager.ManageParitition parameter or at least as a NewOffsetManager parameter rather than a part of Config. The later approach forces the user to make a client connection per consumer group;
metadata should probably be a parameter of SetOffset rather than settable on its own.
PartitionOffsetManager.Close blocks forever because errors channel is never closed in the implementation.
If a PartitionOffsetManager is closed a pending offset may not be committed.

@wvanbergen are you using your kafka repo for offset management (in production)? Or something else?

@aaronkavlie-wf yup that is what we are using.

I tried this out yesterday and did some simple testing. It seems to be storing/retrieving offsets just fine.
I agree with @horkhe that it's cumbersome to manage the partition offset managers; baking it into the consumer as @eapache mentions will be a welcome improvement.

@aaronkavlie-wf beware of the issues that I listed above.

@aaronkavlie-wf I have been thinking about offset manager interface and actually agree with @eapache that it is not a higher level abstraction as I thought before but ruther a lower level building block that along with partition consumer can be used to build a high level balanced consumer.
But the way in our org we have implemented it a little bit differently to ensure that commits are never lost during shutdown and that it is easy to use with multiple consumer groups. I am not sure if it is suitable for the master but maybe later I will submit a PR for you to consider.
An interface that we come up with is a bit different:
// OffsetManager uses Kafka to store consumed partition offsets.
type OffsetManager interface {
	// ManagePartition creates a PartitionOffsetManager on the given
	// group/topic/partition. It returns an error if this OffsetManager is
	// already managing the given group/topic/partition.
	ManagePartition(group, topic string, partition int32) (PartitionOffsetManager, error)

	// Close terminates all spawned PartitionOffsetManager's and waits for them
	// to commit pending offsets. So it is not necessary to call Close methods
	// of spawned PartitionOffsetManagers explicitly.
	Close()
}

// PartitionOffsetManager uses Kafka to fetch consumed partition offsets. You
// MUST call Close() on a partition offset manager to avoid leaks, it will not
// be garbage-collected automatically when it passes out of scope.
type PartitionOffsetManager interface {
	// InitialOffset returns a channel that an initial offset will be sent down
	// to, when retrieved.
	InitialOffset() <-chan FetchedOffset

	// CommitOffset triggers saving of the specified commit in Kafka. Actual
	// commits are performed periodically in a background goroutine. The commit
	// interval is configured by `Config.Consumer.Offsets.CommitInterval`.
	CommitOffset(offset int64, metadata string)

	// Errors returns a read channel of errors that occur during offset
	// management, if enabled. By default errors are not returned. If you want
	// to implement any custom error handling logic then you need to set
	// `Consumer.Return.Errors` to true, and read from this channel.
	Errors() <-chan *OffsetCommitError

	// Close stops the PartitionOffsetManager from managing offsets. It is
	// required to call this function before a PartitionOffsetManager object
	// passes out of scope, as it will otherwise leak memory.
	Close()
}

type FetchedOffset struct {
	Offset   int64
	Metadata string
}

type OffsetCommitError struct {
	Group     string
	Topic     string
	Partition int32
	Err       error
}

The implementation is here.

Most recent push makes some of the API adjustments that have been suggested (group is now per-offset-manager, and offset/metadata are set/fetched together). It also fixes the missing close(error).
I haven't figured out the best behaviour yet regarding "If a PartitionOffsetManager is closed a pending offset may not be committed.", since we probably don't want to spin forever trying to commit an offset before we give up, but there's also no harm in that as long as Close hasn't been called... do we have the user configure a retry-count that is only valid when shutting down? That seems a bit odd.

@eapache Did you amend to your previous commit? This PR is only showing a single commit, "wip".

I was actually starting on some tests; separate commits would make it easier to merge those in.

Did you amend to your previous commit?

Ya, when I'm working on something incomplete like this I'll typically always just amend a single WIP commit.

I was actually starting on some tests; separate commits would make it easier to merge those in.

Sure, I can do that in future. I'll do one more amend to give it a useful commit message and then I'll put any other changes as separate commits on top.

@horkhe what does your fork do in the event it can't commit offsets on shutdown? Spin forever trying again?

@eapache yes, it spins forever. The idea is that Kafka is highly available so it will get back to service eventually. And any service redeployment/restart will be delayed by the spin until this happens.

Latest version here should spin forever; it feels weird, but I can't think of any other better behaviour.
At some point we'll probably want to permit configuring a timeout after which to give up, but that's really easy to do with the way things are set up now.

@eapache I'm looking at this again, and the purpose of the metadata argument added to SetOffset() is not clear to me.

Per https://cwiki.apache.org/confluence/display/KAFKA/Offset+Management

These fields should be mostly self explanatory, except for metadata. This is meant as a way to attach arbitrary metadata that should be committed with this offset commit. It could be the name of a file that contains state information for the processor, or a small piece of state. Basically it is just a generic string field that will be passed back to the client when the offset is fetched. It will likely have a tight size limit to avoid server impact.

Tangentially: I just force-pushed a rebase of this branch in order to resolve a small merge conflict and to pull in @horkhe's test infrastructure.

I am making some progress on tests for this.
At least, for basic functionality such as Offset(), SetOffset().
Testing OffsetCommitResponse errors is tricky; they trigger chain of logic & server interactions and it deadlocks if you don't have everything just so. I haven't sorted that out yet.

it deadlocks if you don't have everything just so

It's entirely possible that your tests will find actual bugs; I can't say much about the current code right now beyond the fact that it compiles. In other words: don't waste too much time trying to figure out why a test is hanging if it looks like it ought to work :)

Discovered a bug
The broker offset manager mainLoop is initialized, and starts committing offsets on a timed interval independent of fetching offsets. If the fetchInitialOffset call returns ErrOffsetsLoadInProgress, it will start committing offset 0.
Here's the test run, with copious logging statements sprinkled throughout offset_manager.go
(run with config.Consumer.Offsets.CommitInterval = 1 * time.Millisecond):
=== RUN TestFetchInitialInProgress
2015/08/13 12:44:28 selectBroker
2015/08/13 12:44:28 refreshing...
2015/08/13 12:44:28 coordinator
2015/08/13 12:44:28 FETCHINITIAL
2015/08/13 12:44:28 flushing...
2015/08/13 12:44:28 About to commit offset 5 on partition 0
2015/08/13 12:44:28 ErrOffsetLoadInProgress
2015/08/13 12:44:28 Sleeping...
2015/08/13 12:44:28 flushing...
2015/08/13 12:44:28 About to commit offset 0 on partition 0
2015/08/13 12:44:28 CommitOffset response:
2015/08/13 12:44:28 (*sarama.OffsetCommitResponse)(nil)
2015/08/13 12:44:28 Trying again...
2015/08/13 12:44:28 FETCHINITIAL
2015/08/13 12:44:28 err from broker.broker
2015/08/13 12:44:28 returning nil, err
--- FAIL: TestFetchInitialInProgress (0.25s)
	offset_manager_test.go:151: kafka: broker not connected

Nice find; pushed a fix.

@eapache wow, that was an amazingly quick fix!
:+1: :beer:

You explained it super-clearly and I happened to have the code open already while working on a different PR :)

I noticed that some of my tests weren't shutting down cleanly, so I made sure to call .Close() on everything. However, I still have a runaway brokerOffsetManager infinite loop.
Test case:
// Test recovery from ErrNotCoordinatorForConsumer
// on first fetchInitialOffset call
func TestFetchInitialFail(t *testing.T) {
	om := initOM(t)

	// Error on first fetchInitialOffset call
	responseBlock := OffsetFetchResponseBlock{
		Err:      ErrNotCoordinatorForConsumer,
		Offset:   5,
		Metadata: "test_meta",
	}

	coordinator.SetHandler(func(req *request) (res encoder) {

	})

	fetchResponse := new(OffsetFetchResponse)
	fetchResponse.AddBlock("my_topic", 0, &responseBlock)
	coordinator.Returns(fetchResponse)

	// Refresh coordinator
	newCoordinator := newMockBroker(t, 3)
	refreshResponse := new(ConsumerMetadataResponse)
	refreshResponse.CoordinatorID = newCoordinator.BrokerID()
	refreshResponse.CoordinatorHost = "<IP_ADDRESS>"
	refreshResponse.CoordinatorPort = newCoordinator.Port()
	broker.Returns(refreshResponse)

	// Second fetchInitialOffset call is fine
	fetchResponse2 := new(OffsetFetchResponse)
	responseBlock2 := responseBlock
	responseBlock2.Err = ErrNoError
	fetchResponse2.AddBlock("my_topic", 0, &responseBlock2)
	newCoordinator.Returns(fetchResponse2)

	if _, err := om.ManagePartition("my_topic", 0); err != nil {
		t.Error(err)
	}

	broker.Close()
	coordinator.Close()
	newCoordinator.Close()
	testClient.Close()

	// Wait to see what the brokerOffsetManager does
	time.Sleep(500 * time.Millisecond)
}

logging output (again, all my random logging added to offset_manager.go):
=== RUN TestFetchInitialFail
2015/08/14 11:57:56 selectBroker
2015/08/14 11:57:56 refreshing...
2015/08/14 11:57:56 coordinator
2015/08/14 11:57:56 FETCHINITIAL
2015/08/14 11:57:56 ErrNotCoordinatorForConsumer
2015/08/14 11:57:56 selectBroker
2015/08/14 11:57:56 unref
2015/08/14 11:57:56 refreshing...
2015/08/14 11:57:56 coordinator
2015/08/14 11:57:56 FETCHINITIAL
2015/08/14 11:57:56 ErrNoError
2015/08/14 11:57:56 NOT returning nil, err
2015/08/14 11:57:56 1 subscriptions
2015/08/14 11:57:56 Offset commit request: offset 5 on partition 0
2015/08/14 11:57:56 CommitOffset response:
2015/08/14 11:57:56 (*sarama.OffsetCommitResponse)(nil)
2015/08/14 11:57:56 aborting
2015/08/14 11:57:56 rebalancing...
2015/08/14 11:57:56 selectBroker
2015/08/14 11:57:56 unref
2015/08/14 11:57:56 refreshing...
2015/08/14 11:57:56 kafka: tried to use a client that was closed
2015/08/14 11:57:56 No err
2015/08/14 11:57:56 rebalancing...
2015/08/14 11:57:56 selectBroker
2015/08/14 11:57:56 refreshing...
2015/08/14 11:57:56 kafka: tried to use a client that was closed
2015/08/14 11:57:56 No err
2015/08/14 11:57:56 rebalancing...
2015/08/14 11:57:56 selectBroker
2015/08/14 11:57:56 refreshing...
2015/08/14 11:57:56 kafka: tried to use a client that was closed
2015/08/14 11:57:56 No err
2015/08/14 11:57:56 rebalancing...
2015/08/14 11:57:56 selectBroker
2015/08/14 11:57:56 refreshing...
2015/08/14 11:57:57 kafka: tried to use a client that was closed
2015/08/14 11:57:57 No err
2015/08/14 11:57:57 rebalancing...
2015/08/14 11:57:57 selectBroker
2015/08/14 11:57:57 refreshing...
--- PASS: TestFetchInitialFail (0.71s)

@eapache

Per @horkhe's request, the manager will not shut down until it successfully commits all outstanding offsets. Based on your logs that is not the case.

I suspected that may be the reason, but it does pose a problem for isolating tests.
It doesn't seem that it should try to commit if the pom is not dirty (indicating uncommitted offsets).

I fixed a few minor issues around this area, although none of them critical. I also found the missing return call after bom.abort() which, given you'd put that in your PR as well, prompted me to cherry-pick your other fixes to this branch immediately (with proper attribution) just to avoid me having to debug problems you've already solved :)
After more thought, this is not an infinite loop in the brokerOffsetManager, it is an infinite loop in the partitionOffsetManager because you call ManagePartition() but never Close the result. Closing the underlying client will not close any individual PartitionOffsetManagers.

I am using this to implement offset management for my kafka consumer: https://github.com/wvanbergen/kafka/pull/72
One issue I ran into: it commits the last processed offset into Kafka. When resuming a consumer, it will return the exact same value as initial offset. If you feed that to sarama, you end up consuming the message with that offset twice.
This is a simple off by one error. The JVM implementation that uses Zookeeper simply stored the last committed offset + 1 into Zookeeper, so when you read it you can just feed the obtained value directly to ConsumePartition.
Should the offset manager take care of this off by one error, or is that the responsibility of the user?

Another idea: when you have never committed an offset before, Offset() will return -1. We could add a setting Consumer.Offsets.Initial that can be set to OffsetOldest or OffsetNewest. the offsetmanager would then return this value whenever the call to the server returns a negative value.

Should the offset manager take care of this off by one error, or is that the responsibility of the user?

What does the JVM consumer do when you tell it to use kafka? Does it send latest+1 or does it just send latest? Either way, the question is if offset manager should handle it or the high-level consumer; the user shouldn't care.

We could add a setting Consumer.Offsets.Initial that can be set to OffsetOldest or OffsetNewest.

Hmm, not sure how best to handle this case; an alternative would be a parameter to ManagePartition like the one we take in ConsumePartition.

@eapache

After more thought, this is not an infinite loop in the brokerOffsetManager, it is an infinite loop in the partitionOffsetManager because you call ManagePartition() but never Close the result. Closing the underlying client will not close any individual PartitionOffsetManagers.

That makes sense, thanks.

Final suggestions: add Close() error to the OffsetManager interface, and require it to be called during shutdown. This may be a noop for this implementation, but other implementations may need to free up shared resources that are not related to a PartitionOffsetManager. This is also more in line with basically any other type we expose in sarama.

Are there any remaining todos for this PR?

@aaronkavlie-wf Just one outstanding PR to finish up: #520, and the question about the error type, but after we resolve those we can merge this I think.
I am mostly concerned about getting the interface right; if we find any bugs we can fix them later, but fixing the interface is harder due toe API stability guarantees.

https://github.com/Shopify/sarama/pull/520 proposes some API changes, and fixes a bug.

I am happy with this now, but @eapache may want to take a final look.

 If I don't find anything, and nobody else raises any last-minute issues, we can merge this as-is on Monday.

@horkhe you may be interested in the minor API changes we made:

Offset (the getter) is now NextOffset and returns +1 the current offset so it is always the correct offset to start consuming at
SetOffset (the setter) is now MarkOffset as that seems more precise

A few more questions to consider (none of them block merging this though):

Should MarkOffset validate that the offset being marked is > (or >=) the offset already stored?

Do we need any other "flush triggers" than a simple timer (maybe based on number of uncommitted offsets or something)?
Should we expose a Dirty() bool reader on the POM?

OK, I am happy with this now. I would probably like to have a few more of the above questions answered before we put this into a versioned release (1.6?) but it can go into master now and we iterate from there.

Should MarkOffset validate that the offset being marked is > (or >=) the offset already stored?

I see a valid use case of "resetting" a consumer by setting all the offsets to a negative value. However, that might be easier using low level primitives (Coordinator() and crafting a custom request). So I am OK with adding this.

Do we need any other "flush triggers" than a simple timer (maybe based on number of uncommitted offsets or something)?

I have no problem if this gets added, but let's first see if anybody is actually interested in this.

Should we expose a Dirty() bool reader on the POM?
Would it be handy to expose a Flush() method on the POM which blocks until !dirty (just like the equivalent syscall)?

I don't think Dirty() bool is not too useful without a Flush() error, so it only makes sense if we add them together. I am not so sure if I can think of good use cases for this. The only time you really want to be sure the last offset is flushed is when you stop consuming from the partition. In that case you should always call Close()


fix(check:policy): Exclude tsc --watch tasks from policy

Scripts that call tsc with the --watch flag should be excluded from the fluid-build task policy since fluid-build doesn't need to run them.

The test failure is because of #18105.

/azp run Build - build tools

/azp run Build - build-tools

Thanks for re-running the pipeline, @alexvy86!


Help regarding updating a subelement in an array?

Hi there! im trying to update a subelement in a document but having some troubles, i tried using array filters with no luck as it told me it was an uknown modifier, is there a better way to get the index and update the specific attribute? any help or direction is greatly appreciated
Thanks
document example
%{
  "details" => [
    %{
      "type" => "sedan",
      "color" => "blue"
    },
%{
      "type" => "sedan",
      "color" => "orange"
    },
  ],
  "make" => hyundai
}

code example
Mongo.update_one(
        conn,
        "cars",
        %{"details.color" => new_color},
        "$set": ["details.0.color": new_color]
      )

I guess you need to use the $-operator because you are using a fixed index. And your filter looks a bit strange. You are searching for the new_color value to set it to the new_color value. That does not make sense. Try this:
Mongo.update_one(
        conn,
        "cars",
        %{"details.color" => "blue},
        "$set": ["details.$.color": "red"]
      )

@zookzook yeah my apologies, that is not supposed to have new_color as the search :sweat_smile: , i tried what you suggested and it does not seem to modify the record , in my response i get
{:ok, %Mongo.UpdateResult{
   acknowledged: true,
   matched_count: 1,
   modified_count: 0,
   upserted_ids: []
 }}

@zookzook same response :grimacing:

Maybe your document does not fit to the update command. Here is a working example:
iex [16:36 :: 2] > Mongo.insert_one(conn, "cars", %{details: [%{type: "sedan", color: "blue"}, %{type: "sedan", color: "orange"}]})
16:38:25.872 [info] CMD insert "cars" [
  documents: [
    [
      _id: #BSON.ObjectId<6283b3614ae06ebf4a8d413a>,
      details: [
        %{color: "blue", type: "sedan"},
        %{color: "orange", type: "sedan"}
      ]
    ]
  ]
] db=259.3ms (module=Mongo function=do_log/5 line=1800 )

{:ok, %Mongo.InsertOneResult{acknowledged: true, inserted_id: #BSON.ObjectId<6283b3614ae06ebf4a8d413a>}}
iex [16:36 :: 3] > Mongo.update_one(conn, "cars", %{"details.color": "blue"}, %{"$set": %{"details.$.color": "red"}})
16:39:04.991 [info] CMD update "cars" [
  updates: [
    %{
      multi: false,
      q: %{"details.color": "blue"},
      u: %{"$set": %{"details.$.color": "red"}}
    }
  ]
] db=58.7ms (module=Mongo function=do_log/5 line=1800 )

{:ok, %Mongo.UpdateResult{acknowledged: true, matched_count: 1, modified_count: 1, upserted_ids: []}}
iex [16:36 :: 4] > Mongo.find_one(conn, "cars", %{})
16:39:23.300 [info] CMD find "cars" [limit: 1, batchSize: 1] db=1.1ms (module=Mongo function=do_log/5 line=1800 )

%{
  "_id" => #BSON.ObjectId<6283b3614ae06ebf4a8d413a>,
  "details" => [%{"color" => "red", "type" => "sedan"}, %{"color" => "orange", "type" => "sedan"}]
}


iam resource policy checker, sns & sqs augmentation

Work in progress, posting for reviews

Coverage decreased (-0.3%) to 80.758% when pulling 324fb6e12579e3438ffbc9f08e642d70306aefcd on kapilt:feature-cross-account-policy into c4a181e6feb69224af79eea76ab4daf0d6e687b0 on capitalone:master.

Coverage decreased (-0.4%) to 80.596% when pulling 2f277615db754524c97e8e227097f254783f3cd4 on kapilt:feature-cross-account-policy into c4a181e6feb69224af79eea76ab4daf0d6e687b0 on capitalone:master.

Coverage decreased (-1.1%) to 79.94% when pulling 8cf684454f1621792e406159eae498330aa239e3 on kapilt:feature-cross-account-policy into c4a181e6feb69224af79eea76ab4daf0d6e687b0 on capitalone:master.

one additional case that needs handling is the s3 vpce/vpc endpoint matching, under StringEquals/StringLike. Afaics, the vpc endpoint policy does allow cross account sharing.
Given we have a basic implementation with some verification, switching out to a more advanced parser might be reasonable to handle additional policy syntax around Condition
Condition

Key

Operator

Coverage decreased (-1.3%) to 80.985% when pulling 8c4964843aa49ab1ca87c662578ac2efb0c5f7de on kapilt:feature-cross-account-policy into a315a75049d8e3e62c304d2f3c053964d07a09d5 on capitalone:master.

Coverage decreased (-1.5%) to 80.87% when pulling 624c49fd980857cd5406d8879623b7ecdb00f334 on kapilt:feature-cross-account-policy into a315a75049d8e3e62c304d2f3c053964d07a09d5 on capitalone:master.


HeaderNav is missing role

The HeaderNav component should have an element with role="menubar" somewhere in its HTML.
Spec for contained menuitem elements:

Authors MUST ensure that menu items are owned by an element with role menu or menubar in order to identify that they are related widgets.

I believe the role attribute was intentionally removed upstream as it caused other accessibility issues: https://github.com/carbon-design-system/carbon/issues/7377#issuecomment-740826351

Should I raise the issue upstream then?

You can, but I think it was intentionally removed because of a11y violations
https://github.com/carbon-design-system/carbon/pull/7462


Change notification icon size

Hi,
thanks for this awesome library !
I was wondering if it was possible to change the size of the icon like so for example
Notification::new()
    .summary("Hello")
    .body("How are you")
    .icon("/path/to/icon.png")
    .icon_size(128) // change the size of the icon
    .show()
    .unwrap();

I haven't tried this specifically, but I don't think the standard mentions this. Besides the implementations vary from kde to gnome to unity etc. From what I know in kde and gnome they are being resized automatically. We might want to consult @h4llow3En about the macOS possibilities

There's nothing here for macOS, the image size is set by the system and can't be changed.

Seems like there is not much we can do on the client side. I'll link the major specifications here, just as a reference. If somebody finds something I'd love a PR though :D
http://www.galago-project.org/specs/notification/0.9/index.html
https://developer.gnome.org/notification-spec/
https://wiki.ubuntu.com/NotifyOSD

Yes the icon size on macOS is fixed, as @kiliankoe mentioned before

I see, personally, I use Awesome Window Manager, and it comes with an API that let you choose the size of the icon in the notification, but I understand that it's far from the standard.
In fact, I don't know how notifications work on other desktop managers, but on Awesome WM, if the image of the icon is 500x500, the size of the image in the notification will be the same, and so, it'll take half my screen, and that's what I want to avoid.
If the icon size is not specified in the standard, maybe I should not try to solve my problem in this, but in the configuration of my Awesome WM.
Anyways, thank you for taking time to answer my questions !


LimitOffsetParams don't work

This way pagination works fine:
from fastapi_pagination import pagination_params

When switching to limit-offset version:
from fastapi_pagination.limit_offset import pagination_params

same code and adjusted request fail with: "Page should be used with PaginationParams"
Request:
http://<IP_ADDRESS>:8080/api/v1/call_records?date_from=2021-01-01&date_till=2021-02-01&limit=50&offset=0

Endpoint:
@router.get('/call_records', response_model=Page[CallRecord], dependencies=[Depends(pagination_params)], tags=['calls'])
@handle_exceptions
async def get_call_records(date_from: datetime.date, date_till: datetime.date):
    records = await core.get_call_records(database, date_from, date_till, None)
    page = paginate(records)
    return page

Can you try to use this example?
from fastapi_pagination.limit_offset import pagination_params, Page
from fastapi_pagination import using_page

using_page(Page)  # set page to use

@router.get('/call_records', response_model=Page[CallRecord], dependencies=[Depends(pagination_params)], tags=['calls'])
@handle_exceptions
async def get_call_records(date_from: datetime.date, date_till: datetime.date):
    records = await core.get_call_records(database, date_from, date_till, None)
    page = paginate(records)
    return page

In case if you want to use another page model you must call using_page function with page model class.
from fastapi_pagination import using_page

using_page(Page)

Nice! Thanks, that worked! Just curious if I missed something in docs...

It's my fault, I didn't add docs how to use limit-offset pagination. I will update docs, thanks for a issue)

I am closing this issue. I have added docs and example for limit-offset based pagination.

@uriyyo I think it would be helpful to add the using_page method in the documentation. Also, is there any reason there is a using_page method, wouldn't it just be better to use the ResponseModel of the route? Happy to send a PR unless you already tried this.

@uriyyo Would we have to use the using_page method to set the Page response mode that is being checked via add_pagination?

@rushilsrivastava  No need to do anything) add_pagination will automatically configure your application.
You can take a look at examples. That's how you should use fastapi-pagination since 0.6.0 version.

Sorry slight typo - meant to say the use_params function to set the custom Params.

With new implementation, every Page class has Params type associated with it.
Basically, Page class has __params_type__ attribute wich is pointing to Params class.
For instance, how to create change default Params:
from typing import TypeVar, Generic

from fastapi import Query

from fastapi_pagination.default import Page as BasePage, Params as BaseParams

T = TypeVar("T")

class Params(BaseParams):
    size: int = Query(500, gt=0, le=1_000, description="Page size")

class Page(BasePage[T], Generic[T]):
    __params_type__ = Params

I just implemented the same come on the latest version (0.6.1) and don't think that the Custom Params (or even default Params for that matter) is being exported to the generated OpenAPI docs. I am using the same code you have shown above.

They must be exported. The idea is to have link between Page and Params. I have just double-checked and generated OpenAPI schema is correct.
Please, correct me in case if I missunderstood your answer.

Hmm, upon further testing it seems that the exporting keeps failing when Uvicorn reloads. Sometime, it exports to OpenAPI schema correctly and is shown on the docs, but most of the time only a few routes are exported. Despite not being exported to the schema, the route still works as expected.
Will report back with code that can reproduce the error.

Upgrading to the latest Uvicorn (0.13.4) fixed this problem for me - thanks for the help!

Your welcome, happy to hear that)


zsbmv & csbmv

I am using the full fortran LAPACK distribution and not the LAPACK distribution that ships with OpenBLAS.
So when  trying to build the LAPACK test suite (3.5.0) and linking it to OpenBLAS, there is a linking error because zsbmv and csbmv which are defined in TESTING/LIN and TESTING/EIG and that conflicts with the implementation of these functions in OpenBLAS.
As far as I can tell these are not standard BLAS functions.
http://www.netlib.org/lapack/explore-html/d6/dee/group__complex__blas__level2.html
Is there a reason they are included in the main library?

I do not think this is netlib/lapack's fault, since a standard BLAS implementation is not expected to have these functions (though to be fair, LAPACK is not a paragon of code cleanliness either, it comes with two identical copies of this function - one for the linear solver tests and another for the eigensolver tests).

I am not sure if OpenBLAS is supposed to be a "standard" BLAS implementation in the sense of providing only what was historically present in a 1990s BLAS. There are numerous extensions in MKL and others as well, which you are probably just as likely to hit eventually. (And I still do not get why you want to build an outdated version of something that comes with OpenBLAS out-of-tree)

There are two different issues. Version and out of tree.
The version thing is not really relevant, because the same thing is true for the latest version of LAPACK.
Why I am building out of tree is because thats how our LAPACK and BLAS build is structured right now,  LAPACK build is independent of the BLAS implementation which can vary from platform to platform. So I am in the process of trying to replace our BLAS build with OpenBLAS, and in the process I need to make sure that LAPACK tests pass on top of whatever BLAS we are using.
At a future date, I'd like to use the LAPACK build from OpenBLAS too, but its just BLAS for now.

I do not yet see a convincing case to remove the OpenBLAS implementations of c/zsbmv after about 10 years (they were present in GotoBLAS2-1.08, but probably not yet in GotoBLAS1), in particular if it is just about getting the LAPACK test code working. IMHO Reference-LAPACK should change the name of their private implementations in EIG and LIN (though that would not work retroactively for your 3.5.0)

Oh I am not asking you to remove it, I was mostly just curious about the reasoning.  I can work around this. Thanks for the background.


Does not detect Active Game

I'm connected to the active game, and the cheat does not detect it.
I also updated it hazedumper, and did not work

.

https://github.com/Blaumaus/le_chiffre/releases/tag/v1.4.10


Prefetch: Remove "variables" field when no variables present

Setting prefetch: true on a query with no variables sends a request with the following field
"variables" => true
Might it be better to either remove that field, or send an empty json object by default instead?
Using prefetch() { return {} } in my implementation at the moment.
cc: @benwilson512 for input on the absinthe-graphql side.

Yeah just to weigh in, this is the only instance where I've seen the "variables" param come in as something other than a json object or a string.


Server side pagination

Is it possible implement pagination from server side?

Yes, have you looked at the lazy loading example here?


Support MDS on workflows and workflow_detail pages

Description

Added support for using MDS on workflows and workflow_detail pages
Update all route path to make it accept dataSourceId when it has truthy value

Issues Resolved
closes #228
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

https://github.com/user-attachments/assets/0c5091fe-4aba-4685-8613-1092f4071861

Could you mark the PR as a draft until you have completed the end-to-end testing? It includes:

Turning on/off these two feature flags to ensure backward compatibility works fine.

# Set the value of this setting to true to enable multiple data source feature.
#data_source.enabled: false
# Set the value of this setting to true to hide local cluster in data source feature.
#data_source.hideLocalCluster: false

Switching between two remote data source clusters to ensure workflow-related data and data source IDs are shown correctly (in url, breadcrumbs.. any possible places).
Proving that the workflow you created when remote data source A was connected can also be found in cluster A.
Since we don't have tests here, try to click on all possible buttons to test all functionalities. For example, the delete and link button on the Manage Workflows page.
Making sure CIs are passing

Not ready for review. Will upload the testing screen recording in some time.

Thanks for the change! Two things I noticed from the screen recording:

I don't see dataSourceId= in the url on new_workflow page when local cluster is selected. Let's add that to keep all pages url in consistent when the feature flag is enabled
Could you click on the delete and link icon button on the Workflows page and to see if they all work as expected?

@saimedhi looking great! Once my remaining comments on public/route_service.ts are addressed and you can confirm all of the functionality is still working as expected, I am happy to approve :)

screen-capture (2).webm

Changes LGTM! One more request to add details in the PR description, including some of the low-level implementation details. See existing merged PRs as an example. This makes it much easier to track down issues that may come up in the future, and keeps a detailed log of what and when changes were made.

Remote data source should be consistent in breadcrumbs. When go back to workflow page, local cluster should be selected instead of selecting the default data source. Could you address this bug?

@jackiehanyang, I will fix it in a followup PR. Thank you.

I will make sure when we go back to workflow page, data source selected will be retained .

Remote data source should be consistent in breadcrumbs. When go back to workflow page, local cluster should be selected instead of selecting the default data source. Could you address this bug?

@jackiehanyang, I will fix it in a followup PR. Thank you.

I will make sure when we go back to workflow page, data source selected will be retained .

screen-capture (3).webm

I see a bug - when you opening flow framework from the side nav, default remote data source should be selected.

Please thoroughly test the following scenario:

when you opening flow framework from the side nav, default remote data source should be selected
when you opening flow framework from a bookmarked url, the data source that's in the url should be selected
when you select a non default remote data source, it should still be selected if you refresh the page, instead of jumping back to the default one.
data source should be consistent through breadcrumbs
data source should be consistent through in-app side nav
considering the project goals and customer perspective, test all possible workflows to ensure that multiple data sources are supported correctly.


icons: Add container for copy-code and code-playgrounds icons.

Fixes: #29165

As discussed on CZO, we should add a container with an appropriate placement which contains the view_in_playground_button and copy_button.
This prevents both buttons from overlapping each other whenever their placement is altered.

Screenshots and screen captures:
Spoiler Code-block ->

Code-Block->

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[x] Explains differences from previous plans (e.g., issue description).
[] Highlights technical choices and bugs encountered.
[] Calls out remaining decisions and concerns.
[] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[x] Visual appearance of the changes.
[x] Responsiveness and internationalization.
[x] Strings and tooltips.
[x] End-to-end functionality of buttons, interactions and flows.
[x] Corner cases, error conditions, and easily imagined bugs.

@timabbott I hope the existing updated logic resembles what this comment says.
Trying to update the tests according to the existing logic so please let me know if I understood the logic right.

Thank you so much for your work on this, @shashank-23002. Closing, as we discussed on CZO, in favor of #29247


feat: swap out smee with homegrown proxy client

Pull Request

Proposed Changes

This gets rid of smee as the preferred local webhook client. This does not require a webhook url anymore in the env but does require setting PUBLIC_ORG
This fixes #100
Readiness Checklist
Author/Contributor

[x] If documentation is needed for this change, has that been included in this pull request
[x] run npm run lint and fix any linting issues that have been introduced
[x] run npm run test and run tests

Reviewer

[x] Label as either bug, documentation, enhancement, infrastructure, maintenance, or breaking

Should this also be removed now? Looks like a dead code path if your not using smee any longer: probot-forwarder.js

@zkoppert Ah yea I was debating leaving the smee client in there in case there's a bug with this one but I now realize that's why we have version control 😄


⚠ Bofrost.de has degraded performance

In ff22e16, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 815 ms

Resolved: Bofrost.de performance has improved in 56f14fc after 6 minutes.


flutter_webview_plugin breaks flutter build aar command

I have tried the Depend on the Android Archive (AAR) approach from the Add2App  documentation. Everything works fine for a sample app, but the flutter build aar command broke as soon as I added all the plugins from my project.
I get the following error
Poojas-MacBook-Pro:metrobikes_flutter pooja$ flutter build aar
Running "flutter pub get" in metrobikes_flutter...                  0.9s
Initializing gradle...                                              0.9s
Resolving dependencies...                                           1.6s
Running Gradle task 'assembleAarRelease'...
Running Gradle task 'assembleAarRelease'... Done                   16.9s
Gradle task assembleAarRelease failed with exit code 1

After some trial-n-error, I realized removing flutter_webview_plugin creates a successful build.
Poojas-MacBook-Pro:metrobikes_flutter pooja$ flutter build aar
Initializing gradle...                                              0.8s
Resolving dependencies...                                           1.4s
Running Gradle task 'assembleAarRelease'...
Running Gradle task 'assembleAarRelease'... Done                   71.2s
Built build/host/outputs/repo.

Now when I try the same source code with the second approach - Depend on the module's source code on the Add2App documentation, the app runs successfully. There were some merged manifest errors during this approach. Even after fixing them, I tried flutter build aar, but I get the following error.
Gradle task assembleAarRelease failed with exit code 1
1. What is the issue with AAR and Webview plugin?
2. Can we get a more descriptive error log during flutter build aar?
Flutter Doctor
Poojas-MacBook-Pro:metrobikes_flutter pooja$ flutter doctor -v
[✓] Flutter (Channel master, v1.8.5-pre.117, on Mac OS X 10.14.6 18G87, locale en-IN)
    • Flutter version 1.8.5-pre.117 at /Users/pooja/flutter
    • Framework revision 83a8a575ee (3 days ago), 2019-08-10 12:38:20 -0700
    • Engine revision ff49ca1c6e
    • Dart version 2.5.0 (build 2.5.0-dev.1.0 f29f41f1a5)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.0)
    • Android SDK at /Users/pooja/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-29, build-tools 29.0.0
    • Java binary at: /Applications/Android Studio 3.5 Preview.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS
    ✗ Xcode installation is incomplete; a full installation is necessary for iOS development.
      Download at: https://developer.apple.com/xcode/download/
      Or install Xcode via the App Store.
      Once installed, run:
        sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
    ✗ CocoaPods installed but not initialized.
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To initialize CocoaPods, run:
        pod setup
      once to finalize CocoaPods' installation.

[✓] Android Studio (version 3.5)
    • Android Studio at /Applications/Android Studio 3.5 Preview.app/Contents
    • Flutter plugin version 38.1.3
    • Dart plugin version 191.8369
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] Connected device (1 available)
    • Android SDK built for x86 64 • emulator-5554 • android-x64 • Android 9 (API 28) (emulator)

! Doctor found issues in 1 category.

I'm not sure if this is the right solution, but I can get an AAR depending on flutter_webview_plugin to build by patching its build script.
In a local copy of https://github.com/fluttercommunity/flutter_webview_plugin/blob/master/android/build.gradle, go to the allprojects / repositories section and add a google() repository.
Without this, flutter build aar will fail in the :flutter:buildPluginReleaseFlutterWebviewPlugin task.  You can see this by passing the -v verbose log flag to the flutter tool.
Debugging further requires extracting some of the gradle and flutter tool command lines invoked while executing buildPluginReleaseFlutterWebviewPlugin.  That task runs flutter build aar within the pub cache directory containing the flutter_webview_plugin sources.  That in turn runs another Gradle task that fails because it can't find some androidx packages in the repository search path.
@blasten is there a way to see the logs from these nested runs of Flutter and Gradle tooling?

@jason-simmons, are you saying that the root cause of this issue is that flutter_webview_plugin's own build rules didn't list out its own transitive dependencies "implementation group: 'androidx.appcompat'" since that dependency seems to be fulfilled from the google maven repository only rather than jcenter? https://stackoverflow.com/questions/57125966/could-not-find-androidx-appcompatappcompat1-0-2
If so, then this seems like a bug that should be filed on https://github.com/fluttercommunity/flutter_webview_plugin for being non-self-sufficient in its own build rule.

Thank you @jason-simmons @xster Closing this issue because it is an error on the plugin's part, and not on flutter's.

is there a way to see the logs from these nested runs of Flutter and Gradle tooling?

@jason-simmons No from the Flutter tool. However, if you run gradle or ./gradlew in the project's directory you can pass -Pverbose=true and that will add the --verbose flag to the nested runs of the tool. (We should fix the tool though)
Jason is correct that AndroidX dependency is available on Google Maven only, so the plugin must add google() to the allProjects -> repository function call.
I'm going to reopen the issue, since it would be great if we can run Gradle in verbose mode (always), but show the logs only if an error occurred. This can help save time while troubleshooting.

The plugin flutter_webview_plugin could not be built due to the issue above.
0      throwToolExit (package:flutter_tools/src/base/common.dart:28:3)
1      buildPluginsAsAar (package:flutter_tools/src/android/gradle.dart:726:7)
2      _asyncErrorWrapperHelper. (dart:async-patch/async_patch.dart:80:45)
3      _rootRunBinary (dart:async/zone.dart:1146:38)
4      _CustomZone.runBinary (dart:async/zone.dart:1039:19)
5      _FutureListener.handleError (dart:async/future_impl.dart:153:20)
6      Future._propagateToListeners.handleError (dart:async/future_impl.dart:692:47)
7      Future._propagateToListeners (dart:async/future_impl.dart:713:24)
8      Future._completeError (dart:async/future_impl.dart:532:5)
9      Future._asyncCompleteError. (dart:async/future_impl.dart:580:7)
10     _rootRun (dart:async/zone.dart:1126:13)
11     _CustomZone.run (dart:async/zone.dart:1023:19)
12     _CustomZone.runGuarded (dart:async/zone.dart:925:7)
13     _CustomZone.bindCallbackGuarded. (dart:async/zone.dart:965:23)
14     _microtaskLoop (dart:async/schedule_microtask.dart:43:21)
15     _startMicrotaskLoop (dart:async/schedule_microtask.dart:52:5)
16     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
17     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:175:5)


[#1956] fix: include roll flavour in damage rolls

Suggested for for issue #1956

I have been looking into making an automation module where I want to take damage types into consideration. I wanted to reuse Item5e.rollDamage() but this seems to omit the flavour texts, would it be possible to add those to the roll?

I wasn't 100% sure on what the best practice is on merging flavors, but currently flavors seem to be appended so I have copied that behaviour. Example, if I have the damage formula 1d8[if uninjured] with damage type fire it would become 1d8[if uninjured fire]. An alternative result I could think of (but did not implement) would be 1d8[fire: if uninjured]
This would be a rather unique case, in 99% of the cases the flavor is simply appended. Example: damage formula 8d6 with damage type fire it would become 8d6[fire].

Is it possible to instead put the flavor somewhere in the internal rolls' data or in the damage roll messages' flags, rather than changing the visuals of every roll for all users?

Would this not be an improvement as is regarding Dice So Nice?
If I use the roll command /roll 1d6[bludgeoning] + 1d8[fire] + 1d10[force] I get the associated dice for their damage types, but if I make this configuration for an item damage roll it does not.
If this is too much of a breaking change I have something in mind to accommodate but I want to check if the current suggestion might be fine.

What about complex damage formulas like 1d8 +<EMAIL_ADDRESS>?
As @krbz999 mentioned, hooking on damage rolls in your module and adding extra information to the message/roll's flags would be the less invasive option.

Hm.. I see what you mean, there seems to be a bug in DamageRoll that deletes flavor texts for ParentheticalTerm. That would need to be fixed in order for this to work.

In a situation where DamageRoll evaluates without removing flavor, I would expect the result to be 1d8 + 1d(@item.level)[force] (or whatever damage type it should be).
However, how would you suggest I hook into it externally? Let us create a fictional item with 2 damage parts

8d6 (fire)
1d8 + 1d3 (force)

Today this evaluates to 8d6 + 1d8 + 1d3 which is not accounting for actor bonuses, ammo, spell scaling and other modules editing the roll like the new Build-a-bonus. I would then somehow need to reverse link the resulting roll to the original damage types which I don't really see how I could do that.
For me I see 2 clear solutions to solve this problem.

It would be added to the dnd5e damage roll, but I am feeling that this will not be the case.
Before I call the damage roll, I can temporary edit the data of the item and add the damage type as a flavor and revert after the roll.

I would like to avoid option 2 as it does feel quite hacky, though it would get the job done but can break with any new version release. I could also just copy paste the roll damage method. While this would make it "stable", that would mean my module would never be able to enjoy the improvements added from the dnd5e system until I notice it and also update so I do not consider this an option.
Allow me to present a more complex scenario. I can fairly easily detect the damage types, while I wouldn't have a clue on how do detect this without the flavor.

I have a magical mace which deals 1d6 + @mod bludgeoning damage & flat 3 (no damage type) & 1d8 fire damage
I have an actor bonus of 1d2
I have used smite at lvl1 which is added through a hook in dnd5e.preRollDamage => 2d8 radiant damage
I have added a situational bonus of 1d12

The result would be the following (assuming the smite bonus uses flavor)

However, how would you suggest I hook into it externally?

Hooks.on("dnd5e.rollDamage", (item, roll) => { /* do things */ });

I think there is an incorrect assumption about operator precedence at work in the above cases. With 1d8 + 1d(@item.level)[force], the flavor is part of the ParentheticalTerm rather than the ultimately resolved Die. The dnd5e system simplifies the above by breaking out the faces number from the intermediate parenthetical, and the flavor is tossed along with it. If you want to associate the "force" flavor with the second Die expression, it should be<EMAIL_ADDRESS>(1d@item.level)[force], or even (1d(@item.level))[force].


Acknowledged delivery: semantics

What semantics should "acknowledged event delivery" have?
An EVENT is to be answered by EVENT_RECEIVED

.. as soon as the event was received (that is potentially even before the user subscription handler fires)
.. after the user subscription handler has ended (regardless whether that user code did raise or not)
.. after the user subscription handler has ended - but only if it ended successfully
.. after the user subscription handler has ended - and the EVENT_RECEIVED contains a flag indicating the success/failure of the user subscription handler

Depending on what we chose for above (what semantics does MQTT has?), we then might want to change the name of the new WAMP message (EVENT_DELIVERED, EVENT_PROCESSED)

The code that needs to be touched is here

So, as far as I see it:
4, I think, doesn't have much use (what does the router do with the message? Does it forward it to the original publisher? What would the publisher then do with it?).
3, I feel, mixes up user code and the WAMP transport. WAMP is responsible for getting your messages there, but I don't think it should care about whether the subscription handler was successful. Again, I'm not sure what it would do -- would the router resend it if it failed? It will probably just make code that isn't written entirely transactionally do bad things. Also, what if the subscription handler never finishes, or finishes in a long time comparatively for QoS?
2 has the same problem about long running things.
1, I think, is the way to go. It follows the semantics of other protocols in that QoS is entirely the transport responsibility and not the application responsibility. MQTT, for example, uses it for transferring 'responsibility' from point to point, it does not take into account what the application on the other end does with it. I think this is the most obvious, as it doesn't bring into the question what a failure means to WAMP -- an Exception could be totally reasonable, and we don't want the QoS resending a packet that is valid on the transport, received on the other end, but is invalid according to the semantics of the application. It is, coincidentally, also the simplest -- you send a EVENT_RECEIVED immediately, so there is low latency (lowering the risk of duplicated packets if the connection goes down in the middle), and it removes the question of what we do on failure (we simply log it as we do already) and what happens if the event handler never ends (doesn't matter, as the ACK has been sent already). It also makes it similar to how QoS2 will have to work -- because there are 2 RTs instead of 1, we then don't have to decide when we send each part of the QoS -- we simply do it as it is received.

I agree with @hawkowl that all WAMP should care about is getting the event there. What happens inside the user code is not our concern, and we shouldn't go there. Acknowledgement that components have successfully executed code based on a received subscription are an application-level concern. This will often involve more than just the subscription handler executing without an error, and components other than the publisher may be the ones concerned with the outcome.

Ok, agreed: PUBLISH/PUBLISHED and EVENT/EVENT_RECEIVED, the latter with the semantics 1) is appropriate and sufficient for QoS 1 (but not QoS 2!).

QoS 2 needs to tie into the app code to achieve the two-phase commit, transactional semantics. Eg think about receiving a "charge event" that deducts money from an account. You want to process such events exactly once, where "exactly once" extends right into the transactional database. A client that really does QoS 2 will need a local persistent storage (better database).

I also think EVENT_RECEIVED is appropriately named. QoS 1 is one mode of assured delivery. QoS 2 another. I have to re-read the MQTT spec parts to see if it is really "transactional messaging" in the hardcore, ACID database sense (there are transactional messaging system .. I just don't know if MQTT is one of them).

Do we have QoS mode option for SUBSCRIBE already? I mean in AB, like in message.SUBSCRIBE.options? We need that too ..

The relevant MQTT spec section about QoS is here

In particular, this:

Figure 4.2 – QoS 1 protocol flow diagram, non normative example
Initiate onward delivery of the Application Message (1)

and the footnote:

(1) The receiver is not required to complete delivery of the Application Message before sending the PUBACK. When its original sender receives the PUBACK packet, ownership of the Application Message is transferred to the receiver.

"complete delivery of the Application Message": I read that as that we do not have to wait for the user code subscription handler to finish - we may do so (it does not say MUST NOT), but why should we? ;)
So think what we agreed to in above for QoS 1 is in line with MQTT QoS 1 semantics.

Interestingly, the MQTT spec has this footnote for QoS 2:

Figure 4.3 – QoS 2 protocol flow diagram, non normative example
The receiver is not required to complete delivery of the Application Message before sending the PUBREC or PUBCOMP. When its original sender receives the PUBREC packet, ownership of the Application Message is transferred to the receiver.

This means, MQTT can not be used for full blown ACID compliant transactional messaging. Which is good, because that would require to make a 2-phase-commit, distributed transaction manager (integrated down to the protocol) available/accesible at the app layer. Which not only sounds complex, but is complex;)

Btw, in case you wonder about the answer to this question: "MQTT - QoS - what does TCP not provide which MQTT does?"
I think this is a question that naturally arises, and the question is formulated nicely.
The answer is in the spec:

4.4 Message delivery retry

When a Client reconnects with CleanSession set to 0, both the Client and Server MUST re-send any unacknowledged PUBLISH Packets (where QoS > 0) and PUBREL Packets using their original Packet Identifiers [MQTT-4.4.0-1]. This is the only circumstance where a Client or Server is REQUIRED to redeliver messages.

and here

Non normative comment
Historically retransmission of Control Packets was required to overcome data loss on some older TCP networks. This might remain a concern where MQTT 3.1.1 implementations are to be deployed in such environments.

That means, we don't need any timers or such (because TCP takes care). It also means: without session resumption, bot QoS 1 and 2 are void.

overcome data loss on some older TCP networks

I am really wondering what they are talking about here. TCP is reliable regardless of underlying OSI 1/2 tech. So there is no "data loss" to overcome .. on what "older TCP networks".
I don't get that part.


Macy.recalculate() performance degrades quickly as the set gets larger

I'm really liking the way Macy's looking so far. Simple API, small size.
But I'm running into an issue while creating an infinite-scrolling site. The more I load, the longer Macy.recalculate() takes. It actually begins to slow down significantly fairly early, with only a hundred or so items.

You may want to consider minimizing layout thrashing with FastDom or a similar implementation.
Or try to track which items have already been calculated, and skip them. And run a full recalculate on resize.

This should be now resolved in v2


make a clear change in readme file

added a dot of the heading of the readme file

I'm not sure what issue this fixes.
Also, I don't think headers require dots at the end in this case.
Closing for now, but feel free to comment


Complete refactor of the ion plugin to use Kotlin and support future versions of IntelliJ.

Creating V2 of the Amazon Ion plugin which provides several improvements to Ion syntax highlighting.
Description of changes:
Complete refactor of the ion plugin to use Kotlin and support future versions of IntelliJ. Bumping the plugin version to 2.0 and make several improvements to Ion Syntax highlighting.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Hi, thanks for the PR! The diff for this is quite extensive and I was wondering whether you would be willing to break it down a bit for easier review. Relatively few of the 300+ files that were updated seem to contain interesting logic changes.
It would be helpful to have separate commits for:

Removing files (both source files and build artifacts) from the old version.
The (presumably automated?) change that was made to add @NotNull and @Overrides to several dozen classes.
Adding all of the new files containing the updated plugin logic.

If you could give a high-level overview of the files in #3, that would also be appreciated. There are dozens of files and unfortunately GitHub likes to alphabetize them, which isn't always the most sensible ordering. Where's the best place for a reader to start?

Hey zslayton@, sure I can break this into smaller PRs as mentioned. Just to be clear, the intention was to just completely wipe the existing plugin code and start over anew.

PR for 1. https://github.com/amzn/ion-intellij-plugin/pull/14

Hey zslayton@, sure I can break this into smaller PRs as mentioned.

I was looking for separate commits, not necessarily full-blown PRs, as that would still allow me/others to filter out a few hundred files as we went to review it.


bug(node): list of number input causes panic

LanceDB version
No response
What happened?
repro:
const data = [
  { id: 1, vector: [0.1, 0.2], list_of_num: [1,2,3] },
  { id: 2, vector: [1.1, 1.2], list_of_num: [4, 5] }
]
const table = await con.createTable("myTable", data)

error:
     Error: internal error in Neon module: called `Option::unwrap()` on a `None` value

Are there known steps to reproduce?
No response

this doesn't seem to be an issue on 0.4.16 of vectordb or @lancedb/lancedb
vectordb
import * as lancedb from "vectordb";
const uri = "data/sample-lancedb";
const db = await lancedb.connect(uri);
const data = [
  { id: 1, vector: [0.1, 0.2], list_of_num: [1,2,3] },
  { id: 2, vector: [1.1, 1.2], list_of_num: [4, 5] }
]
const table = await db.createTable("myTable", data)
console.log(table);

lancedb
import * as lancedb from "@lancedb/lancedb";
const uri = "data/sample-lancedb";
const db = await lancedb.connect(uri);
const data = [
  { id: 1, vector: [0.1, 0.2], list_of_num: [1,2,3] },
  { id: 2, vector: [1.1, 1.2], list_of_num: [4, 5] }
]
const table = await db.createTable("myTable", data)
console.log(table.display());

@changhiskhan are you able to provide a minimum reproducible example?

If we aren't able to reproduce anymore, I think it's fine to close this.


Question about VSCode terminal

VSCode terminal use canvas to show command result.
it's so amazing! i wonder how it's done?
like this?
input command -> native terminal -> output command -> syntax to html -> draw html to canvas

You can check out this blog post I wrote about it for a high-level summary: https://code.visualstudio.com/blogs/2017/10/03/terminal-renderer
Here's the PR that goes into more technical details: https://github.com/sourcelair/xterm.js/pull/938
Essentially this is how it was before:
input -> pty -> output -> create html based on output

And now:
input -> pty -> output -> draw text to canvas based on output


adds config for cypress component tests

part of RHCLOUD-31955
This adds cypress configs for component testing with an example component test.

Preview: https://react-component-groups-pr-component-groups-130.surge.sh


Android - LHN -Drafted messages made on Web did not synced to app

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Issue found when executing PR https://github.com/Expensify/App/pull/11211
Action Performed:

Go to staging.new.expensify.com
Log in with the expensifail account
Make sure you are in default view mode
Make sure you have several pinned chats
Verify the group of pinned chats are sorted alphabetically in the LHN
Pin a new chat
Verify it gets added in the proper alphabetical order in the LHN
Make sure you have draft chats
Verify the group of draft chats are sorted alphabetically in the LHN
Add a new draft to a new report
Verify it gets added in the proper alphabetical order in the LHN
Go back with the same account to Android app
Check your drafted messages

Expected Result:
Drafted messages made on Web should be visible in app
Actual Result:
Drafted messages made on Web did not synced to app
Workaround:
Unknown
Platform:

Where is this issue occurring?

iOS
Android
Desktop App

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: Yes
Email or phone of affected tester (no customers): any
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

This is actually a duplicate of https://github.com/Expensify/App/issues/5536
Both are a feature request to persist draft reportActions in the server (not a bug)


feat(popover): adds sm and lg width modifiers for the popover

Closes #1101
Similar to the modal, the popover now has a default width of 100%, with sm and lg modifiers to modify its max-width. React can pass a width prop to specify a custom width.

Deploy preview for pf-next ready!
Built with commit 5c23b0f02c2143d2bd154bb94164ae25e4686d30
https://deploy-preview-1462--pf-next.netlify.com

@christiemolloy this looks really good. My only concern is that you can use appendTo in react to append the popover to an element. I'm curious if, for example, you have an icon or small text box that you append the popover, too, and if the popover has width: 100%; max-width: [whatever];, then the width of the popover will only be as wide as the width of the thing it's going inside of if the popover is position: absolute and the element it's appended to is position: relative;.  I think this might require testing in react to see how it behaves.
Also I'm wondering if we should remove width: 100%, and follow the behavior of pf3 popovers where the popover is only as big as the content inside of it up to a max-width? We could do something similar to the button component where by default the popover is only as wide as the content inside of it, but you can use a .pf-m-block modifier to make it width: 100%? What do you think? That's likely also a question for design.

@mcoker agreed I had actually made those changes to the tooltip, and I confirmed with @mceledonia that we want the same styles for both. Just need to decide should we have a small max-width as .pf-m-sm and a large max-width as .pf-m-lg,  or one standard max-width on the component.

Just need to decide should we have a small max-width as .pf-m-sm and a large max-width as .pf-m-lg, or one standard max-width on the component.

The pros of having 2 classes are it's more flexible and allows for people to have a big popover. The con is the component now requires a modifier class or react prop (I'm assuming anyways, maybe isSmall or whatever would be the default?). Are there any other considerations. I could go either way with it. It would depend on what users want/expect. I'm inclined to offer a single width just to make it easy, and something like a tooltip/popover will likely run into issues if it's too big, since it's positioned absolutely on the page and has the potential to go off screen easily unless the tooltip is in the middle of the page. FWIW, looks like carbon and bootstrap offer a single width as far as I can tell, and that's 240px and 276px.

Made changes @mcoker . I agree with just having one max-width. @mceledonia mentioned that the popover should wrap to the size of the content in it. And for that to happen it looks like the popover needs position: relative/display: inline-block OR position: absolute . only having position: relative and a max-width doesn't allow the popover to wrap to the content inside of it. Let me know what you think about this.

the popover needs position: relative/display: inline-block OR position: absolute

react sets the position of the popover to absolute. You can see in the popover component on react how that works. Here's a screenshot of what it does currently with with little content.

Does anything need to change with this? the only difference I see is there is no longer a min-width and the old max-width was calc(var(--pf-c-popover--c-button--sibling--PaddingRight) + #{pf-size-prem(300px)});, which accounts for the right-padding of the element that follows the close button. I think your change to remove it is fine if visuals are fine with it, noting that the overall popover width is 300px, but the content is more narrow because of the popover's left/right padding and then the element that follows the close button is more narrow than that.

Thanks for explaining the positioning - makes sense now. I think we should just close this Pull Request since we decided not to have the small and large modifiers like the issue was intended.


Make LocalFileSystem an Optional Feature

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Currently LocalFileSystem is always provided, and this in turn bring in a dependency on tokio and walkdir. There are scenarios were users may not need LocalFileSystem support and/or may not wish to link in tokio.
Describe the solution you'd like

Add an fs feature flag that is enabled by default and feature gates LocalFilesystem and its dependencies. It is unclear if this would constitute a breaking change, and so it might need to wait for a breaking release
Describe alternatives you've considered

Additional context

Hi, after search(query prompt: use tokio) the source code, I found there is no tokio dependency in local.rs
./*ivy-occur counsel-rg "tokio"*:8:./src/client/token.rs:20:use tokio::sync::Mutex;
./*ivy-occur counsel-rg "tokio"*:10:./src/buffered.rs:33:use tokio::io::{AsyncBufRead, AsyncRead, AsyncSeek, AsyncWrite, ReadBuf};
./*ivy-occur counsel-rg "tokio"*:13:./src/buffered.rs:477:    use tokio::io::{AsyncBufReadExt, AsyncReadExt, AsyncSeekExt, AsyncWriteExt};
./*ivy-occur counsel-rg "tokio"*:24:./src/client/mock_server.rs:31:use tokio::net::TcpListener;
./*ivy-occur counsel-rg "tokio"*:25:./src/client/mock_server.rs:32:use tokio::sync::oneshot;
./*ivy-occur counsel-rg "tokio"*:26:./src/client/mock_server.rs:33:use tokio::task::{JoinHandle, JoinSet};
./*ivy-occur counsel-rg "tokio"*:46:./src/throttle.rs:403:    use tokio::time::Duration;
./*ivy-occur counsel-rg "tokio"*:47:./src/throttle.rs:404:    use tokio::time::Instant;
./*ivy-occur counsel-rg "tokio"*:56:./src/upload.rs:25:use tokio::task::JoinSet;
./*ivy-occur counsel-rg "tokio"*:71:./src/local.rs:219:/// a tokio context, this will use [`tokio::runtime::Handle::spawn_blocking`] to dispatch
./*ivy-occur counsel-rg "tokio"*:101:./src/lib.rs:278://! # use tokio::io::AsyncWriteExt;
./*ivy-occur counsel-rg "tokio"*:102:./src/lib.rs:307://! # use tokio::io::AsyncWriteExt;
./*ivy-occur counsel-rg "tokio"*:103:./src/lib.rs:339://! # use tokio::io::AsyncWriteExt;
./*ivy-occur counsel-rg "tokio"*:104:./src/lib.rs:378://! # use tokio::io::AsyncWriteExt;
./*ivy-occur counsel-rg "tokio"*:105:./src/lib.rs:445://! # use tokio::io::AsyncWriteExt;
./*ivy-occur counsel-rg "tokio"*:109:./src/lib.rs:1082:    /// a tokio context, this will use [`tokio::runtime::Handle::spawn_blocking`] to dispatch
./*ivy-occur counsel-rg "tokio"*:113:./src/lib.rs:1295:    use tokio::io::AsyncWriteExt;
./*ivy-occur counsel-rg "tokio"*:124:./src/limit.rs:32:use tokio::sync::{OwnedSemaphorePermit, Semaphore};
./*ivy-occur counsel-rg "tokio"*:125:./src/limit.rs:283:    use tokio::time::timeout;

Here are a few:
https://github.com/apache/arrow-rs/blob/199ce9190cddd139e78601341d842b0ec2c70455/object_store/src/local.rs#L564-L563
https://github.com/apache/arrow-rs/blob/199ce9190cddd139e78601341d842b0ec2c70455/object_store/src/local.rs#L846
Github search fu suggests there are 32 uses: https://github.com/search?q=repo%3Aapache%2Farrow-rs+tokio+path%3A%2F^object_store\%2F%2F+path%3A%2F^object_store\%2Fsrc\%2Flocal%2F&type=code

Thanks for pointing out.
If fs feature in enabled by default, this is not a breaking change for users?


Suggestion: Do not auto-fill that captcha click

The captcha is already in the human mouse pointer movement. If you auto-fill it by machine, it triggers another harder graphical captcha (since mouse-captcha failed), and also need to click "Next".
It's faster (and less disrupting) to just do 2 clicks :)

I am human
Next

What do you mean by auto-filling captcha? Captcha process is not automated. If you provide the extension with an email, it will use that to prefill the username for you and will try to continue. If continuing fails due to captcha, it'll be highlighted with red border since you haven't passed the captcha by the openai.

Ah, ok. I was assuming "I am human" get's auto-clicked, since it gets red.
You are right, I provide email and password.
How do you try to "continue"? Maybe not continuing could help :)

Well, that's the automated part. If you provided email/password, the extension assumes you'd want to speed your process up and clicks on buttons automatically after filling that information. It's not always the case that you see a captcha, so it makes sense to attempt to continue to speed your login flow up.

Please update to the latest recently added support for CAPTCHA automation using either nopecha or 2captcha. Keep in mind that this package will be updated to use the official API as soon as it's released, so things should get much easier over time.
feat: minor improvements to captcha bypass in version
v3.3.7


odo project delete: Confusing message about ServiceCatalog

When deleting project there is a confusing message about ServiceCatalog not being available.
▶ odo project create asdf
 ✓  New project created and now using project : asdf

▶ odo project delete asdf
 ⚠  Unable to access Service Catalog instances, may not be enabled on cluster
? Are you sure you want to delete project asdf Yes
 ✓  Deleting project asdf [6s]
Deleted project : asdf

There should be no such message

/kind bug
/priority medium
/milestone 1.0

This was implemented here: https://github.com/openshift/odo/pull/1809
What should we do with regards to the error message? When we perform application.List we now also check ServiceInstances.

We could drop this into verbose / hidden output.

Yes, both are ok, up to you.
The message is pretty confusing especially for the user that doesn't have service catalog enabled on their cluster and never used or deployed any Service, no suddenly they see some message complaining about Service Catalog when deleting a project.


How to use

Hello friend
I am new to this issue. I want to make a PLC using the STM32F103 chip working with IEC 61131. Could please let us know how can I use your code?

Hi.
Please, see the configuration file: https://github.com/andreili/STM32FX_FM/blob/master/Examples/STM32VLDISCOVERY/inc/stm32_conf.h#L80
All initialization parameters (PLL, etc.) are hardcoded to compile-time for a minimize the binary file size and more perfomances.
But, this code unsupported by me, sorry :(


Fix Docker local setup for docs site local development

Where is the problem?
Our current Docker local setup does not work. Our GitHub workflow that uses the Docker container works, but we cannot develop locally using Docker.
This works in GitHub Actions for link checking: https://github.com/Kong/docs.konghq.com/blob/main/.github/workflows/build-test.yml
Dockerfile: https://github.com/Kong/docs.konghq.com/blob/main/Dockerfile
docker-compose file: https://github.com/Kong/docs.konghq.com/blob/main/docker-compose.yml
What is your suggestion?
Get our Docker local setup working so we can develop within a Docker container.
Also, write up some simple instructions on using Docker locally that we can add to our README.
Code of Conduct and Community Expectations

[X] I agree to follow this project's Code of Conduct
[X] I agree to abide by the Community Expectations

I can try to solve this issue.
/assign


Clones of clones do not inherit sprite only variables

When using the "Create a clone with (variable) set to %s," some sprite specific variables are lost. This happened specifically when making a clone of a clone, and by my estimates they seem to only inherit the non-clone variables. Although not a big deal, this does not seem like intended behavior.

it seems to be because it ends up using sprite name, which then goes back to the original one
https://github.com/TurboWarp/extensions/blob/e521d47f7acc13daea4769345ec17dbc77ba1289/extensions/Lily/ClonesPlus.js#L426-L435
@LilyMakesThings if you are interested


Unable to get usage statistics for workbook

Summary
I cannot use TSC to get the usage statistics for a specific workbook.
I'd like to get the usage statistics for a given workbook.
I can do this through postman, but not TSC.
GET 10az.online.tableau.com/api/-/content/usage-stats/workbooks/some_luid

How do I do this using TSC?
Description
tableau_auth = TSC.PersonalAccessTokenAuth(token_name, token, site_id="some_company")
server = TSC.Server("https://10az.online.tableau.com", use_server_version=True)
with server.auth.sign_in(tableau_auth):
    workbook = server.workbooks.get_by_id('some_luid')
    workbook._get_views_for_workbook() # this no longer works

Could you explain what you mean by "this no longer works"? What did you see before, and what do you get now instead?

I saw that the _get_views_for_workbook method has a parameter, usage,  to get the usage statistics.
I've attempted to get the usage statistics using TSC, but I'm not able to. Am I doing this incorrectly?

Got it - sorry, I totally skimmed over the example url in the first post. I'll have to check with the team that owns the feature on whether they knew/intended that this was broken, before I know how to update the library.

The usage parameter needs to be passed in to the populate_views() method, like this
           server.workbooks.populate_views(sample_workbook, usage=True)             print("\nName of views in {}: ".format(sample_workbook.name))             print([(view.name, view.total_views) for view in sample_workbook.views])
The output I get from this code is
Name of views in JungleBook:
[('AnimalSightings', 44), ('Image Mapper - Jungle Book', 41)]

Any update on this? Any way I could help?

This seems to still not be supported (api/-/content/usage-stats). The query is successful using the REST API but not available via the library. See https://help.tableau.com/current/api/rest_api/en-us/REST/TAG/index.html#tag/Content-Exploration-Methods/operation/UsageStatsService_BatchGetUsage.


[Bug] [dolphinscheduler-common] hive-serde-2.1.0 conflict with hive3.1.2

Search before asking

[x] I had searched in the issues and found no similar issues.

What happened
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/SerDe
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)

What you expected to happen
run sql successful all times

How to reproduce
不是一个稳定复现的问题，但每隔几天出现2-3次
Anything else
No response
Version
1.3.8
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

@caishunfeng
ds with 2.0.5 import hive version is 2.1.0 ,this problem already exists, i suggest upgrade this version, we have a lot customers use ds, but now ds is  not supportted in emr product , tks.

@caishunfeng ds with 2.0.5 import hive version is 2.1.0 ,this problem already exists, i suggest upgrade this version, we have a lot customers use ds, but now ds is not supportted in emr product , tks.

Hi @SbloodyS do you have time to take a look?  I'm not familiar with EMR

Hi @lklong . I did a quick small test in 2.0.5-release. The following sql can be execute successfully through sql node in AWS EMR 6.3.1 which hive version is 3.1.2.
select count(distinct spu) cnt from dws.dws_pub_spu_info_td

Can you check if 2.0.5-release is normal?


add hecm-afrique.net record

Comments
This university started in 1999, but it still hasn't given its students official email addresses with the university's name. So, most students who graduate from there don't have official university email addresses.
Fixed applied after #19202 closed

@ludndev I have to close this request without merging, because you requested to add a non-existent domain to this repository: Host student.hecm-afrique.net not found: 3(NXDOMAIN) Please ensure that you are familiar with this repository rules located at the bottom of this page and submit a new request with the correct information. Thank you for your understanding!


not working

everything running perfectly but bot not working

Clone again and rebuild it , i have patched the issue


redo-sh: 1.2.6 -> 2.0.3

Semi-automatic update. These checks were done:

built on NixOS
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-dot -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-dot --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-dot help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange -V and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange -v and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange --version and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange version and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange -h and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange --help and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifchange help and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifcreate -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifcreate --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifcreate -V and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifcreate -v and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifcreate --version and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifcreate -h and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ifcreate --help and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood -V and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood -v and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood --version and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood version and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood -h and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood --help and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-ood help and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-sources -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-sources --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-sources help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-targets -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-targets --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/redo-targets help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-wrapped -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-wrapped --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-dot-wrapped -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-dot-wrapped --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-dot-wrapped help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped -V and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped -v and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped --version and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped version and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped -h and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped --help and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped help and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifcreate-wrapped -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifcreate-wrapped --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifcreate-wrapped -V and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifcreate-wrapped -v and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifcreate-wrapped --version and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifcreate-wrapped -h and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifcreate-wrapped --help and found version 2.0.3
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-sources-wrapped -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-sources-wrapped --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-sources-wrapped help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-targets-wrapped -h got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-targets-wrapped --help got 0 exit code
ran /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-targets-wrapped help got 0 exit code
found 2.0.3 with grep in /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3
found 2.0.3 in filename of file in /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3

cc @sternenseemann

Success on x86_64-linux (full log)
Partial log (click to expand)
/nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-targets-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/sh"
/nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ood-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/sh"
/nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-always-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/sh"
/nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifcreate-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/sh"
/nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-dot-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/sh"
/nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-sources-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/sh"
/nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-ifchange-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/sh"
/nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3/bin/.redo-stamp-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/sh"
checking for references to /tmp/nix-build-redo-sh-2.0.3.drv-0 in /nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3...
/nix/store/2fsvjrj28vrwn7jymslg7ykmdi03p97z-redo-sh-2.0.3

Failure on x86_64-darwin (full log)
Partial log (click to expand)
Package ‘redo-sh-2.0.3’ in /private/var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/lnl7-mac/pkgs/development/tools/build-managers/redo-sh/default.nix:25 is not supported on ‘x86_64-darwin’, refusing to evaluate.

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowBroken = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowBroken = true; }
to ~/.config/nixpkgs/config.nix.

Success on aarch64-linux (full log)
Partial log (click to expand)
/nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3/bin/.redo-stamp-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/sh"
/nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3/bin/.redo-sources-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/sh"
/nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3/bin/.redo-ood-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/sh"
/nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3/bin/.redo-ifcreate-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/sh"
/nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3/bin/.redo-ifchange-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/sh"
/nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3/bin/.redo-dot-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/sh"
/nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3/bin/.redo-always-wrapped: interpreter directive changed from "/bin/sh" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/sh"
/nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3/bin/.redo-wrapped: interpreter directive changed from "/bin/sh -eu" to "/nix/store/lnws52qanbi5vz8qg24v4ym57nznfndg-bash-4.4-p12/bin/sh -eu"
checking for references to /build in /nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3...
/nix/store/dzyjw787jw7jw4kq5589dhs9p2929kal-redo-sh-2.0.3


[KERNEL-ESTIMATOR] [poig] [2022-11-3]

Team members
Tan Jun Liang
Submission summary
Final changes,
Additional comments
I am not sure why I got latex syntax errors like label 'eq:ms' multiply defined on github notebook render, there is no such problem on qbraid lab, and the same thing appears and doesn't pop up an error.
It would be great If folder management can be better, like a thing that can choose to move or copy to where, so I don't need to spend extra time on it like pulling the file or typing commend.
Thanks for the lab server fixed, much better, even though kernel dead problem still exists, but happened lesser.
Thanks for the Challenge, I enjoy a lot.

@poig Hi there, thank you for your final submission and for your participation in the Hackathon! Your solution to the kernel estimator was amazing. The entire judge's panel was extremely impressed, especially given the fact you were working in a team of one.
We really appreciate the feedback on the qBraid platform as well. If you haven't already, do you mind noting these bugs and your overall thoughts in our exit survey? It helps us a great deal to have all that information in one place.
Most importantly, after seeing your work throughout the two weeks, I would love to open a conversation with you about potential employment opportunities in quantum software development at qBraid, now or in the future. What's the best way to reach you outside of Github?

@ryanhill1 thanks for the amazing hackathon, I enjoy it a lot. you can dm me on discord Jun Liang Tan#4881.


Unable to override avatar_default.png

Gitea version (or commit ref): 1.0.2
Git version: 2.11.1
Operating system: Linux
Database (use [x]):

[x] MySQL

Can you reproduce the bug at https://try.gitea.io:

[x] Not relevant

Description
Using a build without bindata, I'm able to override images like gitea-lg.png and favicon.png, but not avatar_default.png through placing them in custom/public/img.

Do you have Gravatar enabled?

You can overwrite any file the origins from the public/ folder within custom/public/

Got it fixed, it was actually my fault. My STATIC_ROOT_PATH was pointing to a wrong path.