v0.0.21

Notable changes

No-op instead of error on empty read/write (81ac54a)

@nodejs/wasi could somebody review/approve ?


[analyzer] add a fix for a generative constructor for NOT_INITIALIZED_NON_NULLABLE_INSTANCE_FIELD

class A {
  int a;
}

The correction message is

Try adding an initializer expression, or a generative constructor that initializes it, or mark it 'late'.

Currently, NOT_INITIALIZED_NON_NULLABLE_INSTANCE_FIELD is associated with AddLate.
It would be better to also have something like "AddConctructor" or "AddToConstructor".
NOT_INITIALIZED_NON_NULLABLE_INSTANCE_FIELD_CONSTRUCTOR could also be considered

I guess your question is more about the usage.
However about:

Create a constructor

class A {
  int a;
}

to
class A {
  int a;
  A(this.a);
}

or add to existing

class B {
  int a;
  int b;
  B(this.a);
}

to
class B {
  int a;
  int b;
  B(this.a, this.b);
}

or

class C {
  int a = 1;
  int b;
  C(int a) {
    this.a = a;
  }
}

to
class C {
  int a = 1;
  int b = null;
  C(int a, int b) {
    this.a = a;
    this.b = b;
  }
}

Having a new parameter means are references need to be also changed, this can be done by:
a. nothing, we let the user add them
b. add default value (null), which would gives errors for the user to fix
c. have a refactor to ask for the default value
AddMissingRequiredArgument took this step and adds null or empty string for missing ones.


WIP: moshi integration

Hi! Can you explain why you integrate moshi in the project? I discovered that toType function already exists in kohttp-test gradle project, but contains jackson under the hood. What the reasons to use moshi instead of jackson?

Hi! Can you explain why you integrate moshi in the project? I discovered that toType function already exists in kohttp-test gradle project, but uses jackson under the hood. What the reasons to use moshi instead of jackson?

Hey! Different projects use different (de)serialisation libraries and techniques, Moshi and Jackson are just two possible implementations. We don't want to limit our users to any exact one.
Moshi uses codegen-based approach for deserialisation and it's a preferable way for Android. However, Jackson is reflection based.


RFP - glow

Checklist

[x] The package I am requesting does not already exist on https://chocolatey.org/packages;
[x] There is no open issue for this package;
[x] The issue title starts with 'RFP - ';
[x] The download URL is public and not locked behind a paywall / login;

Package Details
Software project URL : https://github.com/charmbracelet/glow/
Direct download URL for the software / installer : https://github.com/charmbracelet/glow/releases/download/v1.4.0/glow_1.4.0_Windows_x86_64.zip
Software summary / short description: Render markdown on the CLI

Added the missing https scheme to the project URL.

Status: Review
https://chocolatey.org/packages/glow

Status: Published
https://chocolatey.org/packages/glow


Connected to WiFi, but won't acknowledge it.

I have a unique WiFi connection because it is a school WiFi (WPA2-Enterprise), so it doesn't seem like the HomeSpan library has the capability to connect to it. In order to get around this, I used a different method to connect to the WiFi, but HomeSpan won't try to use this connection because it is stopped by, "No WiFi credentials found". Is there a way to try and bypass this and get HomeSpan to try and work anyways? Here is the code that I got to connect to the WiFi, and was successfully able to make get requests. (I don't how to get it to look fancy)
#include <WiFi.h>
#include <esp_wpa2.h>
#include <esp_wifi.h>
const char *ssid = "wifi";
#define EAP_IDENTITY "username"
#define EAP_PASSWORD "password"
int status = WL_IDLE_STATUS;
void setup()
{
Serial.begin(9600);
WiFi.disconnect(true);
WiFi.mode(WIFI_STA);
esp_wifi_sta_wpa2_ent_set_identity((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));
esp_wifi_sta_wpa2_ent_set_username((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));
esp_wifi_sta_wpa2_ent_set_password((uint8_t *)EAP_PASSWORD, strlen(EAP_PASSWORD));
esp_wifi_sta_wpa2_ent_enable();
while ( WiFi.status() != WL_CONNECTED) {
WiFi.begin(ssid);
// wait 10 seconds for connection:
delay(10000);

}
// print your WiFi shield's IP address:
Serial.print("IP Address: ");
Serial.println(WiFi.localIP());
}

Sorry, HomeSpan does not presently support WPA-Enterprise.  It's not obvious that HomeKit will work when connected to that type of network, since the router may not permit certain protocols (or it may - hard to know without testing).
As a potential work around (though I have no idea if this will work), you can try adding your code above to the beginning of the sketch to establish a WiFi connection before anything else.  Then, use homeSpan.setWifiCredentials(const char *ssid, const char *pwd); to force HomeSpan to think there are credentials, even though they won't be used.

That worked perfectly. Thank you!


Adding CNAME to examples/web folder to make this config persistent

Description
Adding the CNAME file to the examples web folder, so we don't need to recreate it every time that the examples is deployed.
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[ ] My PR includes unit or integration tests for all changed/updated/fixed behaviors and are passing (See [Contributor Guide]).
[ ] I updated/added relevant documentation (doc comments with ///) and updated/added examples in doc/examples.
[ ] I have formatted my code with ./scripts/format.sh and the Flame analyzer (./scripts/analyze.sh) does not report any problems.
[x] I read and followed the [Flame Style Guide].
[ ] I have added a description of the change under [next] in CHANGELOG.md.
[x] I removed the Draft status, by clicking on the Ready for review button in this PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require Flame users to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (please indicate a breaking change in CHANGELOG.md).
[x] No, this is not a breaking change.

LGTM. is this a github bug where it creates the file on the wrong place via the UI?

Nope, it create in the correct page, but since the site is deployed automatically, and we replace the old site with the new one, we need that file our source folder, otherwise every new deploy will happen without the site.


Don't suggest types that are not accessible

ok I think the suggestions are fixed.
Regarding reporting of multiple errors:

@dsyme this looks supicious. we want do error recovery here, but the error was already reported

This fixes the part of #1548 where inaccessible types where reported

@dotnet-bot test this please

@dotnet-bot test this please

@kevin_c_ransom all PRs have to rebased for the tests to succeed. That's nuget for you

rebased.

1) Failed : FSharp-Tests-Core+TypecheckTests.sigs neg group13
Error running command 'D:\j\workspace\release_ci_pa---866fd2c3\tests\fsharp\..\fsharpqa\testenv\bin\diff.exe' with args 'neg15.err neg15.bsl normalize' in directory 'D:\j\workspace\release_ci_pa---866fd2c3\tests\fsharp\typecheck\sigs'. ERRORLEVEL 1
at TestFramework.checkResult(CmdResult result) in D:\j\workspace\release_ci_pa---866fd2c3\tests\fsharp\test-framework.fs:line 243
at SingleTest.fsdiff@185-2.Invoke(String a, String b) in D:\j\workspace\release_ci_pa---866fd2c3\tests\fsharp\single-test.fs:line 190
at SingleTest.singleNegTest(TestConfig cfg, String testname) in D:\j\workspace\release_ci_pa---866fd2c3\tests\fsharp\single-test.fs:line 195
at FSharp-Tests-Core.TypecheckTests.negGroup(IEnumerable`1 negs) in D:\j\workspace\release_ci_pa---866fd2c3\tests\fsharp\tests.fs:line 1705

fixed!

This file has been added:
tests/fsharpqa/Source/Warnings/Repro1548.dll

I don't think it should be there?

IIRC (it's long time) but that file is needed for the repro. Please take a
look at the corresponding test.
Am 24.11.2016 11:00 nachm. schrieb "Don Syme"<EMAIL_ADDRESS>
This file has been added:
tests/fsharpqa/Source/Warnings/Repro1548.dll
I don't think it should be there?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/visualfsharp/pull/1552#issuecomment-262849289,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADgNOSaZ7CSDSv8yih0NDV-5ccTEFWaks5rBgjwgaJpZM4KAddK
.

Oh yes https://github.com/Microsoft/visualfsharp/pull/1552/files#diff-982e344f51d5cf82b7b82545f89e871dR32

Merged, great to have this in.


Any unused GPIOs?

Hi,
do you know unused GPIOs that could be used to sense a door sensor (a reed switch)?
Had use the T1 in the past, there I found a GPIO that flags my door status to Tasmota.
Now I want to use the T5 for that door (balkony).
PS: I had the T1 mounted sideways, so the buttons show the functions: Upper is open, lower is closing.
For the T5 I plan the same, but that means changing quite some led stuff.
LG, Dieter Fauth

sonoff_T5_Development_Material.pdf
perhaps this can help?


Migration docs

[ ] Migration docs set 1
[ ] Migration docs set 2

Closing this issue as the task is completed. Migration docs were removed from public docs.


Describe how to check for broken links

This is one way to check for broken links,
but I'm happy to adopt something that is
better.

Let's review and merge after https://github.com/NVIDIA-Merlin/NVTabular/pull/1711.  Several broken links are resolved in that PR.

rerun tests

rerun tests

rerun tests

I know that I suggested waiting until another PR was merged, but when Jenkins finally says you can merge, you really, really need to merge.


Large uploads

It would seem like the client has issues with really large uploads (3-4GB). Probably because the FileReader call is attempting to load the entire file into memory at once =)
Currently looking into a nodejs + js solution to handle >4GB uploads to avoid having to rely on HTTP limitations.
Most large upload (chunked uploads) libraries are cumbersome or outright buggy, I love the simplicity of effectiveness of this so far so it would seem that it's the only thing missing =)

I did a quick and dirty test, which seems to work fine, but it really needs a second pair of eyes to determine if it has any issues.
It has only had a few tests with large files but I haven't sanity-checked the logic or put it through more exhaustive scenarios.
It reads the file in chunks the size that are specified and sends the entire chunk at once (to avoid extra complexity).
// note that this function invoked from call/apply, which has "this" binded
function _upload(file, options) {
	options = options || {};

	var self = this;
	var socket = this.socket;
	var chunkSize = this.chunkSize;
	var transmissionDelay = this.transmissionDelay;
	var uploadId = file.uploadId;
	var uploadTo = options.uploadTo || '';
	var fileInfo = {
		id: uploadId,
		name: file.name,
		size: file.size,
		chunkSize: chunkSize,
		sent: 0,
		read: 0
	};

	uploadTo && (fileInfo.uploadTo = uploadTo);

	// check file mime type if exists
	if (self.accepts && self.accepts.length > 0) {
		var found = false;

		for (var i = 0; i < self.accepts.length; i++) {
			var accept = self.accepts[i];

			if (file.type === accept) {
				found = true;
				break;
			}
		}

		if (!found) {
			return self.emit('error', new Error('Not Acceptable file type ' + file.type + ' of ' + file.name + '. Type must be one of these: ' + self.accepts.join(', ')));
		}
	}

	// check file size
	if (self.maxFileSize && self.maxFileSize > 0) {
		if (file.size > +self.maxFileSize) {
			return self.emit('error', new Error('Max Uploading File size must be under ' + self.maxFileSize + ' byte(s).'));
		}
	}

	// put into uploadingFiles list
	self.uploadingFiles[uploadId] = fileInfo;

	// request the server to make a file
	self.emit('start', {
		name: fileInfo.name,
		size: fileInfo.size,
		uploadTo: uploadTo
	});
	socket.emit('socket.io-file::createFile', fileInfo);

	// read file
	var fileReader = new FileReader();
	fileReader.onloadend = function () {
		var buffer = fileReader.result;
		fileInfo.read += buffer.byteLength;

		if (fileInfo.aborted) {
			return;
		}

		self.emit('stream', {
			name: fileInfo.name,
			size: fileInfo.size,
			sent: fileInfo.sent,
			uploadTo: uploadTo
		});
		socket.once('socket.io-file::request::' + uploadId, nextChunk);
		socket.emit('socket.io-file::stream::' + uploadId, buffer);

		fileInfo.sent += buffer.byteLength;
		self.uploadingFiles[uploadId] = fileInfo;
	};

	function nextChunk() {
		if (fileInfo.sent >= fileInfo.size) {
			socket.emit('socket.io-file::done::' + uploadId);
			return;
		}
		if (fileInfo.read < fileInfo.size) {
			var slice = file.slice(fileInfo.read, fileInfo.read + fileInfo.chunkSize);
			fileReader.readAsArrayBuffer(slice);
		} else {
			console.log('Upload complete ' + fileInfo.read + ' - ' + fileInfo.size);
		}
	}

	socket.once('socket.io-file::request::' + uploadId, nextChunk);
	socket.on('socket.io-file::complete::' + uploadId, function (info) {
		self.emit('complete', info);

		socket.removeAllListeners('socket.io-file::abort::' + uploadId);
		socket.removeAllListeners('socket.io-file::error::' + uploadId);
		socket.removeAllListeners('socket.io-file::complete::' + uploadId);

		// remove from uploadingFiles list
		delete self.uploadingFiles[uploadId];
	});
	socket.on('socket.io-file::abort::' + uploadId, function (info) {
		fileInfo.aborted = true;
		self.emit('abort', {
			name: fileInfo.name,
			size: fileInfo.size,
			sent: fileInfo.sent,
			wrote: info.wrote,
			uploadTo: uploadTo
		});
	});
	socket.on('socket.io-file::error::' + uploadId, function (err) {
		self.emit('error', new Error(err.message));
	});
}

Your code looks better than me, nice job!
I didn't thought that someone trying to upload huge binary files, so I needed some tests. If your code works fine, it would be great to merge your code.

Thank you! =)
I'm not very well versed with github but I could possibly work up a better implementation and do a pull request if that's preferable. This transfer method could always be made optional using settings, rather than straight up replacing the current one.

@CloakedAlien That sounds great. What should I call your implementation? Any suggestion?

My resume PR #10 will also support large file uploads. That's why I wanted the resume functionality. I've tested with ~8GB files.

Seems like this issue is now gone. I'm closing this, but you can re-open it if you have any issue!

Hey rico!  I need a solution for uploading 1GB file by breaking into chunks and reasseble into db - not the filesystem at all.  Can I do this with your module?  Thannks


refactor device to make supported more specific

with the below refactor, It's easy to find all supported devices.
Besides when add new supported, just modify the SUPPORTED_DEVISES, it will auto define the detect method like xbox_one?.
Hope it helpful for the project. Thank you.

I like the idea of extracting all requires to their main files (I implemented that on master). But I prefer keeping methods defined instead of using metaprogramming. Other parts of code can't do the same (e.g. platforms) and I like having symmetry between different parts of code.


Possible a mistake in IGListDisplayHandlerTests

Here is where I find might be wrong.
[[self.mockAdapterDelegate reject] listAdapter:self.adapter   willDisplayObject:self.list  atIndex:path.item];

Because I forked to our private repo, and replace IGListAdapterDelegate as follows:
- (void)listAdapter:(IGListAdapter *)listAdapter
  willDisplayObject:(id<IGListDiffable>)object atIndex:(NSInteger)index;

Then a compile error occurs.
Should it be [[self.mockAdapterDelegate reject] listAdapter:self.adapter willDisplayObject:self.object atIndex:path.item]; ?

#605

looks like a bug to me 👍


benefits updated

Closes
Closes #1073

Description

Screenshots

Note to reviewers

Checklist:

[x] I have linked the PR to the correct issue.
[x] I have read the Contribution Guidelines
[x] I have built and tested the changes, and they do not break or show any errors.

@Vaishnavi-Patil2211 please review the pr


Tenant Metrics

Describe the feature
We would like to have more metrics being exported about the current tenant controlled by an operator. Some Metrics t hat would be helpful:

Basic info metric (Tenant Active State, Namespace Quota and used Namespaces, so we can count up how many tenants there are)
Cordoned tenants (Consider the tenant cordoning label, to tell which tenants are cordoned and which are not)
Quota Usage (Evaluate how much of o quota spanned over a tenant is used vs max given quota). Same on namespace basis

These are the most important ones i could think of. Another interesting feature (which most other metric exporter are lacking) ist the possibility to add labels to my resource which then are added as metric label. So le't's say I want to be able to show all tenants of certain customer. I would add the label metrics.clastix.io/customer=a to a tenant cr and then the label customerwith the value a would show up in the metric. This way everyone has much greater flexibility to organize metrics, even if they come from the same controller. Simply if you would check if there's any label matching metrics.clastix.io/* and then register it with the actual metric.
What would the new user story look like?
Doesn't change.
Expected behavior
A clear and concise description of what you expect to happen.

@oliverbaehler thanks for submitting this request. Implementing them is pretty easy and straightforward. Would you like to submit a PR too?

@oliverbaehler any progress on this issue? Do you need for help?

In consultation with @oliverbaehler I would like to take on this topic. However, the implementation does not seem as easy as written by @bsctl . How are metrics implemented in general by capsule and how to add custom metrics?

@prometherion Any idea where to get started?

Hey @adberger, thanks for the ping here!
Since Capsule is built on top of controller-runtime, we already have some controller metrics exported: this is really convenient since we don't have to put in place any additional webserver for metrics exposure, and implement the collector.
I started working on this a few days ago, just pushed on branch issues/451 the work in progress, if you could take a look at it would be perfect so I can share the remarks I found so far.
Let's take the example of summarizing the total overall count of cordoned and active Tenants: I would expect a metric named capsule_tenants_status_count with two labels, such as cordoned and active. Exposing this kind of metric using the prometheus.NewCounterFunc is not possible since we cannot use labels that would do the trick.
However, since you're really active on this topic, would be great to have feedback from you regarding the structure of these metrics: would be bad to have metrics such as capsule_tenants_active_count and capsule_tenants_cordoned_count?
To me, honestly, having two time series is pretty odd, since Prometheus labels are solving this kind of problem.

Hey @adberger, thanks for the ping here!
Since Capsule is built on top of controller-runtime, we already have some controller metrics exported: this is really convenient since we don't have to put in place any additional webserver for metrics exposure, and implement the collector.
I started working on this a few days ago, just pushed on branch issues/451 the work in progress, if you could take a look at it would be perfect so I can share the remarks I found so far.
Let's take the example of summarizing the total overall count of cordoned and active Tenants: I would expect a metric named capsule_tenants_status_count with two labels, such as cordoned and active. Exposing this kind of metric using the prometheus.NewCounterFunc is not possible since we cannot use labels that would do the trick.
However, since you're really active on this topic, would be great to have feedback from you regarding the structure of these metrics: would be bad to have metrics such as capsule_tenants_active_count and capsule_tenants_cordoned_count?
To me, honestly, having two time series is pretty odd, since Prometheus labels are solving this kind of problem.

Thank you very much.
I'll look into it this week and give feedback.

@prometherion I might be doing something wrong but I don't see the metrics yet.
I did https://capsule.clastix.io/docs/contributing/development/#fork-build-and-deploy-capsule with a kind cluster
Then I did kubectl port-forward service/capsule-controller-manager-metrics-service -n capsule-system 8888:8080 and opened http://localhost:8888/metrics in a browser.
Regarding your question: Having two metrics (capsule_tenants_active_count & capsule_tenants_cordoned_count) seems fine for me. You just need different PromQL queries, but the result stays the same.

I might be doing something wrong but I don't see the metrics yet

Are you referring to the custom or basic ones?
Becuase for the latters:
curl -s localhost:8888/metrics | wc -l
1618

I might be doing something wrong but I don't see the metrics yet

Are you referring to the custom or basic ones?
Becuase for the latters:
curl -s localhost:8888/metrics | wc -l
1618

I meant the custom ones

Solved with https://github.com/kubernetes/kube-state-metrics/blob/main/docs/customresourcestate-metrics.md
Example with kube-prometheus-stack Helm Chart (https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack):
kube-state-metrics:
  rbac:
    extraRules:
      - apiGroups: [ "capsule.clastix.io" ]
        resources: ["tenants"]
        verbs: [ "list", "watch" ]
  customResourceState:
    enabled: true
    config:
      spec:
        resources:
          - groupVersionKind:
              group: capsule.clastix.io
              kind: "Tenant"
              version: "v1beta2"
            labelsFromPath:
              name: [metadata, name]
            metrics:
              - name: "tenant_size"
                help: "Count of namespaces in the tenant"
                each:
                  type: Gauge
                  gauge:
                    path: [status, size]
                commonLabels:
                  custom_metric: "yes"
                labelsFromPath:
                  capsule_tenant: [metadata, name]
                  kind: [ kind ]
              - name: "tenant_state"
                help: "The operational state of the Tenant"
                each:
                  type: StateSet
                  stateSet:
                    labelName: state
                    path: [status, state]
                    list: [Active, Cordoned]
                commonLabels:
                  custom_metric: "yes"
                labelsFromPath:
                  capsule_tenant: [metadata, name]
                  kind: [ kind ]
              - name: "tenant_namespaces_info"
                help: "Namespaces of a Tenant"
                each:
                  type: Info
                  info:
                    path: [status, namespaces]
                    labelsFromPath:
                      tenant_namespace: []
                commonLabels:
                  custom_metric: "yes"
                labelsFromPath:
                  capsule_tenant: [metadata, name]
                  kind: [ kind ]

This is definitely gold, thanks for sharing @adberger: could we transform this issue from a code-based feature to a documentation one?
Looking forward to reviewing a PR from you!

@prometherion Sorry, I currently don't have any intention to make a contribution to the documentation of capsule because I'm quite busy at the moment.


Broadcast level loaded event on tech.

This allows consumers to get at custom metadata from the HLS manifest.

Seems reasonable, let's make it more general:
Object.keys(Hls.Events).forEach(function(key) {
  var eventName = Hls.Events[key];
  hls.on(eventName, function(event, data) {
    tech.trigger(eventName, data);
  });
});

Good point. I've gone ahead and updated this to broadcast all events.


[ISSUE #8912] Rebalancing logic shift in POP consumption mode

Which Issue(s) This PR Fixes

Fixes #8912
Brief Description

https://shimo.im/docs/Ee32mZB85xcK87A2
How Did You Test This Change?

Codecov Report
Attention: Patch coverage is 13.79310% with 175 lines in your changes missing coverage. Please review.

Project coverage is 47.50%. Comparing base (66ba456) to head (f214dcf).

Files with missing lines
Patch %
Lines

...e/rocketmq/client/impl/consumer/RebalanceImpl.java
3.00%
95 Missing and 2 partials :warning:

...cketmq/client/impl/consumer/RebalancePushImpl.java
17.50%
32 Missing and 1 partial :warning:

...rocketmq/client/impl/factory/MQClientInstance.java
0.00%
22 Missing :warning:

.../rocketmq/client/impl/ClientRemotingProcessor.java
0.00%
11 Missing :warning:

...g/apache/rocketmq/client/impl/MQClientAPIImpl.java
0.00%
6 Missing :warning:

...ache/rocketmq/broker/client/net/Broker2Client.java
83.33%
2 Missing :warning:

.../java/org/apache/rocketmq/client/ClientConfig.java
50.00%
2 Missing :warning:

...apache/rocketmq/tools/admin/DefaultMQAdminExt.java
0.00%
1 Missing :warning:

...he/rocketmq/tools/admin/DefaultMQAdminExtImpl.java
0.00%
1 Missing :warning:

Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #8923      +/-   ##
=============================================
- Coverage      47.61%   47.50%   -0.12%
+ Complexity     11749    11740       -9
=============================================
  Files           1304     1304
  Lines          91046    91246     +200
  Branches       11672    11710      +38
=============================================
- Hits           43351    43343       -8
- Misses         42353    42561     +208
  Partials        5342     5342

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Why not reduce the computational load on the server side by adding a cache to the broker, instead of adopting a client-side computation approach?

Do you mean the results of the distribution cached on the broker side? When POP consumption mode is enabled, I think there are also a large number of network requests, and each topic subscribed by the corresponding client is a network request, which does not seem to solve the problem completely. Transferring to the client side can unify the rebalancing logic. Because it doesn't seem easy to completely isolate POP and PULL rebalancing logic in the original version, and the initial client execution is ideally local rebalancing

Why not reduce the computational load on the server side by adding a cache to the broker, instead of adopting a client-side computation approach?

And the server cache should also consider the problem of sensing and updating cache in time, and the delay of cache update and rebalancing at the same time


TF-TRT execution context management for multiple threads / streams

This issue is created to discuss and possibly improve execution context management during TF-TRT inference.
The TRT best practices guide says: "A common pattern is incoming requests dispatched to a pool of waiting for worker threads. In this case, the pool of worker threads will each have one execution context and CUDA stream. Each thread will request work in its own stream as the work becomes available. Each thread will synchronize with its stream to wait for results without blocking other worker threads."
Note that we have a different pattern in TF-TRT. Let's consider the implicit batch case with  a single engine and a single execution context. For any given TRTEngineOp we keep a global execution context object in the resource manager. We guard the access to this IExecutionContext by a mutex until we prepare the buffers and equeue the inference job. But the mutex is released when we return from ExecuteTRTEngine, and the inference job might still be enqueued at that time when we release the mutex.
The TRT best practices guide says: to "Create a CUDA stream using cudaStreamCreate for each independent batch and an IExecutionContext for each independent batch." ~I would~ One should interpret this the following way: if we want to enque work on multiple GPU streams then each independent batch of work has to have a separate IExecutionContext object. Alternatively we can use a single stream with a single execution context.
So the current scheduling of TF-TRT inference would require that TRTEngineOp_0 puts all its work always on the same stream. This (while not efficient) would guarantee correctness.
Is it guaranteed that the following call would always return the same stream for the given TRTEngineOP object?
https://github.com/tensorflow/tensorflow/blob/9b53275852f37fb1f48e22ed99237f1baef761c5/tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc#L416-L420
Tagging @bixia1

From how class GPUDeviceContext manipulates its stream_ field, there should be a fixed stream associated with the field for a given GPUDeviceContext class object. Therefore, the above call for the same TRTEngineOp class object should always return the same stream. https://github.com/tensorflow/tensorflow/blob/a8ac8f901a3948a71510dedee258119ff4a77363/tensorflow/core/common_runtime/gpu_device_context.h#L82-L84

Thanks @bixia1 for the answer. One more question: Assume that we have a GPUDeviceContext assigned to TRTEngineOp. Can this context object change during future inference calls, or it remain fixed once it is assigned?

The GPUGeviceContext  comes from here
https://github.com/tensorflow/tensorflow/blob/aae5c218bea680b2c5ef581a4f215501ccd66b4e/tensorflow/core/common_runtime/gpu/gpu_device.cc#L479-L486
which is initialized here and can't be changed for the object
https://github.com/tensorflow/tensorflow/blob/aae5c218bea680b2c5ef581a4f215501ccd66b4e/tensorflow/core/common_runtime/gpu/gpu_device.cc#L359-L371

Thanks Bixia. In general it looks like  that we have a single compute stream for a physical GPU. That ensures that the TRTEngineOp running on that GPU will have correct stream ordering. That was the main question discussed in this issue.
It is a little bit confusing though that the gpu_device_context could be overriden.  I see that when executor calls ComputeAsync, it passes the context through params, that was initialized here. Assuming no one has overwritten the default behavior in GPUDeviceBase which ensures a single stream,  we should be all good.
The issue can be closed. I will refer this issue in the docstring of execution_contexts.


Sign out method with completion block and ideally with a new anonymous user being returned.

After signing out, I am signing in the user anonymously if sign out succeeds.
At first, it would be best if the signing in was done asynchronously with a completion block.
Ideally for me there could also be a sign out method that automatically returns an anonymous user post signing out successfully.
What I do now is the following:
NSError *error;
[[FIRAuth auth] signOut:&error];
if (!error) {
            [[FIRAuth auth] signInAnonymouslyWithCompletion:^(FIRUser *_Nullable user, NSError *_Nullable error) {
...
...

However, I have seen the anonymous sign in fail twice with this code, among tens of successful times, but still... I am wondering if this is because the anonymous sign in happens right after the sign out. Perhaps, I should add a small delay between the two.
Please let me know if I could do something alternative, or if any of my requests could be implemented in the future. Cheers!

You should use an auth change listener.

Yes, this is the best way I think. Thanks @morganchen12 !


Suggested Improvement

The code in MainLayout at present is
    protected void onAttach(AttachEvent attachEvent) {
        RouteRegistry registry = attachEvent.getUI().getInternals().getRouter()
                .getRegistry();

        UI ui = attachEvent.getUI();
        fillSideBar(attachEvent.getUI());
        routesListener = registry
                .addRoutesChangeListener(event -> ui.access(() -> {
                    fillSideBar(ui);
                    if (!isCurrentRouteExists(ui)) {
                        ui.navigate(DeregisteredRouteView.class,
                                ui.getInternals().getActiveViewLocation()
                                        .getPath());
                    }
                }));

    }

This results in a rather untidy view if the remnant of a previous view remain in the address bar.
The suggestion is to provide a direct navigate to the MainView for all remnants.
I have tried the following but it did not work for me.
    protected void onAttach(AttachEvent attachEvent) {
        RouteRegistry registry = attachEvent.getUI().getInternals().getRouter()
                .getRegistry();

        UI ui = attachEvent.getUI();
        fillSideBar(attachEvent.getUI());
        routesListener = registry
                .addRoutesChangeListener(event -> ui.access(() -> {
                    fillSideBar(ui);
                    if (!isCurrentRouteExists(ui)) {
                        ui.navigate(MainView.class);
                    }
                }));

    }

As the result of using 20.0.0.rc1 to fix system fail on stop of a bundle the ugliness of the process is more predominate now when menu items are removed. The current process would confuse users.
Please advise if this issue should be raised elsewhere, but it may be specific to Karaf version only.

Yup, this is just related to this demo. May modifying the Route annotation in DeregisteredView to @Route(value = "unknown", layout = MainLayout.class) would be enough? Then the message would be shown in the "main layout".


What to do about raster tiles?

/cc @rclark is the proper thing here to use tiletype and sniff for the format of each buffer?

Yeah that's what we do, and tiletype is very cheap since it isn't decoding much.

Added tiletype to avoid parsing everything ^


weird behavior w optifine / shaders

the mod works with optifine without any issues but after installing and enabling sildurs enhanced default shaders i get this behavior permanently.
Screenshot taken after disabling shaders and restarting minecraft.

Is there any way of fixing this?

fixed by turning Use VBOs to off


Add StableHLO to TTIR conversion for get_dimension_size OP

Add support for StableHLO to TTIR conversion for get_dimension_size OP, and add unit tests to tt-torch and tt-xla.
Can be const folded in TTIR, no need for TTNN op.

Note: issue still not closed since we are currently not able to test it in tt-xla, since we do not support returning scalars from our compiler to pjrt.


Implements delete_nodes() and clear() for Weviate, Opensearch, Milvus, Postgres, and Pinecone Vector Stores

Description
Implements the delete_nodes() and clear() methods for Weviate, Opensearch, Milvus, Postgres, and Pinecone Vector Stores.
New Package?
Did I fill in the tool.llamahub section in the pyproject.toml and provide a detailed README.md for my new integration or package?
Not a new package.
Version Bump?
Did I bump the version in the pyproject.toml file of the package I am updating? (Except for the llama-index-core package)

[x] Yes
[ ] No

Type of Change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] Added new unit/integration tests
[ ] Added new notebook (that tests end-to-end)
[x] I stared at the code and made sure it makes sense

Suggested Checklist:

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added Google Colab support for the newly added notebooks.
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I ran make format; make lint to appease the lint gods

Hi @logan-markewich,
I have implemented adelete_nodes() and aclear() to PGVectorStore. I also modified the unit tests to test these methods.

Awesome, thanks @jonathanhliu21 !

Congrats on the first PR 🚀


[1.4.2] Import of previous exported identities failed

I´ve exported domains and identities from a previous installation of modoboa (1.3.5) and want to import these files into a new installation with a new database. If I import the domains all entry will be imported but the import dialog will not close and a spinning import will be shown and not ended. If I import identities all mailboxes will be imported and only one alias - nothing more. The interface always show the spinning import.

Hi, can you provide your CSV file please ?

You´ve got mail!

Sadly it doesn´t work with 1.4.4 - I upgraded 1.3.5 to 1.4.4, then I exported domains and identities and then I tried to import the domains and identities via web frontend on a new installation with a fresh database. Domains could imported but identities import stopped after the first alias.


OA: add point type

Add the waypoint type (mavlink command associated with the waypoint) to the points in the OA interface

Did you see the CI failures? http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/PR-12349/1/pipeline
../../src/lib/FlightTasks/tasks/Utility/ObstacleAvoidance.cpp:55:1: error: narrowing conversion of '255' from 'int' to 'bool' inside { } [-Werror=narrowing]

 };

 ^

SITL MC avoidance as well.
http://ci.px4.io:8080/blue/organizations/jenkins/PX4_misc%2FFirmware-SITL_tests/detail/PR-12349/1/pipeline

@dagar We cannot merge this one until there is a mavros update and consequent CI update. Problem is that TRAJECTORY_REPRESENTATION_WAYPOINT changed so the mavlink receiver is dropping all the messages coming from ROS because the definition don't match. It isn't an extention of the message but a proper change since it's still marked WIP in mavlink.
Also updating the mavlink submodule on its own will break the avoidance CI.


How to add a new worker in kubernenes cluster?

I have deployed single node example with https://docs.reana.io/administration/deployment/deploying-locally/ instructions,
Is it possible to extend the nodes?

Hi @arm2arm, yes you can amend the number of nodes dynamically. The concrete instructions depend on how you set up your Kubernetes cluster.  For example, with OpenStack Magnum, you would do something like:
$ openstack coe cluster resize mycluster 10

to add 9 new nodes to the cluster.
After extending the cluster, you would typically label the new nodes to designate their purpose, some to serve infrastructure pods (node 1 is enough), some batch pods (say nodes 2-4), the other job pods (say nodes 5-9). This may require to amend your Helm values, see https://docs.reana.io/administration/deployment/deploying-at-scale/#advanced-deployment-scenarios
If you run DB and MQ services inside the cluster, then we recommend to have dedicated nodes for them for best performance.
For example, when we deploy "extensible" REANA clusters, we typically start as follows: (using 8 CPU, 16 GB RAM node flavours):

1 node labelled reana.io/system=infrastructure that will run the REANA frontend and backend infrastructure services;
1 node labelled reana.io/system=infrastructuredb that will run the PostgreSQL DB service (unless you have some already-existing DB service running outside of the cluster that could be reused without hosting DB yourself);
1 node labelled reana.io/system=infrastructuremq that will run the Rabbit MQ service;
1 node labelled reana.io/system=runtimebatch that will run the user runtime batch workflow orchestration pods (CWL/Serial/Snakemake/Yadage);
1 node labelled reana.io/system=runtimejobs that will run the user runtime job pods (generated by those workflow orchestration pods).

Then you can simply add or remove new runtime batch and job nodes as the user base grows (the last two labels), all the while keeping the infrastructure nodes (and the Helm chart values) the same (the first three labels).

Thanks for the detail instructions.
Currently we don't have cluster rather a single node like  this:
kubectl get nodes --show-labels
NAME                 STATUS   ROLES                  AGE   VERSION    LABELS
kind-control-plane   Ready    control-plane,master   18d   v1.21.12   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress-ready=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=kind-control-plane,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=

Planning to add more VMs as a workers for the reana.
So if I simply keep the every infra in this current singe node and by add  VMs with reana.io/system=runtimebatch and reana.io/system=runtimejobs labels will be safe to proceed.
Probably things will be much easy if we could move to Rancher. Are there any successful stories on  using reana with rancher?

Ah, I see. Nope, if you use Kind, this is good for local developments only. You could set up more nodes with Kind, but they will run inside the same Docker host, so to speak, so it's not really good for scaling up. For a real multi-node deployment, some "real" Kubernetes cluster orchestration should be set up, using e.g. OpenStack Magnum, or OpenShift, or Rancher, or GKE, etc.
Regarding Rancher, we haven't tried lately to deploy REANA-on-Rancher, but there is some prior success in https://github.com/reanahub/reana/issues/357 I assume it should mostly work, e.g. there were no changes necessary to deploy REANA on GKE, besides configuring the shared storage properly and exposing the ports properly. Hopefully it should be smooth with latest Rancher as well...

Good to know that some one was trying Rancher!
I think we can close this issue, solution is:
To deploy reana on the real cluster  not the Kind isolated or so...


Built in YAML grammar left-strips the first character of scalars that are not surrounded by quotes

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.75.0
OS Version: Windows 10 Enterprise 20H2

Steps to Reproduce:

Disable all extensions.
Create a file with the extension *.yaml.
Start typing some YAML. For example:

foo:
  bar: baz
qux:
  - quxx
  - quxxx

Enable Scope Inspector
Select any of the words in the file.

Expected behavior:
The entire word should show up in the scope inspector window.
Actual behavior:
However, the word is missing the first character. For example "baz", will show up as "az". Selecting to the left of "baz', we see only "b".
Changing the file type fixes the behavior. Use any other file type, "txt" or "json", etc. Even including inline JSON in the YAML file, and surrounding the words with quotes fixes the behavior. This appears to be an issue with the json->yaml tree in the grammar.

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.77.3. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!

@VSCodeTriageBot Issue also happens in 1.77.3.

Interesting. @EarthmanT is this causing a problem for you?

@alexr00 I suppose there's a chance that it's not related to any issues that I'm experiencing. I suspect it is causing word completion in my language server to match based starting on the 2nd character of most tokens.

We currently get our yaml grammar from https://github.com/textmate/yaml.tmbundle, but it looks to be unmaintained. Repurposing this issue to track adopting a new yaml grammar.

Tried out the grammar at https://github.com/redhat-developer/vscode-yaml/tree/main/syntaxes, but it has the same problem.

Hello @alexr00
I was thinking of maintaining the Yaml grammar myself
fix all the bugs, add missing features etc
what would be some things I need to look out for/need to do?
licenses etc
I also see that Tree-sitter is coming along
how long would my fork get used before effectively becoming obsolete?
I did notice that the Yaml Tree-sitter grammars seem unmaintaned
https://github.com/microsoft/vscode/issues/207416#issuecomment-1991675411
would you still my TextMate grammar when Tree-sitter arrives?

Hi @RedCMD, we have no timeline for retiring TextMate support, so I expect a new TextMate grammar for yaml would be used for as long as you maintain it and it is better/better maintained than the tree-sitter grammar. This is just a guess, but 3+ years seems like a safe guess.
Each time we switch a built in language to a tree-sitter grammar there will likely be noticeable syntax highlighting differences, so we will likely stick with an existing, well-maintained, TextMate grammar over a tree-sitter grammar for languages where such a TextMate grammar exists.
As you noticed in https://github.com/microsoft/vscode/issues/207416#issuecomment-1991675411, I have concerns about how well the tree-sitter grammar is maintained, so I would definitely be interested in trying out a yaml TextMate grammar that you fork. For adopting tree-sitter grammars, I will start first with the ones that we'll get the biggest gain from, which from https://github.com/microsoft/vscode/issues/207416 is HTML, XML, and Groovy, before moving on to those that are less certain (yaml and ini).
For the license, if you use MIT then VS Code will be good to use it. As for things to look out for, we do have integration tests for grammars in this repo, but you do need to follow all the steps to build the project then run yarn test-extension -l vscode-colorize-tests so if you find a way to make tests in your repo it would likely be simpler, though I do think there's at least one grammar author that runs our tests.

Awesome. Yes, please just message when you think it's ready!

I may have gone a little overboard
instead of forking tmbundle, I kinda just rewrote the entire grammar
the structure of the old grammar couldn't support all the weird rules that YAML has
requiring a redesign
there's still a few little bugs
tho they are exceptions of exceptions
and were also bugs in the old grammar anyway
I haven't set up a automated test yet
but I have 85kb in test files
tho I'll prob have to sort them into valid and invalid YAML code
I've just ran the yarn test-extension -l vscode-colorize-tests command with my extension enabled
it fails on the test.yaml file
1 failing
AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:

because it contains multiple lines of invalid YAML code
(the other YAML files do pass)

@RedCMD is this the right grammar to use? https://github.com/RedCMD/YAML-Syntax-Highlighter/blob/main/syntaxes/yaml.tmLanguage.json

@alexr00 that's the main file yes

I took a look, and I agree that based on the VS Code tests your grammar looks good! There's one place where I see an obvious difference. In your grammar you have the scope string.unquoted.other.out.yaml instead of string.unquoted.plain.out.yaml. Is there a reason for this difference? I can compensate for the change in the built in VS Code themes, but any extension themes that rely on the old value might be broken.

RE the onEnterRules saga https://github.com/microsoft/vscode/issues/209519, I don't know that I would wait for that before adopting the new grammar in insiders, since we don't currently have any onEnterRules for YAML built in.

For automated testing, it looks like there are some automated tests for shellscript  here: https://github.com/jeff-hykin/better-shell-syntax/blob/6d0bc37a6b8023a5fddf75bd2b4eb1e1f962e4c2/commands/project/test#L3-L7

@alexr00 should I move the 1.2.tmLanguage.json grammar into the yaml.tmLanguage.json file?
it would probably help keep compatibility with old bots copying the single file directly from VSCode
(when they should be copying all 5 files now)
or does it not really matter?

just fixed the string.unquoted.plain.out.yaml issue https://github.com/RedCMD/YAML-Syntax-Highlighter/commit/287c71aeb0773759497822b5e5ce4bdc4d5ef2aa

@alexr00 should I move the 1.2.tmLanguage.json grammar into the yaml.tmLanguage.json file?
it would probably help keep compatibility with old bots copying the single file directly from VSCode (when they should be copying all 5 files now) or does it not really matter?

Doesn't matter, I've already updated the script here: https://github.com/microsoft/vscode/pull/219833

My usual approach is to keep new grammars in VS Code insiders for a couple months before releasing them to stable. This means I'll just revert back to the old grammar before release for the July release. We'll see how many issues we get before the August release and assess if it's time to fully commit to the new grammar in time for the August release.

Grammar switchover is in! We plan to release stable today, which means the new grammar should go out in insiders tomorrow.

Currently folding block scalars show up as red

>1
 folding block-scalar
 shows up as red

because VSCode hasn't copied the "unbalancedBracketScopes" section
https://github.com/microsoft/vscode/blob/d4c164e7b85364cfdc24b4db23858abe2ff7194b/extensions/yaml/package.json#L75-L79
"unbalancedBracketScopes": [
	"invalid.illegal",
	"storage.type.tag.shorthand.yaml",
	"keyword.control.flow"
]

and for some reason the indentation is whack?
tabs vs spaces
https://github.com/microsoft/vscode/blob/d4c164e7b85364cfdc24b4db23858abe2ff7194b/extensions/yaml/package.json#L53-L80

Let's release this in the upcoming release.

Marking as verified because the colorization tests pass.


Error Responde JSON

Failed calling User::jsonSerialize()
vendor\laravel\framework\src\Illuminate\Http\JsonResponse.php
Symfony 2.5.10 update

Not enough information, and symfony 2.5 and laravel 4.2 are not supported other than for security fixes.


tvtv.us Internet/Other channels missing

Site
tvtv.us
Description
If you go to tvtv.us, select any US city, and in the channel lineup select Internet/Other you will see that they now have guides for most of the major FAST services.  For example you can select PlutoTV and it will show you all the channels it knows about that are on PlutoTV.  Not all the services/channels have really good guide data (more than just a generic description of the program) but some do go into more detail.  But none of these are currently included in the tvtv.us guide data.
I would be inclined to suggest putting the streaming services in a separate .xml file to keep the main one from becoming any larger.  I made the mistake of doing a test run using that main list this morning and now if I try to do a run, even using a custom channels list with only two or three channels, I get nothing but "ERR: Request failed with status code 429" errors (and no indication of how long I have to wait before trying again).

The Internet/Other channels can be added to the site folder. Please follow How to add a channel to the guide?

Error code 429 = Too Many Requests.
This is an issue on the server side which can be prevented by breaking the xml channel files down into smaller ones.


Add fields to CognitoEventUserPoolsPostConfirmationResponse

Added fields to response model (CognitoEventUserPoolsPostConfirmationResponse) for custom messages, that field exists in serialized json below but did not exists in struct CognitoEventUserPoolsPostConfirmationResponse.
{
"version": "1",
"triggerSource": "CustomMessage_ForgotPassword",
"region": "us-east-1",
"userPoolId": "XXXX",
"callerContext": {
"awsSdkVersion": "aws-sdk-unknown-unknown",
"clientId": "XXXX"
},
"userName": "XXX",
"request": {
"userAttributes": {
"cognito:email_alias"<EMAIL_ADDRESS>"cognito:user_status": "CONFIRMED",
"email"<EMAIL_ADDRESS>"email_verified": "true",
"name"<EMAIL_ADDRESS>"preferred_username": "XXX",
"sub": "XXXXX"
}
},
"response": {
"emailMessage": nil,
"emailSubject": nil,
"smsMessage": nil
}
}
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

de-duping with #217


[Bug] Column comment is missed by show create table statement

Search before asking

[X] I searched in the issues and found nothing similar.

Paimon version
12ef308b73c25e9a7f37d39bea9eb9215cf7495b
Compute Engine
flink 1.17
Minimal reproduce step
Flink SQL> CREATE TABLE table1 (a STRING, b STRING not null, c STRING COMMENT 'from column c');
[INFO] Execute statement succeed.
Flink SQL> show create table table1;
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                                result |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CREATE TABLE catalog_gjli.default.table1 (
a VARCHAR(2147483647),
b VARCHAR(2147483647) NOT NULL,
c VARCHAR(2147483647)
) WITH (
'path' = 'file:/path/to/table1'
)
|
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set
What doesn't meet your expectations?
The comment is missing after executing show create table.
Anything else?
No response
Are you willing to submit a PR?

[X] I'm willing to submit a PR!

Hi, thanks for reporting. This is a known issue of Flink. This PR fix it but it is not in flink 1.17 release. Currently, DESC statement can show the comment (fixed and tested in https://github.com/apache/incubator-paimon/pull/964).

Hi, thanks for reporting. This is a known issue of Flink. This PR fix it but it is not in flink 1.17 release. Currently, DESC statement can show the comment (fixed and tested in #964).

Got it, thanks for your information.


Misleading "Discard All Changes" Icon

Issue Type: Bug
The "discard all changes" icon in the source control module is completely misleading and dangerous. I clicked that button (and impulsively confirmed) thinking it would reload the commit file list and instead it completely wiped out a month of work (because i hadn't committed in that whole period). If a button is going to delete something i would find a trash can a much better option than a rounded arrow, which resembles the reload button of any browser, even if it's backwards. Given the complete mess that can be done in an involuntary delete like this one, i would recommend changing the icon to something like a trash can and having a restore option of some sort in case someone clicks it by mistake.
VS Code version: Code 1.40.0 (86405ea23e3937316009fc27c9361deee66ffbf5, 2019-11-06T17:09:34.601Z)
OS version: Darwin x64 19.0.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz (4 x 3500)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_offsurface_control: disabled_offsurface_synchronization: enabled_onvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (1.22GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (12)

Extension
Author (truncated)
Version

vscode-ant-design-vue-helper
ant
1.0.6

npm-intellisense
chr
1.3.0

vscode-eslint
dba
1.9.1

sepaswitch
Fut
1.0.2

gc-excelviewer
Gra
2.1.32

node-module-intellisense
lei
1.5.0

dotenv
mik
1.0.1

debugger-for-chrome
msj
4.12.1

vetur
oct
0.22.6

material-icon-theme
PKi
3.9.2

vue-vscode-snippets
sdr
1.8.0

JavaScriptSnippets
xab
1.7.2

Sorry you have lost all your work. There's been lots of discussion around this feature, present in many tools, and we do show a very alarming dialog which you must confirm. The dialog mentions exactly what's going to happen, including the word DELETED in all caps, making the decision quite voluntary. I highly recommend saving your work often and having backups. This could have happened with any tool, including command line git.


Add newer version of stats.py & dockerfile!

This PR is still in draft, as it's not tested on a fresh OS and therefore it's not garantueed this one works withoud maybe enabling something on the host OS first.
This PR adds the stats2.py which fixes the cpu usage as this is incorrectly showing the %, while it technically shows the amount of load as x/4 (not x/100%). Basically it shows the load of the past 1 minute. So on a quad core cpu a load of 4 would mean 100% load, and anything higher would lead to 101% or higher. However it's not the current CPU usage in %.
The stats2.py also revamps some command's and data to fit them properly on the display. The way it's setup it should work withoud overlapping text.
Also the icons that were added in monitor.py are resized to accompany the original design.
Lastly, this PR will ignore .vscode folder as this is something you don't want to commit to a upstream repo, a system file to run the script using systemd (easy to run/stop and enable/disable on startup!).
Todo:

[ ] Get feedback on functionality
[x] Verify if this setup work on a brand new 64-bit OS rpi. I may just opt in to re-read the steps from your site and see if toggeling something back that i don't do in the container brings any problems.
[ ] Get a upstream image for way easier and faster setup for end-users (and updates aswell!)
[ ] Write documentation
[ ] (possibly CI/CD? probably not b/c of overkill, unless we can copy paste smth).

As i though! On the host the interface is not exposed out of the box and can be easily enabled by doing:
sudo raspi-config nonint do_i2c 0
I don't think futher testing is needed from my side. I'll await your feedback.

The only thing i notice running this container is that it has a moderate amount of cpu usage compared to other docker containers:

During the evening i have a sleep of 60, i'll see if that changes the cpu usage somehwat.
Changing the nice value has no effect, i assume this has to do that we basically refresh every second. I'll monitor this during the night

one small bug i found, is that before the first day, the command will show the hour minutes on the stats display.
Technically it still tells the time but not what the numbers mean. But after the first day it makes sense again. The w and uptime command both dont'provie anything handy.

I added a different uptime command that now displays 0 days instead.
I think this is the best methode unless we really specifically wanna say the minutes and after minutes the hours.
uptime -p does have that, but i think it won't be worth the time and effort to find out a clever way to display time as:
1 minute
1 hour
1 day
8 days etc.

cpu usage still is a bit weird. But will monitor how it goes.

Noticed timezone is UTC and that may throw some users off. Need to mount or specify timezone.

so, it seems that the traffic between devices and the container is also measured in the network which is interesting :) but explains why there was any network traffic at all.
Will continue to monitor now more properly in the next hour.

so, monitoring the usage i can't make much head or tails b/c grafana somehow changes the amount of % when i zoom or zoom out on the timeline. Not sure why.
I do believe the script uses way more then other services i have running, but i'm not sure how to minimize the cpu usage of a python script really.
That's all what i have to do/say about the script stuff :) i'll await your reply!

would also help if i added the image itself of the stats2.py ofc :)

fixed the uptime for ya!
now displays x min, the first hour it'll show: x hours, after that only x days!

This latest commit makes it so, that on linux the execution bit is turned on.
Aswell as specifying the python3 path, it's as easy as: ./stats.py to run it :)

I've created a new docker repository to work on the docker version - https://github.com/mklements/OLED_Stats_Docker
This will mainly help with keeping the instruction sets separate as I find new users are easily confused when there are multiple "configuration" or OS options.
I've tried the script out on my Pi and got it running without any major issues. I still need to try it on a fresh install and add in any install steps that may be missing. The Pi I ran it on is one that already had my other scripts running in the past so may have libraries already installed.


Capture updates in a specific collection while the app is in the background

Is it working while app in background or not working?

I think it should work, why you asking?

Hi kfir,
I'm working directly to MongoDB atlas using mongo_dart.
Planning to move to realm because

Working while no network connection
Catch update in specific collection while app on background

I would like to be sure it will work before the moving
Thanks
Kobi
On Sun, 6 Jun 2021 at 22:56 Kfir Matityahu @.***> wrote:

I think it should work, why you asking?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kfiross/flutter_mongodb_realm/issues/36#issuecomment-855453651,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARIYHPEQIDRV6KN3FVJ2Z6DTRPHE7ANCNFSM456KFZMA
.

Hi kobi, I see now what you talked about.
At the moment, the package implements very much the MongoDB Realm's services - but the realm sync is not. There, so the package is online-based

You will have to create your own caching
Working on the background should be on your responsibility, but you can execute observations on a collection

Thanks for your clarification
Kobi
On Mon, 7 Jun 2021 at 1:44 Kfir Matityahu @.***> wrote:

Hi kobi, I see now what you talked about.
At the moment, the package implements very much the MongoDB Realm's
services - but the realm sync is not. There, so the package is online-based

You will have to create your own caching
Working on the background should be on your responsibility, but you
can execute observations on a collection

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kfiross/flutter_mongodb_realm/issues/36#issuecomment-855474679,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARIYHPFGF446UZVJ3OUJNP3TRP24FANCNFSM456KFZMA
.


Can the ion-tree-list be populated with data from a database?

Hello,
We tried to populate the ion-tree-list with data from our database but had no luck.
Can You give us please an example, if You have one?
Thanks in advance!
Patrick

@patrickpirzer are you using $http, ng-resource? I still didn't test the library loading data from an endpoint yet, but wouldn't mind to reproduce your case.
Thanks to you!

I am doing this. I got the parent. child relationship in row (recursive in nature) from a database table and have a javascript routine to build a tree and feed into the ion-tree-list library. Simple but the tree is really slow when parsing into the structure. I am looking for a faster way to achieve almost instant response) for an interactive user experience

I did fetch data through webservice & populated tree with JSON & it is working smoothly.


Can't convert my HTML to Inline for some reason...

How can we reproduce this bug?
Write out the HTML (or Inky code) that causes the issue.

Not sure if you figured it out yet but I had to undo a bunch of changes in my code and got it working again.  Maybe there's an issue with your HTML.

Please check if your HTML is valid. You can test this at https://kangax.github.io/html-minifier/


New JS files cause completion error

VSCode Version: 1.6.1
OS Version: Windows 10

Every time I create a new JS file in my project, when I begin editing it, an error is thrown in the TypeScript console and Intellisense completions fail to work in that file. Restarting VS Code fixes the issue.
Steps to Reproduce:

Create project with at least one JS file and Intellisense enabled
Create new empty JS source file (ctrl + N, save)
Begin typing

Stack trace:
[Error - 11:39:35 PM] 'completions' request failed with error.
Error processing request. Cannot read property 'lineNumberToInfo' of undefined
TypeError: Cannot read property 'lineNumberToInfo' of undefined
    at LSHost.lineOffsetToPosition (c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\typescript\server\typescript\lib\tsserver.js:52031:37)
    at IOSession.Session.getCompletions (c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\typescript\server\typescript\lib\tsserver.js:51466:53)
    at Session.handlers.ts.createMap._a.(anonymous function) (c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\typescript\server\typescript\lib\tsserver.js:50859:50)
    at IOSession.Session.executeCommand (c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\typescript\server\typescript\lib\tsserver.js:51729:28)
    at IOSession.Session.onMessage (c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\typescript\server\typescript\lib\tsserver.js:51746:35)
    at Interface.<anonymous> (c:\Program Files (x86)\Microsoft VS Code\resources\app\extensions\typescript\server\typescript\lib\tsserver.js:53940:27)
    at emitOne (events.js:96:13)
    at Interface.emit (events.js:188:7)
    at Interface._onLine (readline.js:232:10)
    at Interface.<anonymous> (readline.js:365:12)

jsconfig.json
{
    "compilerOptions": {
        "target": "ES6",
        "module": "commonjs"
    },
    "exclude": [
        "node_modules/"
    ]
}

@royaldark thank you for opening this issue. I cannot see this happening with the latest build (possibly this was an error in 1.6 but has now been fixed). Can you download the Insiders build and try and reproduce on your end?
Thank you very much!

I am unable to reproduce in 1.7.1, so I'm closing this issue.


I faced issue please I need help

Traceback (most recent call last):
File "/Volumes/Untitled/anaconda3/envs/rvc-webui/lib/python3.10/site-packages/gradio/routes.py", line 437, in run_predict
output = await app.get_blocks().process_api(
File "/Volumes/Untitled/anaconda3/envs/rvc-webui/lib/python3.10/site-packages/gradio/blocks.py", line 1346, in process_api
result = await self.call_function(
File "/Volumes/Untitled/anaconda3/envs/rvc-webui/lib/python3.10/site-packages/gradio/blocks.py", line 1074, in call_function
prediction = await anyio.to_thread.run_sync(
File "/Volumes/Untitled/anaconda3/envs/rvc-webui/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/Volumes/Untitled/anaconda3/envs/rvc-webui/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2106, in run_sync_in_worker_thread
return await future
File "/Volumes/Untitled/anaconda3/envs/rvc-webui/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 833, in run
result = context.run(func, *args)
File "/Users/yousifshakir/Retrieval-based-voice-Conversion-WebUI/infer-web.py", line 477, in click_train
& set([name.split(".")[0] for name in os.listdir(f0_dir)])
FileNotFoundError: [Errno 2] No such file or directory: '/Users/yousifshakir/Retrieval-based-voice-Conversion-WebUI/logs/myvocie/2a_f0'
2023-09-12 11:41:08 | INFO | httpx | HTTP Request: POST http://localhost:7865/api/predict "HTTP/1.1 500 Internal Server Error"
2023-09-12 11:41:08 | INFO | httpx | HTTP Request: POST http://localhost:7865/reset "HTTP/1.1 200 OK"

@RVC-Boss

Please check your /Users/yousifshakir/Retrieval-based-voice-Conversion-WebUI/logs/myvocie/2a_f0 directory exists. By the way will it cause by a typo mistake of myvoice instead of myvocie?

the same issue


Pass sourcemap option from inside PostCSS config file

Hello,
Is it possible to pass the --sourcemap option from inside the PostCSS config file?
This is what I currently have and it fails:
{
	"use": ["autoprefixer", "cssnano"],
	"autoprefixer": {
		"browsers": ["last 2 versions", "> 5%"]
	},
	"cssnano": {
		"sourcemap": true
	}
}

@Francisc sourcemaps can't work only on some plugins. It's global config thing. So move this issue to postcss-cli.

Thank you for the very quick response.
The postcss-cli project is currently working on this so it's not supported yet.
I understand sourcemaps don't work this way, but the advantage (read as temporary solution) of being able to ask cssnano was I could see output after autoprefixer, so before the optimization performed by cssnano.

@Francisc cssnano do not generate sourcemaps. It's just a plugin for postcss which builds ast in string and sourcemaps. Better use script with postcss call or gulp-postcss

Right, makes sense.
Doing postcss --options-here | cssnano > style.css --sourcemap was giving me that quick fix I mentioned. But I understand what you're saying, it was PostCSS doing the sourcemap generation, cssnano was just passing the config to it.

@Francisc Yeah, you just use here two separate cli with two postcss instances.

Got it. Thanks a bunch for helping me understand.


python-preview bug fix for 6-memory-and-embeddings

Describe the bug
fix some codes for running semantic-kernel/samples/notebooks/python/6-memory-and-embeddings.ipynb
memories = await kernel.memory.search_async(memory_collection_name, ask, limit=5, min_relevance_score=0.77)
will returns empty list because there is missing codes in semantic-kernel/python/semantic_kernel/memory/semantic_text_memory.py
and output printing parts should be changed.
To Reproduce
run semantic-kernel/samples/notebooks/python/6-memory-and-embeddings.ipynb
Expected behavior
semantic-kernel/samples/notebooks/python/6-memory-and-embeddings.ipynb runs well
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: Mac M1
IDE: VS code
NuGet Package Version [e.g. 0.1.0]

Additional context
Add any other context about the problem here.

Merged!


Request examples are not generated if serialize as V2.

VERSION:
5.0.0-rc4
STEPS TO REPRODUCE:

Add Examples filter (https://github.com/mattfrear/Swashbuckle.AspNetCore.Filters)
app.UseSwagger(c => { c.SerializeAsV2 = true; });

EXPECTED RESULT:
Examples should show in the swagger doc
ACTUAL RESULT:
Examples don't show.
ADDITIONAL DETAILS
It works fine when not using SerializeAsV2

Isn't this issue for https://github.com/domaindrivendev/Swashbuckle.AspNetCore ?


tooltips title

how can add tooltips a custom title or delete it

There is a function for that in the V2 :
tooltips:{callbacks:{title:function(){return 'My title';}}}
Or something like that

To add, in version 1 you can change the tooltipTitleTemplate


feat: write data length to first compact share

Closes https://github.com/celestiaorg/celestia-app/issues/725
Draft PR b/c still WIP

thanks for posting even tho its a WIP!

I'm encountering a strange off by one error now. The last byte of the transaction isn't in the tx range
$ go test .
--- FAIL: TestTxSharePosition (0.00s)
    proof_test.go:121:
        	Error Trace:	/Users/rootulp/git/rootulp/celestia/celestia-app/pkg/prove/proof_test.go:121
        	Error:      	"j0\xb3\xbb\x7f\x8e\xfc\xaeND\xe2x\x10\xe2Wz\x8brm\x91+R\xeb\x1e\xc5[\x8c~>\u008d\x9dc\x1b¢lu]\x14\xca4\xd8K\xd7\x1a\xb9\xcfܭ)\x9a\xd29YxsÒ\xd8cB$!\x02b\x97Z \xba\xe9\xfd\xa3\xd9I\xb8\xef\xab\x01\xd5Rm\x06R%\x9a\xb0\x91Ti}\xbdr\x00I\xbd\xcf\xf4\xaf2:&\xb3U\x17\xf1\x1d\xd4>\xe3\xfb%\xec\xce\xc2\xca\xc0\xac\x14\xba\xb6\xe2\xaf:ȫ\f\xbcR\xd2\xc0WQ\f?\x9e\x14Tiny\xbc54\x8e@\xb9\x89\xc8\xdfȟ\xf9\x86Xk\xea\xf3\x12\xfc\x02\xa7\x1f\x80\r`}\xb5\x80\xc5we9(\x86G*\a\xec`<\xdf\xddS\xd0=5f\xbf\xcbK\x95\"\xb3\xd6O\x98G\x0e\x8f\x1dg\x02;`ՉߌR\x98\x85\x8fXdW\xdat\x18\xdc@\x9cG\xf6y\xd9\xeaDZײ\xa2\xf7\xa9\xebm\xbf\x83t:\xbbtiv\xcd6" does not contain "5f\xbf\xcbK\x95\"\xb3\xd6O\x98G\x0e\x8f\x1dg\x02;`ՉߌR\x98\x85\x8fXdW\xdat\x18\xdc@\x9cG\xf6y\xd9\xeaDZײ\xa2\xf7\xa9\xebm\xbf\x83t:\xbbtiv\xcd6\xa8"
        	Test:       	TestTxSharePosition
        	Messages:   	many small tx%!(EXTRA prove.startEndPoints={44 44}, int=61)
FAIL
FAIL	github.com/celestiaorg/celestia-app/pkg/prove	1.378s
FAIL

More logging:
rawTx: [31 237 191 8 10 97 179 202 252 23 133 206 138 165 147 154 45 65 11 64 222 33 202 131 196 25 175 251 69 64 254 233 203 216 117 218 173 37 145 219 161 151 230 113 175 31 59 246 52 178 12 10 193 139 30 19 40 186 162 52 7 76 85 106 96 58 0 225 5 69 240 57 253 76 175 51 28 227 239 145 70 151 213 58 37 121 118 14]
rawTxDataForRange: [7 32 208 225 100 249 176 168 55 171 122 205 161 62 52 4 60 147 0 87 228 59 13 86 237 167 22 223 111 14 137 141 40 246 33 84 33 45 46 199 20 30 40 83 217 13 237 161 73 245 200 72 42 87 4 156 47 111 118 3 196 173 113 179 2 15 101 67 203 159 5 237 108 66 179 185 82 165 115 78 233 157 140 94 57 38 172 255 29 128 127 243 220 114 59 231 24 205 101 95 116 93 194 201 53 57 177 205 117 88 159 35 244 29 132 125 60 238 84 222 54 136 148 124 211 101 65 126 95 43 97 230 195 23 219 218 232 161 190 43 23 43 37 99 69 234 47 185 62 158 26 113 58 44 247 68 51 207 52 88 31 237 191 8 10 97 179 202 252 23 133 206 138 165 147 154 45 65 11 64 222 33 202 131 196 25 175 251 69 64 254 233 203 216 117 218 173 37 145 219 161 151 230 113 175 31 59 246 52 178 12 10 193 139 30 19 40 186 162 52 7 76 85 106 96 58 0 225 5 69 240 57 253 76 175 51 28 227 239 145 70 151 213 58 37 121]

the two missing bytes are actually in the next share. This test claims prove.startEndPoints={81 81} but it should have been prove.startEndPoints={81 82}. Either of these are incorrect and I suspect it is 1:

the txSharePosition calculation
splitting the tx data into shares

The txSharePosition calculation looks correct based on https://github.com/celestiaorg/celestia-app/pull/781/commits/bbcfd1677050f3700f918c021f78e2d37220b2cb

Confirmed a bug in transaction splitting.

If its just the proving code that is failing, then I'm ok w/ commenting those tests out for now, as this is blocking the NID implementation of tx inclusion proofs, and we'll have to rewrite/handle those bugs there anywho

There is a bug in transaction splitting logic, will try to split this PR into refactors that can be merged ASAP but this PR isn't ready as is b/c there are a lot of edge cases that I don't think are handled correctly.

Codecov Report

Merging #781 (d6c0c9e) into main (b6698d6) will increase coverage by 0.49%.
The diff coverage is 85.85%.

:exclamation: Current head d6c0c9e differs from pull request most recent head 1452acf. Consider uploading reports for the commit 1452acf to get more accurate results

@@            Coverage Diff             @@
##             main     #781      +/-   ##
==========================================
+ Coverage   23.29%   23.79%   +0.49%
==========================================
  Files          71       71
  Lines        8790     8866      +76
==========================================
+ Hits         2048     2110      +62
- Misses       6570     6581      +11
- Partials      172      175       +3

Impacted Files
Coverage Δ

pkg/shares/split_compact_shares.go
82.21% <81.08%> (-2.04%)
:arrow_down:

pkg/prove/proof.go
83.78% <100.00%> (+1.69%)
:arrow_up:

pkg/shares/parse_compact_shares.go
66.66% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Need to merge in main, resolve merge conflicts, get re-approval, then this should be ready to merge


Developemnt testing docker image

Description
docker image for testing
#657

@khaledyoussef24 please update this branch and let's see if that's still good to merge


python3Packages.tensorflow-bin build failure on x86_64-linux as of e29253ad

Issue description
Build of python3Packages.tensorflow-bin failed on x86_64-linux as of e29253ad9b847873fa8c5d736a45951d67623eef.
build log
Maintainer cc: @jyp @abbradar @cdepillabout
Other cc: n/a
Steps to reproduce

Checkout nixpkgs at commit e29253ad9b847873fa8c5d736a45951d67623eef
Run nix-build -A python3Packages.tensorflow-bin

Technical details
- system: `"x86_64-linux"`
 - host os: `Linux 5.13.0-1017-azure, Ubuntu, 20.04.4 LTS (Focal Fossa), nobuild`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.7.0`
 - nixpkgs: `/nix/store/ywip1di109ygrclx99m3gi6hgqs61jcc-source`

Misc.
This issue was automatically generated by nixpkgs-upkeep.

CI workflow that created this issue.
Internal tag:<PHONE_NUMBER>358758738

Looks like this is an issue with the tensorflow-estimator version:
ERROR: Could not find a version that satisfies the requirement tensorflow-estimator<2.8,~=2.7.0rc0 (from tensorflow-cpu) (from versions: none)
[397](https://github.com/samuela/nixpkgs-upkeep/runs/5842880089?check_suite_focus=true#step:11:397)
ERROR: No matching distribution found for tensorflow-estimator<2.8,~=2.7.0rc0
[398](https://github.com/samuela/nixpkgs-upkeep/runs/5842880089?check_suite_focus=true#step:11:398)

The best course of action to fix this is to upgrade tensorflow-bin to 2.8.0 (currently 2.7). However, there are no macOS wheels for 2.8.0 as far as I can tell (https://github.com/tensorflow/tensorflow/issues/55581).

If we don't hear back from upstream TF, I propose that we just drop Darwin support for the time being. Users can always use the default source build which does have Darwin support anyways.


HTTP/1.1 Pipelining Support

Hi,
HTTP/1.1 supports HTTP Pipelining which can (sometimes) increase the speed of brute forcing. It's implemented in Turbo Intruder. some people have achieved blazingly fast scans with it.
Thanks

I am not sure if this is compensated with HTTP/2 multiplexing. and also HTTP/1.1 pipelining has some problems like associating a specific request with a specific response, not being implemented by most proxy servers, problems with caching, faulty implementations, etc. so I am not sure if I should close this or not.


sheet_to_json() returns a sparse array

This is a request to add an option to indicate that the worksheet should be completely translated into an array, creating all rows instead of ignoring some.
If the original worksheet file has empty rows, it is stored with no references to the missing data. sheet_to_json() will faithfully convert the sheet to an array, omitting any rows that are empty. The resulting array cannot be used to build a table accurately because only the populated rows are available. There currently is no option to create array elements for the empty rows.
For example, if a work sheet's cells are represented by the object:
   { "A1": "Row 0 is populated", "J4": "Row 4 is populated" }
then, sheet_to_json(_ws, { header: "A" }) returns:
   [ { "A": "Row 0 is populated", __rowNum__: 0 }, { "J": "Row 4 is populated", __rowNum__: 3 } ]
I noticed there is a pseudo-member in each array element named __rowNum__ that seems to specify the physical index of the row in the spreadsheet that the array element was derived from, but that's just my guess. If this were documented, I could use it so I could fill in the missing rows in a multi-step process, but I don't want to create a maintenance nightmare by relying on something undocumented.
Would you consider adding a way so all physical rows are represented in the output array? Or document the __rowNum__ reference to be official?
By the way, thank you all very much for the work you've done. This is a useful project.

We will document __rowNum__ behavior (it is official) and will add tests.  In addition, we will add a new option blankrows to both sheet_to_json and sheet_to_csv.  The behavior will be as follows:

sheet_to_csv will default to generating every row.  If blankrows is explicitly false, rows with no content will not be generated.

sheet_to_json when header is not set to 1 will default to skipping blank rows.  If blankrows is true, the blank row objects will be added.  Irrespective of the blank rows setting, __rowNum__ will be available.

sheet_to_json with header:1 will default to generating every row.  If blankrows is explicitly false, rows with no content will not be generated.

We're aiming to release 0.9.6 before the end of the month.
PS: __rowNum__ and the whole sheet_to_json function started from an older utility sheet_to_row_object_array (the alias is still there) contributed by @nathanathan -- incidentally it was our first issue/PR: https://github.com/SheetJS/js-xlsx/pull/1

That's super! Thank you so much for the quick response! I'm very pleased with the news.


[HOLD for payment 2023-01-12] Uploading all-black PNG avatar makes it impossible to see the image

Only recently (after fixing this issue) we starting being able to upload png images as avatars.
Problem
We have a solid black background in the Avatar image crop modal, so when all-black PNGs (that have transparent parts) are uploaded, we can't see where the edge of the image is vs the black background (see vid here). Also, having an all black background is confusing when uploading PNG images, because it like we converted the transparent background to black, though that was fixed in the issue linked ^
Solution
TBD. My idea is to use some kind of checkboard background pattern (I believe this is used in most websites that show pngs) so there's no issue where a solid colored PNG gets completely hidden among the background.

Here's my first attempt, which has solid black & transparent checkboard pattern. This is a big dark in "dark mode" but still works pretty well.
https://user-images.githubusercontent.com/3885503/205261280-ff5af170-0bc2-423c-a376-038a4f2914ab.mov

@shawnborton asking for your thoughts on this, I was thinking we could brighten up the "light" squares (which are dark gray in this instance)

Hmm what if we just didn't use pure black for the BG and used the dark green we use for the app BG? I think I would prefer to avoid adding the checkerboard pattern, especially since this is likely a very uncommon occurrence.

Hmm well to be clear we could make the checkerboard only be visible on the background of PNG images, otherwise the avatar image selected would fill the entire window (no checkerboard). I think it's uncommon that the entire PNG being uploaded is the same as the background (black or even dark green) but I do think it's more common that a part of PNG could match the full background
Also I think it's a bit confusing that the background is solid black or even dark green, but once the PNG loads, the background we show everywhere is gray... So At least those should be consistent, right?

but once the PNG loads, the background we show everywhere is gray...

Can you show me what that looks like?

Ah I see now. Well if that's the background color we show elsewhere in the app, what if we just reuse that same background color for the upload UI? Then we at least have the consistency there.

Ya I think that makes the most sense for now 👍

Okay cool, so let's just do that small change and call it a day?

That's a good plan! Do you think it makes sense to have the light gray background on all screens for light mode & the dark green background on all screens for dark mode? Orrrrr maybe it's fine to have the background light gray always. I can make some mockups if you think that would help :D

Well I guess we don't have light mode right now... so I think for now we can just make the simple change to use the greenIcon color as the BG color for now?

Haha very true, I'll test that out next week if I have time before heading OOO and we'll see how it looks!

Not overdue

PR in review

@strepanier03 assigned you via the Bug label to please help pay out @eVoloshchak as the C+ who reviewed the PR for this issue: https://github.com/Expensify/App/pull/13925
Thanks!

@eVoloshchak - Please apply for the job here and let me know once you do so I can move forward. Thank you!

@strepanier03, applied
The PR wasn't deployed to production yet, the same 1 week regression period applies here too, correct?

No regression, this is just how it has always worked until now

Thank you for applying @eVoloshchak - that is correct. The payment is held until 2022-01-12. I have accepted your application and sent the offer.

Looks like the PR somehow got merged into production twice? 🤷  so marking the latest comments as duplicate & fixing the title

The contract has been paid in Upwork since today is the 13th. Thank you for the review on this one @eVoloshchak!


Shape issues with controlnet_shuffle

Hello, I'm having some trouble getting the majority of ControlNets to work correctly.
So far I've tried depth, canny, lineart_anime, tile, and shuffle, but have only been able to generate a video with shuffle (but the results looked pretty weird, at least for the first part of the video).

Result GIF with shuffle

The error I get is a shape mismatch in the ControlNet results:

Traceback
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /HUGE/Code/animatediff-cli-prompt-travel/src/animatediff/cli.py:363 in generate                  │
│                                                                                │
│   360 │   │   │   │   if int(k) < length:                                                        │
│   361 │   │   │   │   │   prompt_map[int(k)]=model_config.prompt_map[k]                          │
│   362 │   │   │                                                                                │
│ ❱ 363 │   │   │   output = run_inference(                                                        │
│   364 │   │   │   │   pipeline=pipeline,                                                         │
│   365 │   │   │   │   prompt="this is dummy string",                                             │
│   366 │   │   │   │   n_prompt=n_prompt,                                                         │
│                                                                                │
│ /HUGE/Code/animatediff-cli-prompt-travel/src/animatediff/generate.py:443 in run_inference        │
│                                                                                │
│   440 │                                                                                │
│   441 │   seed_everything(seed)                                                                  │
│   442 │                                                                                │
│ ❱ 443 │   pipeline_output = pipeline(                                                            │
│   444 │   │   prompt=prompt,                                                                     │
│   445 │   │   negative_prompt=n_prompt,                                                          │
│   446 │   │   num_inference_steps=steps,                                                         │
│                                                                                │
│ /home/hans/.conda/envs/hans/lib/python3.10/site-packages/torch/utils/_contextlib.py:115 in       │
│ decorate_context                                                                                │
│                                                                                │
│   112 │   @functools.wraps(func)                                                                 │
│   113 │   def decorate_context(*args, **kwargs):                                                 │
│   114 │   │   with ctx_factory():                                                                │
│ ❱ 115 │   │   │   return func(*args, **kwargs)                                                   │
│   116 │                                                                                │
│   117 │   return decorate_context                                                                │
│   118                                                                                │
│                                                                                │
│ /HUGE/Code/animatediff-cli-prompt-travel/src/animatediff/pipelines/animation.py:958 in __call__  │
│                                                                                │
│    955 │   │   │   │   │                                                                         │
│    956 │   │   │   │   │   cur_prompt = get_current_prompt_embeds(context, latents.shape[2])     │
│    957 │   │   │   │   │                                                                         │
│ ❱  958 │   │   │   │   │   down_block_res_samples,mid_block_res_sample = get_controlnet_result(  │
│    959 │   │   │   │   │                                                                         │
│    960 │   │   │   │   │   # predict the noise residual                                          │
│    961 │   │   │   │   │   pred = self.unet(                                                     │
│                                                                                │
│ /HUGE/Code/animatediff-cli-prompt-travel/src/animatediff/pipelines/animation.py:899 in           │
│ get_controlnet_result                                                                            │
│                                                                                │
│    896 │   │   │   │   │   │   mod = torch.tensor(scales).to(device, dtype=cur_mid.dtype)        │
│    897 │   │   │   │   │   │                                                                     │
│    898 │   │   │   │   │   │   add = cur_mid * mod[None,None,:,None,None]                        │
│ ❱  899 │   │   │   │   │   │   _mid_block_res_samples[:, :, loc_index, :, :] = _mid_block_res_s  │
│    900 │   │   │   │   │   │                                                                     │
│    901 │   │   │   │   │   │   for ii in range(len(cur_down)):                                   │
│    902 │   │   │   │   │   │   │   add = cur_down[ii] * mod[None,None,:,None,None]               │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: shape mismatch: value tensor of shape [2, 1280, 1, 8, 8] cannot be broadcast to indexing result of shape [2, 1280, 1, 1, 1]

When I take a look at the shapes, all of the ControlNets that don't work are indeed different from shuffle, for example:
1_controlnet_shuffle
cur_down = [torch.Size([2, 320, 1, 1, 1]), torch.Size([2, 320, 1, 1, 1]), torch.Size([2, 320, 1, 1, 1]), torch.Size([2, 320, 1, 1, 1]), torch.Size([2, 640, 1, 1, 1]), torch.Size([2, 640, 1, 1, 1]), torch.Size([2,
640, 1, 1, 1]), torch.Size([2, 1280, 1, 1, 1]), torch.Size([2, 1280, 1, 1, 1]), torch.Size([2, 1280, 1, 1, 1]), torch.Size([2, 1280, 1, 1, 1]), torch.Size([2, 1280, 1, 1, 1])]
cur_mid =  torch.Size([2, 1280, 1, 1, 1])

1_controlnet_tile
cur_down = [torch.Size([2, 320, 1, 64, 64]), torch.Size([2, 320, 1, 64, 64]), torch.Size([2, 320, 1, 64, 64]), torch.Size([2, 320, 1, 32, 32]), torch.Size([2, 640, 1, 32, 32]), torch.Size([2, 640, 1, 32, 32]),
torch.Size([2, 640, 1, 16, 16]), torch.Size([2, 1280, 1, 16, 16]), torch.Size([2, 1280, 1, 16, 16]), torch.Size([2, 1280, 1, 8, 8]), torch.Size([2, 1280, 1, 8, 8]), torch.Size([2, 1280, 1, 8, 8])]
cur_mid =  torch.Size([2, 1280, 1, 8, 8])

I'm using the exact same images in data/controlnet_image/test/controlnet_tile and data/controlnet_image/test/controlnet_shuffle. They're all 512x512 which is the same resolution that I'm rendering at (I also tried making the ControlNet images 256, but got the exact same result).
The pre-processed images in output/.../00_detectmap look correct for all of the ControlNets, but still only shuffle has the right shape during generation.
Any idea what I might be doing wrong?

My config
{
  "name": "blobcube",
  "path": "models/sd/v1-5-pruned-emaonly.safetensors",
  "motion_module": "models/motion-module/mm_sd_v15.ckpt",
  "compile": false,
  "seed": [
    -1
  ],
  "scheduler": "k_dpmpp_sde",
  "steps": 40,
  "guidance_scale": 20,
  "clip_skip": 2,
  "prompt_map": {
    "0": "sci-fi futuristic digital machinery on a white background, beautiful cyberpunk cartoon illustration"
  },
  "n_prompt": [
    ""
  ],
  "lora_map": {},
  "controlnet_map": {
    "input_image_dir": "controlnet_image/test",
    "max_samples_on_vram": 999,
    "save_detectmap": true,
    "controlnet_shuffle": {
      "enable": true,
      "use_preprocessor": true,
      "controlnet_conditioning_scale": 1.0,
      "control_guidance_start": 0.0,
      "control_guidance_end": 1.0,
      "control_scale_list": []
    },
    "controlnet_tile": {
      "enable": true,
      "use_preprocessor": true,
      "controlnet_conditioning_scale": 1.0,
      "control_guidance_start": 0.0,
      "control_guidance_end": 1.0,
      "control_scale_list": []
    },
    "controlnet_depth": {
      "enable": false,
      "use_preprocessor": true,
      "controlnet_conditioning_scale": 1.0,
      "control_guidance_start": 0.0,
      "control_guidance_end": 1.0,
      "control_scale_list": []
    },
    "controlnet_lineart_anime": {
      "enable": true,
      "use_preprocessor": true,
      "controlnet_conditioning_scale": 1.0,
      "control_guidance_start": 0.0,
      "control_guidance_end": 1.0,
      "control_scale_list": []
    }
  },
  "upscale_config": {
    "scheduler": "k_dpmpp_sde",
    "steps": 20,
    "strength": 0.5,
    "guidance_scale": 10,
    "controlnet_tile": {
      "enable": true,
      "controlnet_conditioning_scale": 1.0,
      "guess_mode": false,
      "control_guidance_start": 0.0,
      "control_guidance_end": 1.0
    },
    "controlnet_ref": {
      "enable": true,
      "use_frame_as_ref_image": false,
      "use_1st_frame_as_ref_image": true,
      "ref_image": "none",
      "attention_auto_machine_weight": 1.0,
      "gn_auto_machine_weight": 1.0,
      "style_fidelity": 0.25,
      "reference_attn": true,
      "reference_adain": false
    }
  }
}

Just an example of a controlnet image from data/controlnet_image/test

And the preprocessed versions from output/.../00_detectmap:
lineart_anime:

shuffle:

tile:

Thanks for the report.
Could you zip and paste data/controlnet_image/test?

Hi, thanks for your help.
Here's a zip with my data/controlnet_image/test and config: my_controlnet_image_test_and_config.zip
The command I'm running is:
animatediff generate -c config/prompts/prompt_travel_blobcube.json -W 512 -H 512 -L 15 -C 15

Looking at it now, I think I understand why the results of the GIF above were so strange in the first frames. The GIF was with -L 128, so only the first 15-20 frames were getting ControlNet guidance.
Maybe the shuffle ControlNet wasn't working well for that model? Or the ControlNet guidance scale was too high? Or maybe, even though it's shape doesn't cause an error, there is an issue with the shuffle ControlNet as well?

Thanks. I was able to reproduce it in my environment.
It seems that "controlnet_shuffle" is not working properly.
And the start of the frame starts from 0. I think that's why the first frame is wrong.

Just to clarify, while I think there might also be an issue with 'controlnet_shuffle', the other ControlNets crash for me.
If I enable only 'controlnet_canny', for example, then video generation fails completely with the traceback up above.
RuntimeError: shape mismatch: value tensor of shape [2, 1280, 1, 8, 8] cannot be broadcast to indexing result of shape [2, 1280, 1, 1, 1].
Do you want me to open a separate issue for this?

As for shuffle, it seems that the diffusers implementation does not support multi controlnet, so it seems to be sort of expected that it will fail, but it is strange that canny does. I just tried it, and in my environment, if only canny is enabled, it succeeds.

Aha, you're right!
I thought I had tried the others on their own but I guess I've always left 'shuffle' on because it seemed to work.
When I run without 'shuffle' the others work fine, but as soon as I enable 'shuffle' together with another ControlNet it crashes.
Thanks for your help!
For now just ignoring 'shuffle' is a fine solution for me.


Possible to only invalidate a specific DayViewDecorator

For example, I have 10 decorators attached to my calendar view.
I only want to invalidate the first decorator; I don't want to invalidate the others cos it will it make it slow.

@iad24
you can directly hold specific docorator like:
DayViewDecorator myDecorator = ...
calendarView.addDecorator(myDecorator)
calendarView.addDecorators(second,third,...tenth)

then you can remove existing one and add again (how invalidate works :) )
calendarView.removeDecorator(myDecorator)
myDecorator = ...//invalidate here
calendarView.addDecorator(myDecorator)


tf.autograph.experimental.Feature

URL(s) with the issue:
https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/autograph/experimental/Feature
Description of issue (what needs changing):
Clear description
There is no description provided.
Parameters defined
Parameters are undefined.
Returns defined
Return values are not defined.
Raises listed and defined
Errors are not defined?
Usage example
There is not a usage example.

experimental.Feature is not a function, so most of those fields don't apply. I'll send a CL to clarify in that it's an enumeration type and show an example.


Updated documentation links

I've fixed two links that were incorrect, or have been moved since first publishing this documentation.

Can someone please review this pull request? Let me know if there's anything you need from me. Thanks 🙂

@sethbergman looks like you did a "merge" to update the branch, not a rebase, causing some merge commits to get into this branch; could you try doing a rebase instead? (let me know if you need help doing so)

(changes itself LGTM (not a maintainer))