Generic parallel sum

@b-kloss @emstoudenmire

Codecov Report

Merging #14 (4f642a9) into main (cef9301) will increase coverage by 16.39%.
The diff coverage is 40.42%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##            main      #14       +/-   ##
==========================================
+ Coverage   0.00%   16.39%   +16.39%
==========================================
  Files          7        7
  Lines        226      244       +18
==========================================
+ Hits           0       40       +40
+ Misses       226      204       -22

Impacted Files
Coverage Δ

src/broken/distributedsum.jl
0.00% <ø> (ø)

src/mpi_extensions.jl
0.00% <ø> (ø)

src/mpisum.jl
0.00% <ø> (ø)

src/parallelsum.jl
40.42% <40.42%> (ø)

src/partition.jl
100.00% <0.00%> (+100.00%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Cache integration

Hi, as a follow up of this discussion https://github.com/langchain-ai/langchain/discussions/19334 this is the PR for the cache layer class for LLM chats on top of Elasticsearch.
For the documentation, the page https://github.com/SpazioDati/langchain/blob/master/docs/docs/integrations/llms/llm_caching.ipynb seems the right place for this integration. I am going to open a PR for the docs after this is merged, if it is ok for the maintainers.

Nice, we are getting there. Just a couple more linting errors. You can run make lint_package locally to see which ones need to be fixed so you don't have to wait for me triggering CI (which is the default setting for first time contributors).

sorry again, I was executing the wrong commands, lesson learned!

We are going to release a new version with your changes shortly.
@giacbrd Would you be interested in writing a blog post on Elastic Search Labs about how you are making use of the Elasticsearch integration in LangChain in your projects at SpazioDati? It would be nice to see you talk about how this is useful for you and your project could get some exposure through our blog. What do you think?

you are welcome @maxjakob ! do I have to immediately open a PR for the documentation on the main repo of LangChain? for documenting this cache integration along with many others in the page https://github.com/SpazioDati/langchain/blob/master/docs/docs/integrations/llms/llm_caching.ipynb

open a PR for the documentation on the main repo of LangChain?

Yes that would be great, thanks a lot.

We are going to release a new version with your changes shortly.
@giacbrd Would you be interested in writing a blog post on Elastic Search Labs about how you are making use of the Elasticsearch integration in LangChain in your projects at SpazioDati? It would be nice to see you talk about how this is useful for you and your project could get some exposure through our blog. What do you think?

Yes, we really appreciate this opportunity. We would also like to open a second PR with the integration for CacheBackedEmbeddings described here https://github.com/SpazioDati/llm-elasticsearch-cache?tab=readme-ov-file#embeddings-cache-usage . We may write the article after this PR is accepted, is this ok for you?

Yes, we really appreciate this opportunity. We would also like to open a second PR with the integration for CacheBackedEmbeddings described here https://github.com/SpazioDati/llm-elasticsearch-cache?tab=readme-ov-file#embeddings-cache-usage . We may write the article after this PR is accepted, is this ok for you?

Sound good, go for it!

Yes, we really appreciate this opportunity. We would also like to open a second PR with the integration for CacheBackedEmbeddings described here https://github.com/SpazioDati/llm-elasticsearch-cache?tab=readme-ov-file#embeddings-cache-usage . We may write the article after this PR is accepted, is this ok for you?

Sound good, go for it!

Hi @maxjakob ,
We are writing the blog article about LangChain and ES in SpazioDati, we hope to finalize it soon. I'll write you when it is ready


Zsh 5.4 twice as fast, did anyone notice?

Not sure if Zsh startup time is a problem with Oh-My-Zsh. However frameworks want to grow and increase their feature-set, so gradual slow down may occur. I have good news: Zsh 5.4 includes optimizations that divide Zsh startup time by 2. I wonder if any Oh-My-Zsh user noticed this?
Here is link to Zsh mailing list discussion that resulted in the optimization.

Not sure if already implemented but I think async prompt would make a much more noticeable difference to user.

Closing because it's not an issue. The conversation can continue though 😉


Add configuration option to bypass calls to Create___IfNotExists()

Is your feature request related to a problem? Please describe.
The DefaultCosmosContainerService calls both CreateDatabaseIfNotExists() and CreateContainerIfNotExists() during startup. While this implementation is acceptable for hosts with long running processes, such as hosting in IIS and Azure App Service, in Azure Functions it is executed on every function call. The result is an additional 50-60ms to calls that take only ~50ms. This essentially doubles the time it takes for every call to complete.
You can verify the results by creating a new CosmosDb database and executing some inserts. Then, replace the relevant lines in GetContainerAsync() with the following:
var database = await _cosmosClientProvider
    .UseClientAsync(client => Task.FromResult(client.GetDatabase(_options.DatabaseId)))
    .ConfigureAwait(false);

var container = database.GetContainer(containerId);

Yes, this is an edge case. And yes, one could easily provide their own implementation of ICosmosContainerService that does not call the CreateIfNotExists() method, but it'd be a lot cooler if you didn't have to go to all that trouble.
Describe the solution you'd like
Add a configuration option to RepositoryOptions that allows one to choose if they want the CreateIfNotExists behavior.
Describe alternatives you've considered
Creating a different implementation of ICosmosContainerService and registering that into the DI container instead of DefaultCosmostContainerService

Hey @cjbanna how are you using this in your functions? Are you using a functioning startup? As described here -> https://docs.microsoft.com/en-us/azure/azure-functions/functions-dotnet-dependency-injection
Would you be able to share your .AddCosmosRepository(options => ...) for a little more context?

Thanks for the reply. I didn't follow the guidance in the project. After following the example in the Azure Functions project, it works as expected. Closing this issue.


[BUG] nfs Backup Storage high CPU Usage, slow data throughput.

Describe the bug
i created an NFS4.x share on a Windows 2019 server to create backups with Longhorn.
A backup takes a long time, because Longhorn causes a high CPU (16 Core K8s Node 88%) load and has a low data throughput (avg 39Mbps).
The NFS storage on the Windows 2019 server is generally fast, we use this for data backup via BackupExec / VMware without any problems of this kind.

To Reproduce
Steps to reproduce the behavior:

Create an NFS backup target
Generate an NFS backup from a volume

Expected behavior
Fast backup, well over 100 Mbps and significantly less CPU load.
Environment:

Longhorn version: v1.1.0
Installation method: Rancher Catalog App
Kubernetes distro: RKE 2.5.7

Number of management node in the cluster: 2
Number of worker node in the cluster: 3

Node config

OS type and version: openSUSE Leap 15.2
CPU per node: 16
Memory per node: 142GiB
Disk type: SAN Storage FC RAID 10
Network bandwidth between the nodes: 1GiB

Underlying Infrastructure: Baremetal
Number of Longhorn volumes in the cluster:

Hi, can you try to use a Linux NFS server and see if this issue can be reproduced as well? It would help a lot for narrowing down the issue.

I can probably explain the high CPU usage with the corrupt pvc https://github.com/longhorn/longhorn/issues/2553.
But the few / slow data throughput of 35Mbit/sec. remains, regardless of whether Linux NFS or Windows NFS.
An 8GB snapshot takes 8 minutes here. If i copy an 8GB image file from the same node to this NFS share, i have a data throughput of almost 1Gbit, the image is here in 1.5min. copied.
Apparently it takes so long to take the snapshot itself, or is there a reason for this?

Do you mean the speed of directly writing data to the NFS server is 1Gb/s but the speed during the backup creation is 35Mb/s?
The slowdown of the backup creation may be related to the actual implementation. For Longhorn backup creation, it's not a simple copy or a simple sequential write. Instead, Longhorn will divide the data of the snapshot file into 2MB blocks, calculate the checksum, then transfer the blocks one by one, which is more like a random write even if not considering the calculation. I am not sure if there are other factors that could degrade this backup speed...

Besides, I will do benchmarking related to the backup speed and see if we can improve it.

Benchmark result
Directly do benchmark for the export directory in the NFS server:
fio -filename=./data1 -name=randw -ioengine=libaio -direct=1 -iodepth=32 -rw=randwrite -bs=2m -numjobs=1 -runtime=60 -size=10G -group_reporting

 write: IOPS=114, BW=228MiB/s (239MB/s)(10.0GiB/44901msec)

Do benchmark on the mount point of an NFS client:
fio -filename=./data2 -name=randw -ioengine=libaio -direct=1 -iodepth=32 -rw=randwrite -bs=2m -numjobs=1 -runtime=60 -size=10G -group_reporting

 write: IOPS=20, BW=41.6MiB/s (43.7MB/s)(2562MiB/61515msec)

Creating a backup for 1Gi volume data
It takes around 3min.
IOPS=512 / 180 = 2.8, BW=2.8 * 2 = 5.6MiB/s

It seems that we can speed up the backup creation. More specifically, for each 2Mi block of a backup, Longhorn will calculate the checksum and compress it before transferring to the remote backupstore. This cannot utilize disk, CPU, and network resources simultaneously and efficiently.  I think start multiple block processing parallelly would help. This improvement can be similar to the  backup volume listing:
https://github.com/longhorn/backupstore/blob/dfee2468733ec26b3e50561530046ad917b98e2d/deltablock.go#L262-L286

@shuo-wu
Thanks for the detailed analysis, it would be very helpful if the backup / replication process was much faster.

@shuo-wu
Does this 2Mi block mechanism also apply to the replication between the nodes?
The replication between the nodes also takes a long time with large PV.
There have been a number of kernel updates on the nodes recently, which resulted in a drain / reboot.
However, one has to wait a relatively long time until all replicants have been successfully distributed to all nodes before drain the next node and restarting.

You mean the transferred data size of each request during the rebuilding? Right now it's not a fixed value and this leads to some unexpected issues. Then we plan to optimize the rebuilding. I can not tell if we will use a fixed value later.
https://github.com/longhorn/longhorn/issues/1948
https://github.com/longhorn/longhorn/issues/2507

We see occasional 12hr job timeouts for a 100G volume backup

The high CPU usage or the timeout issue may be related to this: https://github.com/longhorn/longhorn/issues/2218#issuecomment-859611023

It seems that we can speed up the backup creation. More specifically, for each 2Mi block of a backup, Longhorn will calculate the checksum and compress it before transferring to the remote backupstore. This cannot utilize disk, CPU, and network resources simultaneously and efficiently. I think start multiple block processing parallelly would help. This improvement can be similar to the backup volume listing:
https://github.com/longhorn/backupstore/blob/dfee2468733ec26b3e50561530046ad917b98e2d/deltablock.go#L262-L286

Maybe it would be a nice feature to have the possibility to deactivate the compression. We have for example plenty of space don't used and would like to waive on compression.

Maybe it would be a nice feature to have the possibility to deactivate the compression. We have for example plenty of space don't used and would like to waive on compression.

I am not sure if we can allow users to disable or enable it on demand since it involves the data format. Let's see how long the compression will take during backing up first. If it's quite cheap, we can retain this feature.

Maybe it would be a nice feature to have the possibility to deactivate the compression. We have for example plenty of space don't used and would like to waive on compression.

I am not sure if we can allow users to disable or enable it on demand since it involves the data format. Let's see how long the compression will take during backing up first. If it's quite cheap, we can retain this feature.

In our case it takes about 8 hours to backup a 80GB volume. Since we have plenty of space and network resources it would be faster to have no compression and just move the data to the backup storage. If we have multiple volume backups running at the same time it will also take much longer to finish. It's also load that someone wouldn't need at the CPU since workloads are also running on the same hosts. Why should they get disturbed with high cpu load from backup jobs?

@PatrickHuetter
The backup creation is actually done by one of the volume replicas, which can be not on the node the volume/the workload is attached to

@PatrickHuetter
The backup creation is actually done by one of the volume replicas, which can be not on the node the volume/the workload is attached to

@shuo-wu Okay that seems good but also the compression is done there. And if this volume replica runs on a worker node (in our case we have hybrid nodes) it also takes cpu resources from the cluster (and then from the applications running on that hosts). It's okay for us but I think we will really run into problems if the volumes get larger. That's why I asked for some possibility to disable compression as it would be very useful for large volume workloads.

Related to #1409.


start type checking as early as possible

Implements #1 and removes also the plugin hooks which became obsolete due this plugin blocks the compilation always.

Codecov Report

Merging #3 into master will increase coverage by 0.42%.
The diff coverage is 88.88%.

@@            Coverage Diff            @@
##           master      #3      +/-   ##
=========================================
+ Coverage   96.68%   97.1%   +0.42%
=========================================
  Files           6       6
  Lines         181     173       -8
  Branches       19      19
=========================================
- Hits          175     168       -7
+ Misses          6       5       -1

Impacted Files
Coverage Δ

src/TsCheckerWebpackPlugin.ts
95.55% <88.88%> (+1.21%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 56e1db6...357c2a5. Read the comment docs.


Scala Common Enrich: add additional click ID definitions to campaign attribution enrichment

The ones we know about are:
"gclid" -> "Google",
"msclkid" -> "Microsoft",
"dclid" -> "DoubleClick"

If you know of any others, please add them to this ticket and we will add them into a future release.

Migrated to https://github.com/snowplow/enrich/issues/188


Mastering desgign patterns in JAVA

Real world examples with explanations for design patterns.

Singleton
Factory
Adaptor
Builder
Decorator
Observer

@Sar-taj107 Please assign this issue to me for contribution


Sema: Don't diagnose declarations more available than unavailable container

It turns out that we must allow declarations with introduced: availability nested inside of other declarations that are unavailable in order to influence weak linking. Stop diagnosing declarations as being more available than their unavailable containers and revert some changes to availability inference that were designed to avoid creating these nestings but caused regressions for declarations marked @_spi_available.
Reverts parts of https://github.com/apple/swift/pull/64310, https://github.com/apple/swift/pull/64015, and
https://github.com/apple/swift/pull/62900.

@swift-ci please smoke test


List: Inconsistency in right Icon guidelines for Modus List Item

Prerequisites

[X] I have searched for duplicate or closed issues
[X] I have read the contributing guidelines

Describe the issue
The current style guide specifies using a tick icon to represent the active state for the ModusListItem right icon. However, this does not match the design presented in the Figma mocks.
PS: The component implemented the right icon as per the style guide, and there is no option to display a different icon, as it is reserved for the tick icon.
Reduced test cases
No response
What operating system(s) are you seeing the problem on?
No response
What browser(s) are you seeing the problem on?
No response
What is the issue regarding ?
@trimble-oss/modus-web-components
What version of npm package are you using ?
No response
Priority
Low
What product/project are you using Modus Components for ?
modus-internal
What is your team/division name ?
modus-internal
Are you willing to contribute ?
None
Are you using Modus Web Components in production ?
No response

@mitch-trimble Need your recommendation to proceed on this issue.


Does not build using VS 2022

Describe the bug
Build of cmake generated project fails
Expected Behavior
Build succeeds
Current Behavior
Running cmake to generate VS projects

cmake.exe --no-warn-unused-cli -DCMAKE_BUILD_TYPE="[Debug | Release]" -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE -H".\aws-sdk-cpp" -G"Visual Studio 17 2022" -B"cmake_gen_dir\x64lib" -T host=x64 -A x64 -DBUILD_ONLY="s3;transcribe;transfer;s3-crt" -DENABLE_TESTING=OFF -DBUILD_SHARED_LIBS=OFF -DFORCE_SHARED_CRT=OFF

Open generated solution and try to build project aws-cpp-sdk-core.
Compile time error:

aws-sdk-cpp\src\aws-cpp-sdk-core\source\utils\event\EventEncoderStream.cpp(31,17): error C2220: the following warning is treated as an error
14>C:\temp\build_aws_cpp_sdk_1.11\aws-sdk-cpp\src\aws-cpp-sdk-core\source\utils\event\EventEncoderStream.cpp(31,17): warning C4834: discarding return value of function with [[nodiscard]] attribute

Reproduction Steps
Run cmake and specify VS 2022 as generator.

cmake.exe --no-warn-unused-cli -DCMAKE_BUILD_TYPE="[Debug | Release]" -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=TRUE -H".\aws-sdk-cpp" -G"Visual Studio 17 2022" -B"cmake_gen_dir\x64lib" -T host=x64 -A x64 -DBUILD_ONLY="s3;transcribe;transfer;s3-crt" -DENABLE_TESTING=OFF -DBUILD_SHARED_LIBS=OFF -DFORCE_SHARED_CRT=OFF

Build generated solution either using msbuild (command line) or Visual Studio
Possible Solution
No response
Additional Information/Context
cmake output

-- TARGET_ARCH not specified; inferring host OS to be platform compilation target
-- Building AWS libraries as static objects
-- Generating windows build config
-- Building project version: 1.11.258
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045.
-- The CXX compiler identification is MSVC 19.38.33134.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done

AWS CPP SDK version used
1.11.258
Compiler and Version used
Microsoft Visual Studio Community 2022 (64-bit) - Current Version 17.8.4
Operating System and version
Windows 10

We can work at getting a fix out, which should be out soon, but if you want a quick work around you can set warnings as errors to off -DAWS_SDK_WARNINGS_ARE_ERRORS=OFF

merged, will be released today

This was fixed with this PR. Please let us know if you're seeing any other build issues on windows with this sdk


Minor change to 1.19 patch

📌 Testing https://github.com/aws/eks-distro-prow-jobs/pull/305
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve
/unhold

/lgtm


Virtual attribute breakage with Objection 1.6.4+

Virtual attributes aren't being added to results after updating Objection to 1.6.4+.  I think this issue is related.
For example, trying to stringify Objection's results won't include virtual attributes.
Comment.query().then(function(results) {
	console.log("comments " + JSON.stringify( results , null, "\t"));
})

I think Feathers-Objection is affected by this. Unsure if it's something F-O can fix or if it needs to be fixed on Objection's end. But reverting to<EMAIL_ADDRESS>works for now.

@Aerlin Did you try with Objection.js latest version 1.6.6?

Yeah. I double checked now and 1.6.6 is affected.
Example of my setup
// models/Comment.js
const { Model, snakeCaseMappers } = require('objection');
const moment = require('moment');
const md = require('markdown-it')({
  html: true,
  linkify: true,
  typographer: true
});
...
module.exports = function(app) {
	class Comment extends Model {
		static get tableName() {
			return 'comments';
		}
		static get virtualAttributes() {
		  return ['created_ago', 'updated_ago', 'html'];
		}
		created_ago() {
		  return this.created_at &&  moment( this.created_at.getTime() ).fromNow(true);
		}

		updated_ago() {
		  return this.updated_at && moment( this.updated_at.getTime() ).fromNow(true);
		}

		html() {
		 // console.log("Comment html()");
		  return this.body && md.render(this.body);
		}
	}
	...
}

// Elsewhere.
const Comment = require('../models/Comment.js')(app);
var builder = Comment.query().then(function(results) {
	console.log("comments " + JSON.stringify( results , null, "\t"));
}).catch(function(err) {
	console.trace(err);
});

In<EMAIL_ADDRESS>the query results contain the created_ago, updated_ago and html fields. In 1.6.4, 1.6.5 & 1.6.6 the fields are missing. I know this isn't Feathers-Objection's fault. Just wanted to let you know the issue was affecting F-O. Although I'm starting to wonder if I broke something as nobody else seems to be reporting the issue on the Objection issues list. 😕

Objection's Model virtuals parameter used to default to true, now defaults to false.
[(https://github.com/Vincit/objection.js/commit/55303773da82e196b732803f112118fcc3bee9b2#diff-008bf8359e2b2c171c571598395ecfb7L22]((https://github.com/Vincit/objection.js/commit/55303773da82e196b732803f112118fcc3bee9b2#diff-008bf8359e2b2c171c571598395ecfb7L22)
https://github.com/Vincit/objection.js/commit/8115f0bc50794a5aed40337c39bf71ff60c4bee6#diff-008bf8359e2b2c171c571598395ecfb7R52
If getVirtuals is set to default to true, the missing virtual attributes return.
function getVirtuals(opt) {
  if (!opt) {
    return true;
  } else if (Array.isArray(opt.virtuals)) {
    return opt.virtuals;
  } else if (opt.virtuals) {
    return true;
  } else {
    return true; // <--------------
  }
}

I 'm not sure if the change was intended and F-O needs to implicitly requests virtuals somewhere now, or if it's a bug that they can fix.

This is a bug in objection. Sorry about this. I'm releasing 1.6.7 right now with a fix.

https://github.com/Vincit/objection.js/commit/df277ea105f5cc46b607993f3b9550265b325864

objection 1.6.7 is now released.


feat(types): add types package

Copied the types package from btckit to our own npm scope

I merged and published this for you. Realised then there is no README but we can add one the next time.
Fixing up this manual process is an urgent task looming on my horizon

Once your UI work in extension is done, I'd love for you to focus automating releases in the mono repo.
This would be a huge productivity win for the team, and essential for wider mono repo adoption cc @markmhendrickson https://github.com/leather-wallet/mono/issues/29


Request: More information about programatically linking Templates to tokens.

I am trying to set up a ItemMacro that should be called via DAE Macro Execute.
I create a MeasuredTamplate (range). To this point everything works fine, but then when trying to attach the template to the token I don't have a clue what I am doing wrong:
//DAE Macro Execute, Effect Value = "Macro Name" @target
const lastArg = args[args.length - 1];
let tactor;
if (lastArg.tokenId) tactor = canvas.tokens.get(lastArg.tokenId).actor;
else tactor = game.actors.get(lastArg.actorId);
const target = canvas.tokens.get(lastArg.tokenId) || token;

//Debug Info ;)
//ChatMessage.create({ content: `${args[0]} ${args[1]} ${args[2]} ${args[3]}  ${args.length}` })

if (args[0] === "on") {
	(async () => {
		let range = MeasuredTemplate.create({
			t: "circle",
			user: game.user._id,
			x: target.x + canvas.grid.size / 2,
			y: target.y + canvas.grid.size / 2,
			direction: 0,
			distance: 30,
			borderColor: "#44975C",
			fillColor: "#44975C",
			flags: {
				DAESRD: {
					PassWithoutTrace: {
						ActorId: tactor.id
					}
				}
			}
		});

		console.log(token);

		// tokenAttacher.attachElementToToken(range, target_token, let suppresNotification=false);
		tokenAttacher.attachElementToToken(range, target, false);
	})();

}

if (args[0] === "off") {
	(async () => {
		let removeTemplates = canvas.templates.placeables.filter(i => i.data.flags.DAESRD?.PassWithoutTrace?.ActorId === tactor.id);
		await target.update({ x: template.x, y: template.y })
		await canvas.templates.deleteMany([removeTemplates[0].id]);
		await tactor.deleteEmbeddedEntity("ActiveEffect", lastArg.effectId);
	})();

}

Maybe you could provide a minimal example how to add a Template to a token. That would be highly appreciated!
Cheers,
Waldleufer

What you're doing looks right except you're not awaiting the creation of the template let range = await MeasuredTemplate.create({

First of all: Thank you for the help! 🙇
I added the missing await and now the MeasuredTemplate gets created properly, and I can also programmatically delete it.
I now also get the notification "Selected Objects attached to Tokens"
however this happens,

when I move the Token afterwards.
When creating a template with the Foundry UI the template neatly follows the target, so I assume I did forget some step in the creation of the MeasuredTemplate.
This is how the Macro looks like after said change.
//DAE Macro Execute, Effect Value = "Macro Name" @target
const lastArg = args[args.length - 1];
let tactor;
if (lastArg.tokenId) tactor = canvas.tokens.get(lastArg.tokenId).actor;
else tactor = game.actors.get(lastArg.actorId);
const target = canvas.tokens.get(lastArg.tokenId) || token;

//Debug Info ;)
//ChatMessage.create({ content: `${args[0]} ${args[1]} ${args[2]} ${args[3]}  ${args.length}` })

if (args[0] === "on") {
	(async () => {
		let range = await MeasuredTemplate.create({
			t: "circle",
			user: game.user._id,
			x: target.x + canvas.grid.size / 2,
			y: target.y + canvas.grid.size / 2,
			direction: 0,
			distance: 30,
			borderColor: "#44975C",
			fillColor: "#44975C",
			flags: {
				DAESRD: {
					PassWithoutTrace: {
						ActorId: tactor.id
					}
				}
			}
		});

		console.log(token);
		console.log(range);

		// tokenAttacher.attachElementToToken(range, target_token, let suppresNotification=false);
		tokenAttacher.attachElementToToken(token, range, false);
	})();
}

if (args[0] === "off") {
	(async () => {
		let removeTemplates = canvas.templates.placeables.filter(i => i.data.flags.DAESRD?.PassWithoutTrace?.ActorId === tactor.id);
		await canvas.templates.deleteMany([removeTemplates[0].id]);
		await tactor.deleteEmbeddedEntity("ActiveEffect", lastArg.effectId);
	})();
}

Cheers,
Waldleufer

@KayelGee sorry for the ping, but you closed this issue already so it might slip your attention 😄
sorry for the late reply 😉

Don't worry I still get a notification on closed issues, I'm just not looking at my emails that often ^^'. I usually suggest to ping me in discord if you need my attention.
Why did you swap range and token in tokenAttacher.attachElementToToken(token, range, false);? This way you're trying to attach the token the the template, which isn't possible.

Oh 🤔 well it wasn’t working beforehand but I will look into that again and drop further information here.

So I updated the comment up there, turns out, that the part of the code works flawlessly:
//DAE Macro Execute, Effect Value = "Macro Name" @target
const lastArg = args[args.length - 1];
let tactor;
if (lastArg.tokenId) tactor = canvas.tokens.get(lastArg.tokenId).actor;
else tactor = game.actors.get(lastArg.actorId);
const target = canvas.tokens.get(lastArg.tokenId) || token;

//Debug Info ;)
//ChatMessage.create({ content: `${args[0]} ${args[1]} ${args[2]} ${args[3]}  ${args.length}` })

if (args[0] === "on") {
	(async () => {
		let range = await MeasuredTemplate.create({
			t: "circle",
			user: game.user._id,
			x: target.x + canvas.grid.size / 2,
			y: target.y + canvas.grid.size / 2,
			direction: 0,
			distance: 30,
			borderColor: "#44975C",
			fillColor: "#44975C",
			flags: {
				DAESRD: {
					PassWithoutTrace: {
						ActorId: tactor.id
					}
				}
			}
		});

		console.log(token);
		console.log(range);

		// tokenAttacher.attachElementToToken(range, target_token, let suppresNotification=false);
		tokenAttacher.attachElementToToken(range, token, false);
	})();
}

if (args[0] === "off") {
	(async () => {
		let removeTemplates = canvas.templates.placeables.filter(i => i.data.flags.DAESRD?.PassWithoutTrace?.ActorId === tactor.id);
		await canvas.templates.deleteMany([removeTemplates[0].id]);
		await tactor.deleteEmbeddedEntity("ActiveEffect", lastArg.effectId);
	})();
}

So the problem lies in the code that I appended below to apply a active effect to the template I just created ;)

I figured it out.
Feel free to include this working version of the Macro somewhere in your Module if you want to.
// DAE Macro Execute: Attribute Key - Change Mode - Effect Value
//                  macro.itemMacro -   Custom    - @item @target
// Requires MidiQOL, DAE, Item Macros, ActiveEffects and TokenAttacher
// Created by Waldleufer

const lastArg = args[args.length - 1];
let tactor;
if (lastArg.tokenId) tactor = canvas.tokens.get(lastArg.tokenId).actor;
else tactor = game.actors.get(lastArg.actorId);
const target = canvas.tokens.get(lastArg.tokenId) || token;
let item = args[3];
console.log(item);

//Debug Info ;)
//ChatMessage.create({ content: `${args[0]} ${args[1]} ${args[2]} ${args[3]}  ${args.length}` })

if (args[0] === "on") {
	(async () => {
		let range = await MeasuredTemplate.create({
			t: "circle",
			user: game.user._id,
			x: target.x + canvas.grid.size / 2,
			y: target.y + canvas.grid.size / 2,
			direction: 0,
			distance: 30,
			borderColor: "#44975C",
			fillColor: "#44975C",
			flags: {
				DAESRD: {
					PassWithoutTrace: {
						ActorId: tactor.id
					}
				}
			}
		});

		// console.log(token);
		// console.log(range);

		// tokenAttacher.attachElementToToken(range, target_token, let suppresNotification=false);
		tokenAttacher.attachElementToToken(range, token, false);

		let templates = await canvas.templates.placeables.filter(i => i.data.flags.DAESRD?.PassWithoutTrace?.ActorId === tactor.id);
		let template = templates[0]; //We can safeley assume that this is the first and only template in the Array.
		let effects = item.efData.changes[0].value[0].effects;
		let disposition = token.disposition
		let templateEffectData = [];
		for( let effect of effects){
			// console.log(effect);
			if(effect.changes[0].key === "macro.itemMacro") {
				// Avoide inifinity loop, only aply actual effects, no macro.itemMacro effect.
			} else {
				let data = { data: duplicate(effect), parentActorId: false, parentActorLink: false, entityType: "template", entityId:template.id, casterDisposition: disposition,castLevel: 1} //args[0].spellLevel}
				templateEffectData.push(data)
			}
		}
		console.log(template);
		await template.setFlag("ActiveAuras", "IsAura", templateEffectData);
		console.log(template);
		ActiveAuras.CollateAuras(canvas.scene._id, true, false, "spellCast")
		return  {haltEffectsApplication: true}
	})();
}

if (args[0] === "off") {
	(async () => {
		let removeTemplates = canvas.templates.placeables.filter(i => i.data.flags.DAESRD?.PassWithoutTrace?.ActorId === tactor.id);
		await canvas.templates.deleteMany([removeTemplates[0].id]);
		await tactor.deleteEmbeddedEntity("ActiveEffect", lastArg.effectId);
	})();
}

It creates a template and applies any DAE that are on the item (except the one that is calling the Macro) to the template.
And due to your awesome Template Attaching this works out as an improved "aura" effect 😄
Thank you for your help with that again!
Cheers,
Waldleufer

No problem. I'm not sure I should include your macro as it has a few dependencies and is also system specific because of midi qol as I understand. But I can point to this issue in the readme.

Sounds good to me 😊


0.6.1 Support Notifications -- Docker LoggingFile Config Value is Incorrect

In the docker config file for support-notifications, the value for LoggingFile is incorrect.
Currently, it is set to
LoggingFile = './logs/edgex-support-notifications.log'
It should be set to
LoggingFile = '/edgex/logs/edgex-support-notifications.log'
This is because the latter will write to the shared docker volume as opposed to the local container file-system.
https://github.com/edgexfoundry/edgex-go/blob/california/cmd/support-notifications/res/configuration-docker.toml

Thanks for the catch @tsconn23 - Update has been made and PR #440 submitted.


Catalyst + ReactNative: Typedef redefinition with different types ('uint8_t' (aka 'unsigned char') vs 'enum clockid_t') error when building for mac target

Description
I'm getting the  following errors when building for mac. This is a default application, created via react-native init.
Error 1: Typedef redefinition with different types ('uint8_t' (aka 'unsigned char') vs 'enum clockid_t')
Error 2: ld: in /Users/jh/Documents/projects/VST/Trial3/ios/Pods/OpenSSL-Universal/ios/lib/libcrypto.a(cryptlib.o), building for Mac Catalyst, but linking in object file built for iOS Simulator, file '/Users/jh/Documents/projects/VST/Trial3/ios/Pods/OpenSSL-Universal/ios/lib/libcrypto.a' for architecture x86_64
React Native version:
MacOS: 10.15.4
Xcode: 11.4
RN: 0.62.0-rc.5
RN Cli: 4.8.0

Steps To Reproduce
Provide a detailed list of steps that reproduce the issue.

react-native init App
react-native run-ios - Works fine
Open App.xcworkspace, select mac as a target and clean build.

Expected Results
Application working on Mac too.
Snack, code example, screenshot, or link to a repository:

I commented out the following line in Time.h:
//typedef uint8_t clockid_t;
extern "C" int clock_gettime(clockid_t clk_id, struct timespec* ts);
extern "C" int clock_getres(clockid_t clk_id, struct timespec* ts);
#endif

Then tried to build again and the Error 1 disappeared but Error 2 still exists:

Should I do something for migration to build for mac?

I got it working by deleting Facebook Flipper and everything else related to it. Here are the steps to take:

Open Podfile and delete everything related to Flipper and FlipperKit.
Hit Run and voila.


[question] inject in celery tasks

Hi, what are the best practices for inject in celery tasks?
from celery import shared_task

class FooService:
    @inject
    def __init__(bar: Bar):
          self.bar = bar

    def run_method()
        pass

@shared_task(name='foo_task')
def foo_task(service: FooService)
    service.run_method()

How to implement correctly?

I've never used Celery directly, but it most likely will need an integration layer like https://github.com/alecthomas/flask_injector or https://github.com/blubber/django_injector. I don't know if Celery is extensible in this regard, so there may be a bit of hacking involved.

Hi,
I'm the author of an alternative dependency injector library (dependencies part of the dry-python project).
We already have integration with Celery.
We decide not to fight with Celery worker bootstrap but provide a way to register a task with injector outside of the Celery task function.
If anyone is interested to backport this to the project, I invite you to take a look at our implementation.
Best regards,
Artem.

If I remember correctly, class based Celery tasks (as in the code linked above) are kind of like singletons, so a single instance of a task class may be used for several runs.
This was not acceptable for me so I built a custom task decorator on top of Celery's decorator. Mine gets an injector instance that has been set in the Celery config, which is then used to call the task. It's a bit hacky but it works.

Actually, we use function-based and class-based tasks depending on the bind parameter in the scope of the Injector.
https://github.com/dry-python/dependencies/blob/5fcc9313da03f28af2165aa9cb434aa333d0f1d9/src/dependencies/contrib/_celery.py#L55-L63

I missed that, sorry, and thanks!


MaybeUninit, serde StdError, unittests

This takes the remaining bits from https://github.com/rust-embedded-community/serde-json-core/pull/59 by @jordy25519

MaybeUninint instead of the deprecated mem::uninitialized (given the unstable maybe_uninit_slice feature, in the future this should use slice_assume_init_ref)
Two unittests
serde::*::StdError for our Errors
The bump in serde version further is not needed or helpful AFAICT (this is a library).
The reorganization of the code using MaybeUninit in #59 is not necessary

@eldruin thanks for handling the previous PRs! Could you have a look here as well?
It would be great if we can land this and then release with #69


HI3516CV100 IMX291 support

XiongMai made an IPC module with HI3516CV100 SoC and IMX291 sensor.
Strangely the IMX291 is supported on many other HI3516 chips, but not on this one. Could something be added or changed to allow this combination to work?
XM product ID: IPG-54H20PL-S
PCB Marking: BLK18C-0090-38X38_S
Sensor: IMX291
FM version: V4.02.R11.00002532.10010.242800.00000
SoC: HI3516CV100
I have around 20 cameras I'm trying to refurbish with OpenIPC so maybe I can help with a donation?

IMX291 sensor driver should be written from scratch for this model. It will take 4-6 weeks of full-time reverse-engineer work with close interaction with hardware.
I doubt that it's even reasonable judging you can replace all boards to modern version

Thanks for your honest answer. Then sadly I will have to look for a different solution until someone feels the need to go and make this happen.

I wrote you some questions in PM Telegram, please post as you get a chance.

I made a small crutch with imx138 since we don't have a driver.
Not sure if it will work well but it breathes nonetheless... ;)
https://openipc.org/snapshots/368807

Solution
One commit for sensor init
Set old integrated sensor from Linux
fw_setenv sensor imx138_spi

IPG-54H20PL-S stock firmwares 2532.zip
Attached all the versions of the stock firmware for this board. Maybe this contains a driver somewhere?
The camera briefly worked with the imx138 change, but after changing a couple of settings in the web interface it is now broken, and I have not been able to get it do display an image again.

Sensor driver is compiled into Sofia binary statically. There is no way to extract and use it

Ah ok, I think I understand now, you are looking for something like this?
https://github.com/EliasKotlyar/Xiaomi-Dafang-Software/blob/master/drivers/sensors/imx291/imx291.c

No, this is driver for Ingenic platform (the code is absolutely useless for HiSilicon)

Same issue as #204
At November 2022, my camera was half-working, see my comment, I need to try with the latest OpenIPC firmware.


Violence

Song Name
Violence
Artist Name
ChristianMossuto
Source
Youtube
Youtube Link
https://youtu.be/tfa9DI0PSyI?si=dCpms2MgoVHLTBH9
Direct File Link
No response
Song ID
412437
Extra
No response

reject
aready exists multiple times


ament_package: 0.13.0-1 in 'galactic/distribution.yaml' [bloom]

Increasing version of package(s) in repository ament_package to 0.13.0-1:

upstream repository: https://github.com/ament/ament_package.git
release repository: https://github.com/ros2-gbp/ament_package-release.git
distro file: galactic/distribution.yaml
bloom version: 0.10.7
previous version for package: 0.12.0-2

ament_package
* Add support for appending to environment variables (#130 <https://github.com/ament/ament_package/issues/130>) (#137 <https://github.com/ament/ament_package/issues/137>)
* Update maintainers to Audrow Nash (#135 <https://github.com/ament/ament_package/issues/135>) (#136 <https://github.com/ament/ament_package/issues/136>)
* Contributors: Scott K Logan, Audrow Nash
* Revert "Generate Setuptools Dict Helper Method (#126 <https://github.com/ament/ament_package/issues/126>)" (#131 <https://github.com/ament/ament_package/issues/131>)
* Contributors: Audrow Nash

@audrow the change log is a bit irregular in that it has contributors listed in two separate lines. As far as I know the generate change logs tool doesn't normally do this, any idea why that's the case?

Also: we should hold this PR to let Rolling finish rebuilding.

I had the same thing happen to me. It can occur if someone runs catkin_generate_changelog, but does not run catkin_prepare_release (or update the "Forthcoming" line). The next time someone comes along and runs those two commands, catkin_prepare_release will remove all "Forthcoming" lines from the file. So, I think the changelog contains changes for two different releases.

I'll rerelease this package.

Closing in favor of https://github.com/ros/rosdistro/pull/31871.


setup.py: Remove pathlib2

Title
Remove test dependency pathlib2
Motivation
No longer needed.
Checklist
Before you create the PR, please make sure you have:

[x] Picked a concise, informative and complete title
[x] Clearly explained the motivation behind the PR
[x] Linked to any existing issues that your PR will be solving
[x] Included tests for any new functionality
[x] Checked that all unit tests pass

Workflow
Please avoid rebasing and force-pushing to the branch of the PR once a review is in progress.
Rebasing can make your commits look a bit cleaner, but it also makes life more difficult from the reviewer, because they are no longer able to distinguish between code that has already been reviewed, and unreviewed code.

Thank you!


Remove unused lodash

Strangely enough, I don't bundle lodash, Lambda's process canses error E_PIPE...

I'm getting E_PIPE even with lodash:
2017-01-27T16:04:38.702Z  Error: write EPIPE
at Object.exports._errnoException (util.js:870:11)
at exports._exceptionWithHostPort (util.js:893:20)
at WriteWrap.afterWrite (net.js:763:14)


OIDC: login succeeds, but FE does not show user as loggged in

I  am trying to set up scicat for use at MLZ with OIDC login enabled.
I get correctly redirected to the success URL, but I am still shown as not logged in in the FE.
Debugging shows that the user and authInfo entries seem to be populated correctly.
The login callback gets called returns success.
Any hints what is still missing here?
Best Björn
providers.json:
  "MLZ": {
        "provider": "oidc",
        "authScheme": "openid connect",
        "module": "passport-openidconnect",
        "authPath": "/auth/mlz",
        "successRedirect": "/",
        "failureRedirect": "/login",
        "failureFlash": true,
        "session": true,
        "issuer": "http://<host>/auth/realms/MLZ",
        "authorizationURL":"http://<host>/auth/realms/MLZ/protocol/openid-connect/auth",
        "tokenURL": "http://<host>//auth/realms/MLZ/protocol/openid-connect/token",
        "userInfoURL": "http://<host>//auth/realms/MLZ/protocol/openid-connect/userinfo",
        "clientID": "scicat",
        "clientSecret": "<redacted>",
        "callbackURL": "http://<scicathost>/auth/mlz/callback",
        "callbackPath":"/auth/mlz/callback",
        "loginCallback": "mlzLoginCallback",
        "scope": ["email", "profile", "openid"]

  return function(err, user, identity, token) {

    const logger = require("../../common/logger");
    const request = require("request");

    console.dir(user);
    console.dir(token);
    var authInfo = {
      identity: identity,
    };
    if (token) {
      authInfo.accessToken = token;
    }
    console.dir(identity);

    let pinfo = identity.profile._json;
    authInfo.identity.profile.accessGroups = pinfo.realm_access.roles;
    authInfo.identity.profile.username = pinfo.preferred_username;
    authInfo.identity.profile.emails = [ { value: pinfo.email}  ];
    console.dir(authInfo);
    done(err, user, authInfo);
return;
}

catanie: environment.js
...
oAuth2Endpoints: [{displayText: "MLZ", authURL:"auth/mlz"}],
 loginFormEnabled: true
...

                                                                                49,1          76%

tracebacks:
catanie_1                    | <IP_ADDRESS> - - [22/Nov/2021:11:34:58 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36" "<ip>"
catamel_1                    | Unhandled error for request GET /api/v3/Users/__anonymous__: Error: Authorization Required
catamel_1                    |     at /home/node/app/node_modules/loopback/lib/application.js:433:23
catamel_1                    |     at /home/node/app/node_modules/loopback/lib/model.js:359:7
catamel_1                    |     at /home/node/app/node_modules/loopback/common/models/acl.js:536:16
catamel_1                    |     at /home/node/app/node_modules/async/dist/async.js:3888:9
catamel_1                    |     at /home/node/app/node_modules/async/dist/async.js:473:16
catamel_1                    |     at iteratorCallback (/home/node/app/node_modules/async/dist/async.js:1064:13)
catamel_1                    |     at /home/node/app/node_modules/async/dist/async.js:969:16
catamel_1                    |     at /home/node/app/node_modules/async/dist/async.js:3885:13
catamel_1                    |     at /home/node/app/node_modules/loopback/common/models/acl.js:518:17
catamel_1                    |     at /home/node/app/node_modules/loopback/common/models/role.js:246:9
catamel_1                    |     at processTicksAndRejections (internal/process/task_queues.js:77:11)

@bpedersen2 sorry for the delay, I just saw this.
It's been a long time, so I don't recall the exact reason, but the settings in providers.json that work for me are public URLs into catanie/frontend. So, something like:
        "successRedirect": "https://cataniserver/",
        "failureRedirect": "https://cataniserver/login",

I think this is because the code running this code and redirecting is actually the catamel/backend server, so '/` is interpreted relative to it.

I did some further debugging, and it seems like the pasport-openid lib has evoled faster than the integration in loopback-passport-component. The usage of the callback arguments there did not match.  Needs further investigation...


bump download/upload artifact gha version

🚀 Pull Request
Description

This pull-request replaces #186 and #187.
Migrating to the next version (@3 -> @4) requires to be done consistently within a single pull-request for both of the upload-artifact and download-artifact GHAs.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (8eb79a5) 97.36% compared to head (c711651) 97.36%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #188   +/-   ##
=======================================
  Coverage   97.36%   97.36%
=======================================
  Files           7        7
  Lines         644      644
=======================================
  Hits          627      627
  Misses         17       17

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Implement a user map similar to Vue

This is my first attempt at react coding here. Let me know anything I'm doing wrong.

Adds a user map similar to what's in Vue
Adds the hash attribute to the map to remember where it's been (we might want this in the user map but not e.g. in location selection)
Makes sure cookies are sent with map subrequests through transformRequest
TODO: Currently the map has no interactivity (clicking on users does nothing).

The map is very ugly, I don't know what the right way of making it full-size is (or even if we want that). This seems hard because App.tsx wraps the routes in a <Container maxWidth="md" className={classes.padding}>, so hard to get rid of the padding? I'm not sure.

I've separated out the source + layer for the "clustered users layer" into a separate file, it seemed clean. But now I'm not sure what the right way to bridge the interactivity is. Presumably at some point we want some kind of card or other thing to pop up when you click a user's circle, how to do that "correctly" in the frontend world?

This seems hard because App.tsx wraps the routes in a <Container maxWidth="md" className={classes.padding}>

Yeah, I do find that a bit awkward. Maybe we should extract that container layer further down into each page component so the page itself can decide whether it wants to go full width or fixed width.

Presumably at some point we want some kind of card or other thing to pop up when you click a user's circle

How does that work with mapbox if you want to show a card/pop up? Presumably more object configurations similar to what you've done for the clustered users stuff? If it involves adding custom markup (like what Lucas has done to add in a search bar), putting that card/pop up into its own component is probably the way to go.
Otherwise, I think what you've done there to split the layers into its own might be the best we can do, since we're stuck with mapbox's imperative API and so won't really be able to use React's component model that well in that case.

App.tsx wraps the routes in a , so hard to get rid of the padding?

Let's not worry about this until we have desktop mid-fis.

Presumably at some point we want some kind of card or other thing to pop up when you click a user's circle, how to do that "correctly" in the frontend world?

As discussed, we will just have a side/bottom panel rather than popups, which would be integrated by putting setState (or whatever) calls in the click handlers.


Move CA Certificates documentation next to SSL Cert Verification

When reading the SSL Cert Verification documentation and working on directing requests to use a system's local CA store I never noticed the CA Certificates documentation as it is at the tail end of the Advanced page.  I suggest it would be better to have the CA Certificates documentation immediately follow the documentation regarding SSL Cert Verification.
No content change, just a relocation of existing verbiage.

Thanks @jimbrowne! :cake: :sparkles: :cake:


[FEATURE] wash get inventory output is unreadable

Affected project(s)

[ ] documentation
[ ] examples
[ ] wasmCloud host
[X] wasmCloud CLI (wash)
[ ] wasmCloud dashboard UI (washboard)
[ ] capability providers
[ ] provider bindgen
[ ] control interface client
[ ] other / not sure

Is your feature request related to a problem? Please describe.
Looking at the inventory of hosts is one of the most common operations for wash. However, when using file references or long generated component IDs, it becomes pretty unreadable.

Taking a look at other infrastructure tools, like kubectl and nomad, none of them quite overflow the terminal like wash. I think this is largely due to the fidelity of information, and with wash displaying the full image reference really just isn't quite necessary.
Describe the solution you'd like
I would love for the host inventory to display just the right amount of information for the human readable format. Just like kubernetes, we can hide all of the other information behind the json flag for when it's necessary.

I propose that the host inventory display the following information:

Host ID
Host friendly name
Host Labels
Components list

Component ID
Name
Max count

providers list

Provider ID
Name

I could even be convinced to drop the friendly name / host labels for the inventory operation, but it can be useful at a glance to verify that, for example, your application is spread properly across availability zones.
Describe alternatives you've considered

We could push host information into the wash get hosts command to keep the terminal more clean
We could be smarter with our table library to fit to the width of the terminal and truncate the image reference. This could work, but a truncated image reference or ID is pretty unhelpful

Additional context
No response

I can take this; please assign it to me.

Hello @brooksmtownsend,
Does this look okay? If it does, then, I'll create the PR; otherwise let me know of any other improvements you'd like to see:

@ossfellow this looks wonderful! 😲

Hey @brooksmtownsend this was resolved right?

Yep! Coming soon in a wash 0.30.0 near you. Thanks again @ossfellow


help issue please

Actual Behavior
[18:57:07] Starting PokemonGo Bot....
Traceback (most recent call last):
File "./pokecli.py", line 357, in
main()
File "./pokecli.py", line 68, in main
bot.tick()
File "/home/sskssl2015/Scott/PokemonGo-Bot-dev/pokemongo_bot/init.py", line 63, in tick
if worker(self).work() == WorkerResult.RUNNING:
File "/home/sskssl2015/PokemonGo-Bot-dev/pokemongo_bot/cell_workers/incubate_eggs_worker.py", line 34, in work
incubators.append({"id":incubator.get("id", -1), "used":False})
AttributeError: 'str' object has no attribute 'get'
as you can see this is the error i login okthen its starts the app. im sure i have seen an incubator issue but can not find it. can someone help many thanks

getting same error with most recent dev version

duplicate of this: https://github.com/PokemonGoF/PokemonGo-Bot/issues/1444
but if you are too lazy to read it in there, just do:
sudo pip install -r requirements.txt

yea this works, i forgot to do it again when i installed the dev branch lol :)

excellent i knew there was an issue i saw lol.  im in a walking glitch now shame cause i like the dev versions

Duplicate of https://github.com/PokemonGoF/PokemonGo-Bot/issues/1444.
Issue has been resolved, closing this.
/closed


Getting Started

What's the purpose of this pull request?
Adds getting-started doc
How it works?
Tell us the role of the new feature, or component, in its context.
How to test it?
Describe the steps with bullet points. Is there any external link that can be used to better test it or an example?
References
Spread the knowledge: is there any content you used to create this PR that is worth sharing?
Extra tip: adding references to related issues or mentioning people important to this PR may be good for the documentation and reviewing process

Closing this PR since the English version was already merged. https://github.com/vtex/faststore/pull/459

Closing this PR since the English version was already merged. https://github.com/vtex/faststore/pull/459


Added support for doc_string.

You can verify this by checking out the "Scenario With Text" on the "Displaying Scenarios" scenario.

I'm pretty sure I broke the table thing... We'll be back on this

It should be ok now.

thanks!


Running the model on a video

I would like to thank you for your interesting work.
How can I run your model on a video to detect violence per frame?
Thanks again for your interesting work.

To save time on training, I've extracted the videos into frames using FFmpeg.

Thanks a lot for your kind reply.
Could you please provide to the community the code for running the model on webcam. It will be very useful for the community. Thanks

Please see https://github.com/JimLee1996/AVSS2019/tree/master/src/Demo

Thanks a lot for providing the code to the community. I am interested in your code for research purposes. I found that you are the state of the art currently in violence detection in surveillance videos.
But there is one problem. I tested your pre-trained weight 'DenseNetLean_Kinetics.pth' on a video from the Hockey Fight dataset. Your model does not detect violence at all. It detects all the video as normal. Could you please provide the right weights?

DenseNetLean_Kinetics.pth is pre-trained weight on kinetics and was not trained on violence datasets. I've upload the right weight trained on mix dataset to Demo.

Thanks a lot for your great gift to the community!!!

😉


Abide validates disabled form elements (checkbox, radio)

Don't validate disabled checkbox and radio inputs.

this is a good hamburger


fixing some test events after merge of #1179

to: @airbnb/streamalert-maintainers
related to: #1179, #1115
Background
#1115 changed some test files, while #1179 completely restructured them
Changes

Fixing the above issue.
Updating the logger code cause I discovered it was logging output twice in local runs.

I tested the above change in a lambda function to ensure it works as expected in the lambda environment.

Coverage decreased (-0.002%) to 95.414% when pulling 3cd2d5d5988c807837825fbd7b7ab79192d2d831 on ryandeivert-fix-some-events into 1b704c67a1b8f17791e41d3de63a002be3f625b4 on release-3-1-0.


Uncaught TypeError: Cannot read property 'length' of null

[Enter steps to reproduce below:]

...
...

Atom Version: 0.175.0
System: linux 3.19.0-15-generic
Thrown From: autocomplete-plus package, v2.1.3
Stack Trace
Uncaught TypeError: Cannot read property 'length' of null
At /home/alban/.atom/packages/autocomplete-plus/lib/suggestion-list-element.coffee:116

TypeError: Cannot read property 'length' of null
  at /home/alban/.atom/packages/autocomplete-plus/lib/suggestion-list-element.coffee:116:29
  at /home/alban/.atom/packages/autocomplete-plus/lib/suggestion-list-element.coffee:175:13
  at Array.forEach (native)
  at autocomplete-suggestion-list.SuggestionListElement.renderItems (/home/alban/.atom/packages/autocomplete-plus/lib/suggestion-list-element.coffee:98:11)
  at autocomplete-suggestion-list.SuggestionListElement.itemsChanged (/home/alban/.atom/packages/autocomplete-plus/lib/suggestion-list-element.coffee:47:6)
  at autocomplete-suggestion-list.SuggestionListElement.attachedCallback (/home/alban/.atom/packages/autocomplete-plus/lib/suggestion-list-element.coffee:16:6)
  at OverlayManager.module.exports.OverlayManager.renderOverlay (/usr/share/atom/resources/app/src/overlay-manager.js:42:24)
  at OverlayManager.module.exports.OverlayManager.render (/usr/share/atom/resources/app/src/overlay-manager.js:19:16)
  at [object Object].module.exports.React.createClass.componentDidUpdate (/usr/share/atom/resources/app/src/lines-component.js:149:71)
  at CallbackQueue.mixInto.notifyAll (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/CallbackQueue.js:75:22)
  at ReactReconcileTransaction.ON_DOM_READY_QUEUEING.close (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactReconcileTransaction.js:88:26)
  at ReactReconcileTransaction.Mixin.closeAll (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/Transaction.js:214:42)
  at ReactReconcileTransaction.Mixin.perform (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/Transaction.js:155:16)
  at ReactUpdatesFlushTransaction.Mixin.perform (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/Transaction.js:141:20)
  at ReactUpdatesFlushTransaction.mixInto.perform (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactUpdates.js:98:38)
  at Object.flushBatchedUpdates (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactUpdates.js:176:19)
  at ReactDefaultBatchingStrategyTransaction.Mixin.closeAll (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/Transaction.js:214:42)
  at ReactDefaultBatchingStrategyTransaction.Mixin.perform (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/Transaction.js:155:16)
  at Object.ReactDefaultBatchingStrategy.batchedUpdates (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactDefaultBatchingStrategy.js:70:19)
  at Object.enqueueUpdate (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactUpdates.js:209:22)
  at [object Object].ReactCompositeComponentMixin.forceUpdate (/usr/share/atom/resources/app/node_modules/react-atom-fork/lib/ReactCompositeComponent.js:1232:18)
  at /usr/share/atom/resources/app/src/text-editor-component.js:313:28

Commands
 28x -2:08.1 wheel (atom-text-editor.editor.is-focused.autocomplete-active)
     -2:03.5 editor:fold-selection (atom-text-editor.editor.is-focused.autocomplete-active)
     -2:00.9 core:undo (atom-text-editor.editor.is-focused.autocomplete-active)
  7x -2:00.0 wheel (div.underlayer)
     -1:59.1 core:paste (atom-text-editor.editor.is-focused.autocomplete-active)
 71x -1:58.6 wheel (atom-text-editor.editor.is-focused.autocomplete-active)
  2x -0:46.3 core:backspace (atom-text-editor.editor.is-focused.autocomplete-active)
 44x -0:40.8 wheel (atom-text-editor.editor.is-focused.autocomplete-active)
     -0:26.6 core:delete (atom-text-editor.editor.is-focused.autocomplete-active)
  4x -0:14.1 core:move-down (atom-text-editor.editor.is-focused.autocomplete-active)
 21x -0:07.3 core:select-down (atom-text-editor.editor.is-focused.autocomplete-active)
     -0:03.3 core:delete (atom-text-editor.editor.is-focused.autocomplete-active)
  3x -0:02.7 core:select-up (atom-text-editor.editor.is-focused.autocomplete-active)
     -0:01.4 core:delete (atom-text-editor.editor.is-focused.autocomplete-active)
     -0:00.6 core:move-down (atom-text-editor.editor.is-focused.autocomplete-active)
     -0:00.0 core:backspace (atom-text-editor.editor.is-focused.autocomplete-active)

Config
{
  "core": {}
}

Installed Packages
# User
autocomplete-plus, v2.1.3
go-plus, v3.2.1
linter, v0.12.0
linter-csslint, v0.0.11
linter-jscs, v1.10.0
linter-jshint, v0.1.1

# Dev
No dev packages

/cc @atom/core

Please update your version of Atom to 0.194.0 or greater.

Also, please update autocomplete-plus and other packages to the latest version.

Ok, will do it.
Thanks.
2015-05-04 20:26 GMT+02:00 Joe Fitzgerald<EMAIL_ADDRESS>
Also, please update autocomplete-plus and other packages to the latest
version.
—
Reply to this email directly or view it on GitHub
https://github.com/atom-community/autocomplete-plus/issues/433#issuecomment-98804834
.


GraphicsCreator should not append Graphics object to displayList by default

A bug introduced in b23ad3b1306eeb9ff3c59a506125467d308a363d causes the GraphicsCreator (this.make.graphics(...)) to add Graphics objects to the displayList by default.
Other game objects delegate this responsibility to the Factory, rather than the Creator.

Thank you for submitting this issue. We have fixed this and the fix has been pushed to the master branch. It will be part of the next release. If you get time to build and test it for yourself we would appreciate that.


[Bug]: lsd (eval):1: command not found: _lsd

Version

[X] latest

version
lsd 1.1.2
What OS are you seeing the problem on?
No response
installation
brew
term
screen-256colors
ls-colors
No response
What happened?
When I type "lsd " in my zsh shell and hit <Tab>, it outputs the following:

$ lsd (eval):1: command not found: _lsd

I'm not sure if it's an issue with lsd or zsh but other commands' autocompletions seem to be working fine.

### What expected?

```markdown
I expect completions, e.g.:

$ lsd <Tab>
some-dir/
somefile.txt

What else?
No response

Seems I fixed the issue by running zplug update


[Bug]: Wrong version displayed

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

RAGFlow workspace code commit ID
3307
RAGFlow image version
0.13.0
Other environment information
No response
Actual behavior

Expected behavior
见上图。
Steps to reproduce
见上图。

Additional information
No response

Fixed. Pull the dev version of image.


SkyShell should exist on Mac

From @eseidelGoogle on June 22, 2015 23:27
This would make it possible for us to be able to run test on a Mac.
This should be possible now after the recent refactorings of SkyShell for ios.
@chinmaygarde @collinjackson @johnmccutchan
Copied from original issue: domokit/mojo#258

From @eseidelGoogle on June 23, 2015 16:6
If we had a mojo_shell on mac, we could use it.  But sky_shell is much less code to port and would serve our immediate needs.

From @tonygentilcore on June 23, 2015 9:6
Is there a strong reason this shouldn't be done in the context of mojo shell instead of sky shell?
/cc @qsr

From @viettrungluu-cr on July 7, 2015 16:50
Sure, it's currently trivial to get mojo_shell building/running on a Mac.
But you'd be very disappointed when you realized that there's no native
viewport service, which would be a huge amount of work to get running. (And
as we make mojo_shell more interesting, e.g., by adding sandboxing, it'd
become more and more of a burden to support Mac.)
On Mon, Jul 6, 2015 at 4:56 PM, Tony Gentilcore<EMAIL_ADDRESS>wrote:

I thought @jamesr https://github.com/jamesr was carrying a hacky patch
in his client at one point in time that allowed mojo shell to be built on a
mac and it didn't seem like tremendous amounts of work. Maybe I
misunderstood or misremembered.
Just for my information, would you be willing to expound on the additional
code that has to be ported for mojo shell to work on mac? I'd really like
to be able to have the same nice developer experience for Mojo that Sky
enjoys :)
—
Reply to this email directly or view it on GitHub
https://github.com/domokit/mojo/issues/258#issuecomment-119030570.

From @eseidelGoogle on July 7, 2015 1:6
If he does I'd love to see it.  Bringing up sky_shell on mac now.

From @tonygentilcore on July 6, 2015 23:56
I thought @jamesr was carrying a hacky patch in his client at one point in time that allowed mojo shell to be built on a mac and it didn't seem like tremendous amounts of work. Maybe I misunderstood or misremembered.
Just for my information, would you be willing to expound on the additional code that has to be ported for mojo shell to work on mac? I'd really like to be able to have the same nice developer experience for Mojo that Sky enjoys :)

I think @chinmaygarde is nearly done with this.

SkyShell exists on mac now.


One namespace is enough in my opinion

In my opinion there are too many namespaces. For such a small project, I think uvgrtp is the only namespace needed. Now there are many other namespaces such as formats, zrtp_msg etc..
As a sidenote, can we get rid of the old namespace uvg_rtp?

Fixing this has the problem that in some cases it affect the public API of uvgRTP and thus it is probably better to do with 3.0 release for at least the public facing parts, if those need changing.

I'm starting to think that this will not get changed in the near future, if at all, and it is not worth having an issue for this at this point.


Added a new symbol

I couldn't think on a good name, also it requires a powerline font

Yes it would be good to make the name a bit more descriptive if possible. Are you able to screenshot the symbol?


Look at the OSS Wave Apps for ML parts

Goal
Check how the H2O-3 in OSS Wave Apps are incorporated and see if it makes sense to use wave-ml library instead.
cc @mtanco

We should attempt to use it in the credit risk app, that's the only one as of now that uses H2O-3.

https://github.com/h2oai/wave-apps/tree/main/insurance-churn-risk does too!

Ahh yes, so thats where I mixed up the predictor names by putting the Churn Predictor into the credit app!

Done here: https://github.com/h2oai/wave-apps/issues/41


Bump com.gradle:develocity-maven-extension from 1.22.2 to 1.23

Closed in favor of https://github.com/gradle/develocity-bamboo-plugin/pull/239


Adding InvocationId to FunctionStartedEvent

To allow for start/end correlation in ETW events. @akurmi needs this.

Looks good.


Can't use tensor_parrallel with chatglm2-6B

when i try to test tensor_parrallel with chatglm2-6B, something went wrong.

Please share the branch you use. If you don't use the latest main branch, please take a try.


v0.0.1

Implement core features

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 90.336%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
162

Relevant Lines:
166

💛  - Coveralls


Languages : ISO 3 letters

Hi,
I'm using Mozilla Firefox in Taqbaylit or Kabyle language and noticed after using AmIunque that the mentionned language is georgian "ka" instead of "kab". The details of the headers are OK on the Details tab.
I'll do more tests later on.
Best regards,

Hi,
AmIUnique collects the list of languages through the HTTP Protocol, so collected languages are the preferred language for displaying pages, you can see them in Firefox in Menu->Options->Language and Appearance (Section)->Language->click Choose...
If your Mozilla Firefox is in Taqbaylit (Kabyle) language, it means FireFox will be displayed in that language, so AmIUnique doesn't collect it.
Best regards.

It seems there is a gross misunderstanding of the bug report. The browser of the original reporter was set up to use kabyle, whose code is 'kab'. AmiUnique, wrongly, truncates it to two characters, leaving" ka", which is a completely different language. This is this truncation, the problem!

Hello,
Here is an illustration of the truncated kab :
https://cloud.indie.host/s/1dJMaugRWTRlZpN
Regards,

Hi BoFFire, thanks for reporting this to us.
We've identified the source of the problem and we'll try to fix it next month when we roll out a major update of the whole website.


34 - Set kserve version in demo notebooks to 0.11.0 to fix tay import error

Kserve package version fix approved.


Can you use OnTriggerEnter for this?

I've been trying to get some sort of OnTriggerEnter script to work, but whenever and wherever I put the Sphere Collider on one of the hands, (Sphere, LeftHandController, RightHandController) I either one, cannot move and get stuck in the same position, or two, I'm able to move my hands but I can't move them forward in any direction, I'm still in the same position as where I spawned. I've tried to put two Sphere Colliders, one Trigger and one Normal collider, but that doesn't work either. Any help would be appreciated.

so on the lefthand right hand does it have tag HandTag?? and for each hand give it left=LeftHand right= RightHand spelt exact


Godot 4.0-dev fails importing a specific glTF 2.0 file

Godot version
4.0-dev (20211027)
System information
Windows 10 (NVIDIA Geforce GTX 1060 6 GB, 64 GB RAM, Core i7)
Issue description
When you import this glTF2.0 file containing a scene (MainScene.glb) Godot 4.0-dev imports but the textures have glitches in the importing, but when I import the same glTF 2.0 scene file in Godot 3.4 the file imports well without glitches. In blender 2.93 the glTF 2.0 file imports correctly.
Godot 4.0-dev glb glitched:

Godot 3.4 glb imported correctly:

Steps to reproduce

Download Godot 4.0-dev 20211027 from tuxfamily repo here
Open Godot
Create a new project
Download glTF 2.0 scene file  (here)
Copy the glb file into godot project directory
Preview imported glb file
Then the glitches appears

Minimal reproduction project
There is no need of a project. You can make your own with the glTF scene file.

This is the glTF 2.0 file imported in blender scene:

This is in godot 3.4 and 4.0-dev imported scene:

It seems some inconsistencies in this part too in the two godot versions.

First contact.

the floor is flickering. Changing the transparent mode to transparent changes observed results. The floor doesn't flicker.

Does this occur if you disable LOD generation in the glTF import settings and click Reimport?

LOD option does nothing and therefore changing mesh lod has no effect.

My guess is that it's related to the AABB being negative.

@clayjohn @reduz @JFonS @lawnjelly Any ideas?

My guess is that it's related to the AABB being negative.

In my game I have a lot of models and they import fine, still throwing a lot of AABB being negative errors though.

Can confirm the scene still flickers on my fork of godot engine.

Could well be if you are setting a negative AABB.
I'm not familiar with the parts of the engine that calculate the AABB, or reduz BVH that is currently used for culling in 4.0. I expect it just transforms the base AABB and expects it to be positive (or zero area). But am just guessing here.
So if you have a negative scaling you may have to flip the AABB on import before setting it on the MeshInstance (if you can find the code that does this, there may be existing compensation for this, at least in 3.x, or maybe it is your job in the importer).

Had another little look at this today with @fire , he exported a smaller version (or separated) of the scene, as the original scene was crashing 3.x and blender 2.82 when I tried to import.
It seem to work fine for me in vulkan mobile, but displays the flickering in vulkan clustered.
This would seem to rule out the importer, and probably also juan's BVH which is used for the render tree (as presumably it is used the same in both vulkan backends).
I've also been getting a bunch of RID errors, which I don't know is common for guys using Godot 4 as I mainly work on 3.x.

This bug is still valid on beta1.

Can someone check https://github.com/godotengine/godot/pull/67176? if the bug is related?

Can someone check #67176? if the bug is related?

Just checked, https://github.com/godotengine/godot/pull/67176 does indeed fix this bug :)
Closing as fixed by https://github.com/godotengine/godot/pull/67176


cibuild.sh: Use new version of mono_msbuild_bootstrap* for mono builds.

This fixes build with mono >= 4.4.x

LGTM


Remove unnecessary code

It doesn't seem right to remove them.
UPDATE could be used to acquire a transaction lock ahead.
It needs more comments, but not removal.

There is already a memory lock so that I think they are unnecessary any more. I will add some comments here.


how to flash firmware?

How can I flash it?

http://www.st.com/en/development-tools/sw4stm32.html

what's you'r compiler? is it yagarto??? if yes , how can i to can compile it?


Chapter 6 , effects doesn't display returned users

loader is still displayed when http request for users completes

@datasmithadvtech with what exercise you have this problem? If with using-ngrx-effect, then this is intended - on page 224 you can see 'Important note' describing that.


unknown tasks

The example in the documentation doesn't appear to work.  i'm using grunt 0.4.5 with this Gruntfile:
module.exports = function(grunt) {
grunt.initConfig({
    copy: {
        somefiles: {
            src: "./mydir/**",
            dest: "./myNormallyOutputDir/"
        }
    },
    multidest: {
        copy_some_files: {
            tasks: [
                "copy:somefiles"
            ],
            dest: ["./dist/distdir1", "./dist/distdir2"]
        },
    },
});

grunt.loadNpmTasks('grunt-multi-dest');

}
Running "grunt multitest:copy_some_files" yields this output:
Running "multidest:copy_some_files" (multidest) task
Warning: Task "copy:copy_some_files_copy_somefiles_subtask_0_destdir_0" not found. Use --force to continue.
Bug, or issue with the example?

Oh, never mind.  Missed a dependency.  Sorry about that.


Add the ability to add a copyright

References #237 . Some license types such as BSD require the ability for binaries to reproduce their copyright.
Adds:
app.Copyright = "(c) 2015"

Which yields:
COPYRIGHT:
   (c) 2015

Following the GLOBAL OPTIONS section.

@polds seems reasonable, thanks for the contribution!

@polds looks like this adds an additional blank line if the copyright isn't set -- I think you just need to drop line 34.

ok @jszwedko , I have removed the whitespace.

Thank you!


update hover effect on navbar & fix navbar in page

The .navbar element is fixed at the top, spans the full width, and centers its child elements using flexbox.
The .nav-links element maintains its style and hover effects, with the background color and text color changing smoothly on hover.
https://github.com/mdazfar2/HelpOps-Hub/assets/95361125/da4a4adb-18a4-4f2f-b58d-d07edb531317
Issue : #586

@mdazfar2 please review this

Hi @heli0707, thanks for your pull request! Unfortunately, we can't merge it right now.  We've recently shifted development to Next.js, and your PR targets our previous HTML & CSS codebase.


fix wrong order of entities being parsed in metakg cnstruction

MetaKG construction is creating operations with object/subject in the wrong order going against the pattern that they are stored in the file being read. This PR fixes that order and correctly gets results from previously not working queries.

Note: Automat APIs are doing predicate expansion. This basically leads to "duplication" of edges for BTE since it doesn't recognize that it is getting the same data from a "related_to" query as it did from the corresponding "correlated_with" query.
This will likely have to be addressed in a separate issue...maybe in api-response-transform...so BTE doesn't keep records from Automat APIs where its predicate field =/= the smartapi metadata predicate
Example: POST to https://automat.renci.org/covidkopkg/1.1/query. Even though the query given used the "related_to" predicate, all the edges actually have correlated_with edges. BTE will take something like this and assume all the edges have the "related_to" predicate when they actually dont'...
{
    "message": {
        "query_graph": {
            "nodes": {
                "n0": {
                    "ids":["PUBCHEM.COMPOUND:1935"],
                    "categories":["biolink:SmallMolecule"]
                },
                "n1": {
                    "categories":["biolink:MolecularActivity"]
               }
            },
            "edges": {
                "e0": {
                    "subject": "n0",
                    "object": "n1",
                    "predicates": ["biolink:related_to"]
                }
            }
        }
    }
}

Re-review of the behavior described here:

no longer gets answers is ~striked-out~
new apis (not seen previously) are in bold
apis seem in the other queries above it are in ()

The APIs being called by these queries have changed (no longer is ~striked-out~, repeated apis are in ()):

ChemicalEntity CHEBI:3558 -> Gene uses covidkop, ctd, pharos, ~CORD19~
PhenotypicFeature HP:0004382 -> Disease uses Biolink, Uberongraph, (and covidkop) ~Automat Ontology Hierarchy~
Disease MONDO:0009747 -> Gene uses Robokop (and covidkop, Pharos, Uberongraph)
ChemicalEntity CHEBI:30830 -> Disease uses CORD19, hetio, (and biolink, covidkop, ctd, robokop) ~hmdb~
Disease MONDO:0011565 -> Disease uses (biolink, cord19, covidkop, robokop, uberongraph)~hetio,ontology hierarchy~

AnatomicalEntity UBERON:0001905 -> AnatomicalEntity uses Automat Text Mining (as well as Uberongraph)

My suggestion is to merge / push to prod ASAP.
Then we can handle any bugs we see with Automat down the road...

We also expand the predicate filter I think, so if they expand it I don't think that's a problem is it? if we both have correlated_to as the predicate it should get results if I'm right

In that above example, the problem is that we are querying their API both for related_to edges and correlated_with edges...and getting the same data set for both and not recognizing that they're the same.
We are instead saying there's a set of edges with related_to + a set of edges with correlated_with...

Looks like TRAPI v1.2 APIs might not actually do this...so it's not a problem.


problem with setHTML and getHTML

We use an external data source for our documents and use etherpad for collaborative editing. In order to integrate etherpad into our system we are planning to use the etherpad apis to create a pad and set the content using the setHTML api and once done use the getHTML to capture the content and save it back to our system. The problem we are running into the styles are not honored on the setHTML. On the other side the getHTML does not return the styles and therefore making the result HTML useless for use in other HTML based clients. I noticed that doing a HTML export from the etherpad client returns the styles which is not the case for the getHTML. I am using the font size as an example but there are other formatting and styling requirements that we must maintain as we bring in/out documents to and from etherpad. What are we missing or doing wrong to avoid these issues?
Thank you
Shawn

I can only really look at the font size stuff within this scope, I'm available as a gun for hire if you need direct support and that's pretty much the best way to get me onto projects.  Apologies if this is less helpful than you were hoping for! :)

Should be fixed now, give me a shout if not.


Deal with SELinux permissions when starting the Fn container

On systems where the SELinux security policies are enforced, fn start is not able to mount the current directory as a writable volume in the spawned Fn Docker container (this causes Fn to be unable to write to the default database), nor to run any container through the mount of the local docker socket.
Furthermore, Fn is not able to run containers with the docker-in-docker solution either. Something in SELinux stops it.
As OEL 7.x and other enterprise Linux systems use SELinux, we need to find a solution that lets us run Fn on enforcing systems.

This issue is more complicated than it looks, so I've updated the description.

The docker-in-docker solution seems to be working but it can't access local images deployed with fn deploy --local.

this is going to be the case for every docker-in-docker with an fn deploy --local, not just this one. not sure what kind of wizardry we could apply to let the dind use the host docker's images, highly possible that doing -v /var/lib/docker:/var/lib/docker 'just works' but I haven't tested it.

This is two different things right?  The disk write issues are different from docker in docker issues?  Or no?
Regarding the disk issues, another option would be to just have them run another db instead of sqlite, like docker run ... mysql.


question: improving joint GT for trio's

Hi,
Currently our production pipeline uses the bcbio "batch" to differentiate between trio's on a single flowcell. We use GATK for variant calling up to gvcf, and then do joint genotyping per batch.
However, this is less than ideal since the batch sizes are small (3 individuals). We were thinking to improve on this by joint genotyping all samples on a single flowcell, and differentiating the trio's later on.
My question is:

What's the best way to differentiate between families (trio's) within the batch (e.g. the flowcell)?
Do I use a giant PED file with all data or do I add a ped file per trio and associate the small ped with the correct samples in the config? How will the affect the resulting gemini db?
Alternatively, is there a way to make bcbio joint genotype the entire run and then split the resulting vcf per trio?

Thanks
Matthias

Hi Matthias,
Good question. Samples in the same batch will all get called together, so you could do them all in the same batch and then provide a PED file which describes the family structure. This is good if you are going to be comparing across families, since it will ensure all locations where there is a variant in any sample has calls for every sample.
Joint genotyping is only really useful if you have on the order of thousands of samples, otherwise just calling them all in the same batch should work fine.
There isn't a way to break up the VCF into trios in bcbio at the end, but you should just be able to keep the one big file and subset it however you like. We use GEMINI to do those types of more complex queries.

Thanks for the info!
I noticed vcf2db support bigger db's too? Can we somehow modify the behaviour of bcbio to incrementally push new data to a postgres instance instead of sqlite?
Thanks
M

Hi Matthias,
bcbio uses vcf2db to load the GEMINI database, yup. Incrementally updating the database doesn't work well because the new samples added need to have the variants called all at the same spots as the old samples, so you need to re-call and create a squared off VCF file anyway.

Thanks for the info!
Cheers
M


Port standard instructions to Rust.

Summary
Adds a new Rust enum StandardInstruction representation for standard instructions (i.e. Barrier, Delay, Measure and Reset) and updates the encoding of PackedOperation to support storing it compactly at rest.
The size of a PackedOperation has now been anchored to always be 64 bits, even on 32-bit systems. This is necessary to encode the data payload of standard instructions like Barrier and Delay. See the docstrings within packed_instruction.rs for details.
Details and comments
The implementation works very similarly to what we currently do for StandardGate, but with a bit of extra consideration for an additional data payload used by variadic/template instructions like Barrier and Delay.
Similarly to StandardGate, the newly added StandardInstruction enum serves as the first-class Rust interface for working with standard instructions. The existing OperationRef enum returned by PackedOperation::view has a new variant StandardInstruction which exposes this.
Unlike StandardGate, StandardInstruction is not a pyclass because it is not directly used to tag instructions as standard in our Python class definitions. Instead, the simple integral enum StandardInstructionType is used as the tag, and OperationFromPython::extract_bound further queries the incoming Python object for variadic/template data when building the complete StandardInstruction.
Encoding
The PackedOperation encoding has been updated as follows:

A PackedOperation is now always 64 bits wide even on 32 bit systems, and is now a wrapper around a custom union type, BitField. This union contains three bitfield!(u64) types, StandardGateBits, StandardInstructionBits, and PointerBits, defined using the bitfield-struct crate, which gives us compile-time validation of bitfield layout and eliminates the need for manual masking and bit manipulation. Similar to a CPU ISA, the discriminant identifies which bitfield / layout should be used when decoding the rest of the operation. See inline doc strings for more detail.
The discriminant has been widened from 2 to 3 bits (it is now at its maximum width, but we still have room for 3 more variants).
The discriminant now has additional variant StandardInstruction.

Todo:

[X] Add more detail to PR description.
[ ] Actually test this on 32 bit systems and a big endian 64 bit arch.

I'm concerned that the LoHi union is making this easier to interpret the wrong data, and making it easier to make mistakes on BE and/or 32-bit systems, because the encodings now change between them. It's also effectively an unrestricted transmute (to reinterpret the whole thing using the union) on top of another transmute (the bytemuck stuff), and the purpose of introducing bytemuck was to avoid writing any of that ourselves.
I feel like the bitshift and masking operations could be extended over the whole u64, and that'll all automatically work on BE and 32-bit systems, especially with the size of PackedOperation now fixed to 64 bits - we can even have everything be aligned. The shifts of the payload of instructions can be set such that a u32 payload is always in the high bits, with the padding bits between it and the rest of the discriminants if you're concerned about aligned access, because then the bitshifts will be compiled out into a single register load (just read the u32 from the right place) and the compiler will handle endianness for us.
The LoHi struct to me seems to be forcing every method of PackedOperation to think about endianness all the time, and is effectively making us write the partial register reads/loads ourselves. If we use shifts and masks on a u64 with const masks and shifts, the compiler will do all that work for us, and it's a lot less likely to make a mistake.

I guess my main point is this: the 64-bit PackedOperation can always be thought about as a collection of payloads each of which is stored in a particular mask and needs a particular shift.  These are:

the PackedOperation discriminant at (0b111, 0)
a pointer at (usize::MAX as u64 & 0b000, 0) (regardless of 32-bit or 64-bit)
a StandardGate discriminant at (0xff << 3, 3)
a StandardInstruction discriminant at (0xf << 3`, 3)
a DelayUnit payload at (for example) (0x7 << 32, 32)
a u32 payload at (u32::MAX as u64 << 32, 32)

Introducing LoHi doesn't remove the need to bitshift and mask for most items, it just means that some of my above list are done one way, some are done another, and the union means that the pointer now has more ways to access it.  LoHi also restricts the payload size to u32, when we already have payloads that exceed that (the pointers).
If you want a shade more encapsulation than my loose const shift and mask associated items, would something like this look better to you?
struct PayloadLocation<T: SomeBytemuckOrCustomTrait> {
    mask: u64,
    shift: u32,
    datatype: PhantomData<T>,
}
impl PayloadLocation {
    #[inline]
    fn store(&self, src: T, target: u64) -> u64 {
        let src: u64 = bytemuck::cast(src);
        target | ((src << self.shift) & self.mask)
    }
    #[inline]
    fn load(&self, target: u64) -> T {
        bytemuck::cast((target & self.mask) >> self.shift)
    }
}

const PACKED_OPERATION_DISCRIMINANT = PayloadLocation { mask: 0b111, shift: 0 };
const STANDARD_GATE_DISCRIMINANT = PayloadLocation { mask: 0bff << 3, shift: 3 };
const POINTER_PAYLOAD = PayloadLocation { mask: usize::MAX as u64 & 0b000, shift: 0 };
// and so on

(Bear in mind I just typed that raw without trying it, and I didn't think all that hard about what the trait bound should be.)

Apologies for the delay in responding to your initial comments, @jakelishman! I did read them, and they were ultimately what led me to rewrite the initial draft using bitfields. I meant to reply to your comments at the same time I pushed up these changes, but didn't have enough time yesterday.
Regarding the initial draft:

The shifts of the payload of instructions can be set such that a u32 payload is always in the high bits, with the padding bits between it and the rest of the discriminants if you're concerned about aligned access, because then the bitshifts will be compiled out into a single register load (just read the u32 from the right place) and the compiler will handle endianness for us.

This is exactly what I was concerned about, though I was also trying to accomplish it in a way that the alignment of PackedOperation could be 4 on 32-bit systems, which is partly why I'd ended up with LoHi. In the current approach with bitfields, I've forgone this. The alignment of PackedOperation is now always 8 bytes, even on 32-bit.

Introducing LoHi doesn't remove the need to bitshift and mask for most items, it just means that some of my above list are done one way, some are done another, and the union means that the pointer now has more ways to access it. LoHi also restricts the payload size to u32, when we already have payloads that exceed that (the pointers).

This was true, and ultimately indicative of me trying to jam everything into a single instruction encoding / layout.
I liked the PayloadLocation example you provided, and I actually spent some time locally building it out. I couldn't shake the feeling though that we were implementing a poor-man's bitfield library (and, one without static layout validation at that), so I started poking around and found the bitfield-struct crate. It's really excellent. The author has also already merged a PR I made to optionally disable the derived Clone implementation on its bitfields.
Regarding the current version:

My biggest worry about bitfield-struct as written here is about accessing the PackedOperation discriminant - getting it from the StandardGateBits representation is, I believe, UB. We can swap that to the pointer form, but the fact that it's so easy to produce UB like that makes me nervous.

This I'm not convinced of. Type-punning is allowed in Rust and is not UB, as long as the access doesn't produce an invalid value, if I understand correctly. It's also of course important that we're not reading uninitialized memory, which we shouldn't be here.
From https://doc.rust-lang.org/reference/items/unions.html#reading-and-writing-union-fields

Unions have no notion of an “active field”. Instead, every union access just interprets the storage as the type of the field used for the access. Reading a union field reads the bits of the union at the field’s type. [..] It is the programmer’s responsibility to make sure that the data is valid at the field’s type. Failing to do so results in undefined behavior.

If you're worried about the representations of the fields comprising the union, they are all #[repr(transparent)] structs around a u64. So no matter which field we access, the data gets interpreted as a u64, which is valid. And, the ::new generated by bitfield-struct starts off by setting the inner u64 to 0, so there shouldn't be any case where we are interpreting uninitialized data. The discriminant is located (and statically asserted to be) in the same place for all bitfields, so that should be safe too.

My other hesitation is that generally these aren't entirely bitfields, because of the overlap. All the extra code needed to support the masking out of the low bits of the pointer feels more complex to me than it was before, and I'm not convinced that bitfield_struct is offering us enough in return. If anything, it feels like we might have spread the dangerous assumptions further around across more types and more (type-level) indirection than there was before.

In the 64-bit pointer case, I was also thinking about it as "overlapping" a pointer with a discriminant originally, and that was the reason I hadn't thought to use bitfields. But really, there's no overlapping here. We're simply storing the upper 61 bits (the interesting ones!) of an address inside of a 61 bit field. I'd argue that the code is less complex in terms of bit manipulation: there's no explicit masking at all, we just shift an address down by 3 and store it.

I worry that it's using a pneumatic drill to solve a hammer-and-nail problem here since there really are not many fields at all in our structs, and these structs should not need much manipulation other than construction. My perception of the complexity of manual bitshift-and-mask operations compared to this might be different to other people's.

The original approach was certainly the right one at the time. In earlier commits, I explored extending it for StandardInstruction, but it felt very easy to read/write the wrong part of the field or make the wrong cast with the added complexity. The bitfield approach validates at compile-time that fields aren't overlapping and lets us model PackedOperation's layouts declaratively. I think this is actually quite elegant, overall.

For the UB: bytemuck made it near impossible to get this wrong.  This implementation with bitfield_struct may be right, because we don't call the problematic getter, and the value isn't constructed til then, but it's nowhere near as obvious to me with the new form, and it feels a lot more dangerous to even access an incorrect union type - much easier to change the code to get it wrong when you've already "constructed" the wrapper type that would allow the invalid access, as opposed to still explicitly treating it as "this is a bag of bits".

Re the pointer overlap: a pointer is 64 bits, not 61 bits, so what the bitfield is storing isn't a pointer, it's as you say, the informative bits of the pointer - the true value of the actual pointer really is overlapped with the discriminant.  Strictly, x86_64 pointers are only 48 bits wide and required to be all 0s in the top 16 bits, so we might even want to use those bits too in the future, and they'd be overlapping.  Then it'd really be clearer as a single mask, not a bitfield.

The bitfield approach validates at compile-time that fields aren't overlapping and lets us model PackedOperation's layouts declaratively. I think this is actually quite elegant, overall

I mentioned before that imo, the fields do overlap for pointers, and we're fighting bitfield_struct in those cases because it's trying to assert that nothing actually overlaps.  About elegance: I do appreciate that you can think about it as an extension of bitfields, but my reading of the resulting code here is that there's more boilerplate accessors/whatever than there was before, and there's a lot more complexity in the type system which is requiring more assumptions to be spread between different objects in the code.  To me, the reasoning about this new system is less local (it's the subject of a few of my comments).
I can totally believe that the bitfield_struct crate is really neat.  My problem is that the pointer object just isn't a true bitfield - it's a really simple "mask out the low bits" trick, and we're adding a lot of complexity around that to use an abstraction that doesn't fit quite right.  It might be super obvious to you, having played around with the bitfield_struct crate a load, but it adds more complexity for me reading the code, and a whole other library that I need to know the intricacies of.
Fwiw, my preferred form probably isn't the PayloadLocation encapsulation I attempted above.  As a rough sketch, I was thinking more like: define an unsafe sealed trait that StandardGate and StandardInstruction implement, which supertraits Copy + !Drop, and has methods that encode them into a u64 and back again, that's defined on them (so the reasoning about the data representation is local to those types, including the choices of how to encode themselves), and has a safety comment that they must carry no information in the low three bits.  The Copy + !Drop requirement is what PackedOperation really requires for those right now (though tbh there's probably ways to avoid that), and other than that, the "low 3 bits will be overwritten" is the only thing that the type itself needs to know to implement things.  In that form, there's no shift operations needed within PackedOperation itself, so little to get wrong - PackedOperation does
struct PackedOperation(u64);

fn do_something(&self) {
    let discriminant = self.0 & Self::DISCRIMINANT_MASK;
    let payload = self.0 & !Self::DISCRIMINANT_MASK;
    match discriminant {
        PackedOperationType::StandardGate => <StandardGate as Packable>::unpack(payload),
        PackedOperationType::StandardInstruction => <StandardInstruction as Packable>::unpack(payload),
        PackedOperationType::PyGate => ...,
    }

and all the other bit-twiddling / whatever is local to the types themselves.  In cases where a bitfield representation is appropriate (certainly I could be convinced by StandardInstruction in that case), then we could do that.  I don't think the current implementations of the pointer-like objects or StandardGate are served by the bitfields, though.