RabbitMQ config additions

This is just some ideas for tuning the cluster. I've found it not very stable in my production Rancher environment so I'm thinking of parameters to tune. Any additions to this?

I can't do any test at the moment but I'm going to merge it and build the image on docker hub

@andyshinn I've just created 0.2.0-testing image on docker hub to test the new version and started a new RabbitMQ stack on Rancher changing only the -conf version from 0.1.0 to 0.2.0-testing but something is not working.
The first node goes up but the -conf container reports:
29/07/2016 15:06:352016-07-29T14:06:35Z rabbit-test_rabbitmq_2 /confd[1]: ERROR Unable to process template /etc/confd/templates/rabbitmq.tmpl, template: rabbitmq.tmpl:3: unexpected {{end}}
Scaling causes the new containers going up but the new rabbit node is not joining the cluster and -conf reports same error as before.
Note: I'm working on a separate branch: https://github.com/ddmng/catalog-dockerfiles/tree/andyshinn-rabbitmq_config
As soon as it will work I think I will have to create the 0.2.0 version:

on catalog-dockerfiles
in the corresponding -conf built image on docker hub
on community-catalog
Than I will have to make a new PR to Rancher guys to merge on main catalog.
What do you think?

Oops, there is extra {{end}} in the conditionals a the end of line. I will try it out more soon.

Extra {{end}} later in line 11 too; I tried to fix that config but still doesn't work. I added you as collaborator on the docker hub rabbitmq-conf repo so that you can push there and test quickly.

I've forked this repo and produce my own version with additional configs. However I completely removed conf image and extend original rabbitmq image with confd. It also has rancher template
You can have a look here https://github.com/inabhi9/rancher-rabbitmq
Let me know what do you guys think. I can make a pull request if it makes sense.

Hi @inabhi9 !
Other stacks in Rancher have their own confd service and it seems reasonable to stick on the official images in services where possible: with this in mind I separated the confd from Rabbit and pointed to the offical 3.6.1-management RabbitMQ DockerHub tag. I assume you are using something like supervisord to start both rabbitmq-server and confd and IMHO it's against the logic of microservices, where each container represents a service/process.
I think the best way to bring in this stack the new options that you have added is to manage them in the separate confd service, like all the other Rancher stacks.

@andyshinn I can't see what's wrong in this PR these days, but if you have time to fix it I'll give it a quick run

I think these are fine. I'm testing this in my own forked image in production and the config seems fine.
Unfortunately, these options haven't fixed the stability issues I've been having with RabbitMQ over the Rancher network. It is really unstable and I experience weekly cluster splits that never self heal. I'm still trying to figure out the "magic" config to get it working better.

Also, please double check this PR before building. I don't have my fork public (I was just testing with a locally built image and config). I think I may have mis-placed a comma in this. I'll try and check in the next week when I have some time to play with RabbitMQ again.

Hi @ddmng,
Your points are valid and I agree with you. In this case I would suggest @andyshinn If he wants to look at the config file in my repo. I'm using that image for a week without any issue.

What are you using for RABBITMQ_CLUSTER_PARTITION_HANDLING and RABBITMQ_NET_TICKTIME? As far as I can tell, the config is identical other than formatting. Is there a feature of your config that we should add to this one?

I'm using 'autoheal' and default 60 for net tick time with two node cluster.
I tried config of this PR but somehow server couldn't came up. So I thought there might be config file issue.
Also you may consider the env variable for overriding the template entirely.

Hi @andyshinn @inabhi9
I've just merged your contribs here in andyshinn-rabbitmq_config branch and started a 2 nodes cluster on my servers to see if I can reproduce that @andyshinn is reporting. The corresponding catalog is at https://github.com/ddmng/community-catalog and points to the image at https://hub.docker.com/r/rdaneel/rabbitmq-conf/  pushed with 0.2.0-testing tag.
If you can, please run a test on your sides. When we are ready I'll merge the PR under 0.2.0, fix the READMEs and other small stuff and send a PR to rancher.

Hi @ddmng
I also have a suggestion on scaling mode. I saw that you've to do of avoiding multiple rabbitmq server on same host. As rancher doesn't have feature to handle that much complex rule, I recommend to use global scheduler with host tag like other template does. Anyway most of the users, don't want to host multiple instance on same host.
Let me know what do you think.

I will test the new image soon (I am creating new env next week or two).
I would not want default global container since I have mix of data and service nodes in same env. I just create an anti-affinity rule like so:
io.rancher.scheduler.affinity:container_label_soft_ne: io.rancher.stack_service.name=$${stack_name}/$${service_name}

I also tried soft anti affinity rule but thing is if your node goes down
for more a few second rancher may create another instance on same node and
resync will start (if your using HA) when your old host comes back online,
it won't rebalance and user will have to manually rebalance it.
Global scheduler can be used along with labels. I also use multiple
services on same host like role=rabbit, role=api on same host
On 09-Sep-2016 8:49 AM, "Andy Shinn"<EMAIL_ADDRESS>wrote:

I will test the new image soon (I am creating new env next week or two).
I would not want default global container since I have mix of data and
service nodes in same env. I just create an anti-affinity rule like so:
io.rancher.scheduler.affinity:container_label_soft_ne: io.rancher.stack_service.name=$${stack_name}/$${service_name}
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ddmng/catalog-dockerfiles/pull/1#issuecomment-245808061,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB1Qks7oC6tAbxoJQERpazqiYXRKmXz9ks5qoNBIgaJpZM4JVgLz
.

You can't have two of same role tag on same host. Though, you can do something like api=true and role=true. But "global" to me means "all" in this environment, which I am not too keen on (maybe that makes sense once it works really well and stable, but it hasn't proven that to me yet).

Sorry that I just wrote it that way. And yes you're right global means on all in this environment but you can control that using labels and rules that on which x/y/z node should run the service. Does it make sense?

@ddmng I tested new image and config now being generated fine and loaded correctly by the rabbitmq so It should be good to go.
Are you considering the env variable for replacing the template entirely for more advance use case and config?

@inabhi9 I'd like to test the cluster failure scenario you raised. What I've understood from your description is:

I deploy a 3 nodes RabbitMQ on hosts A, B, C in a cluster of 4 nodes (there is also node D then)
node B goes down (I would simulate with a shutdown command)
Rancher pulls a new RabbitMQ instance on node D trying to respect the Scale=3 setting on the stack
node B comes back and starts all the stopped services, including RabbitMQ

Then what have you experienced at this point?
About labels: labels are causing me a bit headache; I'd like to use them only when needed, e.g. when I have Fiber Channel links connecting to a set of servers to a storage or some other physical constrains that services need for working.
If needed, I think we could include a label in RabbitMQ stack BUT it would at least be configurable . I don't know how to specify a label as a rancher-compose user-specified parameter and passing that to the docker-compose configuration. I also asked that in Rancher Forum but no replies: https://forums.rancher.com/t/use-of-rancher-compose-variable-in-label-name/3102 . Any idea on how it would be done?
About custom template: I've just seen the piece of code in your entrypoint and it look useful for testing. I will include it ASAP. As far as I can see in Rancher's UI you could only pass this custom config when upgrading the service, right?

On Failure scenario:
In your strategy, you have 3 nodes A/B/C and one extra node D with scale number of 3 (so assume no rabbit server on D initially). Now,

If B goes down, rancher will start new instance on D (with soft negative affinity rule but no guarantee if not rule is not specify though we assume here it'll)
Now after few minutes, B comes back. In this case rancher won't kill instance on D and resume service on B as scale constrain already satisfied (somewhat related to https://github.com/rancher/rancher/issues/2558). In this case, If user manually kill the service on D, rancher may reschedule on B.
Let's assume that we have rebalancing in place and rancher kills on D and start on B. But then it can be questioned that Do we really want to spin off new instance as soon as desired scale goes down or we want to wait for few minutes? Because if new instance starts up and if rabbit server configured in auto HA mode, it'll start syncing with new node and it's little costly as per rabbitmq doc.
When we want to scale down, then how and who will tell rancher that which instance would be removed? It may possible that in multiple failure some X node became master and rancher may remove it.

Scaling down rabbitmq is little tricky and I feel that it should be kept manual task.
So what do you think?
About labels: I've seem few catalog which uses labale in rancher-compose. I'll try to find it.
On Custom template: Yes it'll be only possible either on upgrade or on initial setup

OK Clear. So this means that Rancher's rebalance algorithm can, in some cases, cause a perfomance issue on RabbitMQ's cluster.
Even in this case I still think that the limitations of adding a fixed label to the community-catalog's Rabbit stack are too many for the advantages they introduce. If there was a way to specify a label in stack's options so that it can be passed to docker-compose then maybe that would be the optimal solution.

About labels: I've seem few catalog which uses labale in rancher-compose. I'll try to find it.
If you can find it, I think many stacks in the catalog would really benefit from it.

@ddmng Here are a few catalog where you can see the host label config used in docker-compose
https://rancher.abhi9.in/env/1a12/catalog/community:janitor?catalogId=library
https://rancher.abhi9.in/env/1a12/catalog/community:sysdig?catalogId=library

@andyshinn @inabhi9
Polishing work done:

updated READMEs on dockerfile and catalog
added confd configuration override via env variable
moved new stuff to 0.2.0 directory and restored 0.1.0 to original content
pushed image rdaneel/rabbitmq-conf with tag 0.2.0 and aligned catalog

Just started on my cluster; deploy and scale seems to work for now.
I keep it running for a while and waiting for some feedback before issuing a PR to the community catalog.

@ddmng That makes sense to make labels optional. I don't know what rancher does in case of scheduling rule with blank labels that needs to tested other than that there is no conditional based definition in docker compose. So for now, keep it as it is.
And great work. Thanks!


(maint) Update deep_merge to 1.0.1 on x86

Use the updated puppet-win32-ruby including deep_merge 1.0.1 for x86
builds. x64 builds already used 1.0.1, so this makes the gem versions
consistent.

:+1:  the only additional commit this update pulls into the build is ae68cf2 (maint) Update deep_merge to 1.0.1, which updates the deep_merge gem here to match the one provided in the x64 installer

CLA signed by all contributors.


Set object table index for from_dask_dataframe

In the loader method from_dask_dataframe, we were mistakenly not updating the object table's index in the case where the object table is supplied as an argument rather than generated from the source table.
Unit tests are updated based on the identifying issue #230 validating the case where the object table isn't generated and that analysis functions can be called on the generated  ensemble.

Looks great! The only minor issue is a warning in the case of pre-existing index
dask/dataframe/core.py:5251: UserWarning: New index has same name as existing, this is a no-op.
  warnings.warn(

Considering this some more, I've decided to open an issue to clarify documentation or provide a parameter to allow users to force the indices to be overridden: https://github.com/lincc-frameworks/tape/issues/232
However I will go ahead and submit this PR as is while I think about the interface a bit more.
Thanks for the reviews!


create-remix@latest results in hydration errors on unmodified build

What version of Remix are you using?
1.12.0
Are all your remix dependencies & dev-dependencies using the same version?

[X] Yes

Steps to Reproduce
% npx create-remix@latest repro
? What type of app do you want to create? Just the basics
? Where do you want to deploy? Choose Remix App Server if you're unsure; it's easy to change deployment targets. Vercel
? TypeScript or JavaScript? TypeScript
? Do you want me to run `npm install`? Yes

Then:
npm run dev

Repro: https://github.com/wulfmann/remix-repro
Expected Behavior
A fresh bare build does not log hydration errors.
Actual Behavior
Hydration errors are logged complaining about a missing closing <head> tag.

OK, I think I figured this one out
I had to disable the Apollo Client Devtools extension to fix this error
This was observed on Firefox Developer Edition 110, as well as Chrome Canary 112

Sorry I should have included I was on chrome. I also had that extension and disabling it indeed seems to have fixed this issue. Nice find!
Is this an issue with the extension then?

Mind. Blown. 🤯 Sorry for the spam, but thanks SO MUCH! That extension caused me hours of looking through the code why stuff would be different.

This issue has been seen before - there's a proposal about it here: https://github.com/remix-run/remix/discussions/5244

Had this issue happening with the Meteor devtools extension too (Chrome 111).

Same issue, disabling all extensions fixed it, I'm guessing the scripts injection of extensions caused this?
pnpm create remix@latest
# node v18.15.0
# Brave 1.50.114 Chromium: 112.0.5615.49 (arm64)

I believe this is a duplicate of the #5244 and will be (hopefully) addressed when React 18.3 is released.  See https://github.com/facebook/react/issues/24430#issuecomment-1440427646 for more information.
I'm going to close this out as there's not much Remix can do it seems.


IV-3915 Make debug level modifyable on the fly

Add /rll/proc/log_level variable for internal use to rl10 section.

Code Coverage is 82.7%, details at https://s3.amazonaws.com/rs-code-coverage/rsc/2016-08-22_720_Coverage-82.7.txt (sometimes the URL can take a few minutes to be available)

LGTM

Code Coverage is 82.7%, details at https://s3.amazonaws.com/rs-code-coverage/rsc/2016-08-24_732_Coverage-82.7.txt (sometimes the URL can take a few minutes to be available)

Code Coverage is 82.7%, details at https://s3.amazonaws.com/rs-code-coverage/rsc/2016-08-24_735_Coverage-82.7.txt (sometimes the URL can take a few minutes to be available)


Removed 'node_modules' directory from index.

The proper way of storing the npm dependencies is store declare them inside the package.json. What you actually already did.
Tested to be working with my fork. mix admin.install crates css and js as usual.
You can read about storing node_modules here:
https://www.quora.com/Should-I-put-node_modules-in-gitignore
http://stackoverflow.com/questions/11459475/should-i-check-in-node-modules-to-git-when-creating-a-node-js-app-on-heroku

Agree and I know better. I did not notice that it was included. Too many projects on the go :( Good catch and thanks!!


[WIP] Using Model in a loop

Create #509  as a test case. This is currently failing.

Thanks!! (I also added a OpenAI version of the test and fixed the bug)

Oh, I wasn't expecting this to have been merged yet.


Update archives.json to add Scottish archives for #107

Per discussion in #111 . This should close #107.

🎉


Conditionally disable error logging

Discussed in https://github.com/johannschopplich/unlazy/discussions/34

Originally posted by chrisspiegl October 25, 2023
I am trying to just display a thumbhash in some instances.
But if I do this with UnlazyImage I always get the error on the console:
[unlazy] Missing `data-src` or `data-srcset` attribute

I agree that this makes sense, but is there a way to make it known that I really don't want any src to load? Or maybe a way to prevent the src from loading in any kind of automatic way and only do so when there is a parameter set to true?
I have achieved something like this with a workaround to manually create the src from the thumbhash with the provided functions. But thought this could be cleaner by providing something of a parameter?
Warm regards,
Chris

@chrisspiegl Closed by https://github.com/johannschopplich/unlazy/commit/110f14a025e2f28d1b4e77a633311ef2eb05c56f.


Improve process detail table styling

Adds similar styling as the docker component read page tables to the process details tables. Should abstract this table styling to a generic underlying component before merging.

LGTM though having 4 different background colours for table cells feels a little busy to me. Nbd.

LGTM though having 4 different background colours for table cells feels a little busy to me. Nbd.

The idea there is to allow easy scanning of rows and columns alike, but since there are relatively few columns in these tables I could remove that and make it a simple 2-color row striping @BenElgar - let me know if you'd like that change made.


Improve frame-skip

For some animations this gives about 20% less renders (like this one: https://asciinema.org/a/20055)
However, if there's super high-framerate animation/printing this doesn't render anything until the pressure gets significantly smaller, resulting in seconds of screen freeze.
The previous implementation was sending all frames to renderer which was batching them and only rendering the latest one from the batch (reagent doing it in requestAnimationFrame).
With this new implementation we would need to "flush" the rendering once in a while (every 1/30th sec?).
Not sure if it's worth the trouble.


Is there a way to parameterize and pass an instance of Startup rather than just the type?

The intention is to avoid creating a separate TestStartUp class rather - rather inject an instance of Configuration
Sample start up below,
public class Startup
{
  public IConfiguration Configuration {get; set;}

  public void Startup()
  {
    Configuration = new ConfigurationBuilder()
              .AddEnvironmentVariables()
              .Build();
   }

  public void ConfigureServices(IServiceCollection services)
  {
     //uses Configuration object to set up everything
  }
}

In the above implementation, if only
TestServer.CreateBuilder().UseStartup(instanceOfStartup)
is permitted
I can do this,
Startup instanceOfStartup= new Startup();
instanceOfStartup.Configuration = myOwnconfigurationWithKeysRequiredForTest;
//create a test server with
TestServer.CreateBuilder().UseStartup(instanceOfStartup)

Advantage: I dont have to maintain multiple startup class.
Does it make sense?

This issue was moved to aspnet/Hosting#333


Added markup-bundle

This PR add the markup-bundle which was introduced by https://github.com/sulu/sulu/pull/2338

@wachterjohannes The other PR is merged, can you update this one, so that the tests are running?


Fix #1725 Update time_UT.c to use UtAssert_MIR

Describe the contribution
Fixes #1725 Updated time_UT.c to start using the new UtAssert_MIR to avoid incorrect use of UtAssertEx.
This update requires https://github.com/nasa/osal/pull/1122 to be merged first. cFS will fail to compile otherwise.
Testing performed
Steps taken to test the contribution:

Pulled OSAL repo with the UtAssert_MIR changes

Rebuild cFS

Run unit tests.... All passed.

Set a custom epoch time

rebuild and run unit tests... All passed.

Manually run ./coverage-time-ALL-testrunner and verify expected output:
[  MIR] 06.004 time_UT.c:902 - Confirm adding seconds =<PHONE_NUMBER>, subseconds =<PHONE_NUMBER> to configured EPOCH results in time 0214-039-06:28:15.99999

Manually run functional tests and verify expected output:
EVS Port1 66/1/CFE_TEST_APP 5: [  MIR] 15.001 es_misc_test.c:67 - MIR (Manual Inspection Required) for CFE_ES_WriteToSysLog

Expected behavior changes
No expected changes in behavior.
System(s) tested on
Ubuntu 18.04 VM
Contributor Info - All information REQUIRED for consideration of pull request
Jose F. Martinez Pedraza / NASA GSFC

Can you also update the assert in es_mic_test.c to the new macro.

CCB:2021-08-04 APPROVED


feat: adds caching support and GH_TOKEN support

Change Log:

Adds compiler caching to vyper runner
Adds support for GITHUB_TOKEN environment var for fetching vyper releases

https://github.com/storming0x/vvm-rs/issues/2


Groups of EditorialObjects to be represented by subclasses of EditorialObjects

The CCDM group has decided to represent groups of EditorialObjects like a "series", a "season", etc. as subclasses of EditorialObjects. Grouping is then represented by "hasMember" / "isMemberOf" relations.

If we are happy with the current class tree, which is a variant of the proposal above, we can close this issue.

Currently, Groups are not subclasses of EditorialObjects but of BusinessObjects. BusinessObjects have many properties that I would rather see in EditorialObjects. I doubt the need for the class BusinessObjects in general. What is the benefit? We don't need it as super-class for a Group class, because grouping should be represented by "hasMember"-relations (see above). Do we need it for the BibliograpicalObject? Are we really using this in the audio-visual media industry?

This issue seems to be resolved through #176.


Permit polymorphic association as scope

Currently acts_as_list behaves un-intuitively with polymorphic associations, leading to confusing and hard-to-trace logical errors when user attempts to use polymorphic association as scope in intuitive manner, like this:
class Address
  belongs_to :addressable, polymorphic: true
  acts_as_list scope: :addressable
end

A workaround was described here http://jakeboxer.com/blog/2011/10/09/using-acts-as-list-in-a-polymorphic-scope/, and recommends using a construction like:
class Address
  belongs_to :addressable, polymorphic: true
  acts_as_list scope: [:addressable_id, :addressable_type]
end

This commit allows the user to use the first, more intuitive, construction to avoid subtle logic errors that show up later.
Please let me know if I'm contributing wrongly.  Thanks!  - Luke

I just realized I followed exactly none of the recommendations for contributing.  I am closing this pull request until I can meet the contribution guidelines.  Sorry about that.

That's an interesting idea. I personally like the second approach of scoping both fields. It's more explicit.
What does everyone else think?


Overwrite o/p file if it exists

Fixes https://github.com/geohacker/geojson-stream-merge/issues/9
Currently if the O/P file already exists, geojson-stream-merge appends to it, instead of deleting the file and writing into a fresh file. Fix this.
@geohacker let me know what you think, will wait for a review! : )
cc/ @amishas157

@arunasank this is great! thank you, and sorry for the delay here.


npm run start fails due to case sensitivity

I guess that you run this on mac, since by default HFS+ is case insensitive.
Since I'm running this on debian, ./scripts/start.js and ./scripts/Start.js are not the same thing.

Same for dist/index.js, which run expects to be dist/Index.js.

Yes you're right I've only tested running locally on my Mac.  Unfortunately I don't have immediate access to a Linux machine so I'll just make the changes as and when you spot them.  I'll fix these couple up tonight.

No hurries, no worries, those were the only two cases I encountered, I won't PR since I don't know if you'd prefer to rename the file or change the reference.

Should be fixed in https://github.com/channl/dynamodb-lambda-autoscale/commit/4860600411cf273c7380407b4661beb7610dcfcf
Will reopen if you still have problems.


NavigationProcessor should wait for NavBackStackEntry update before completing processAction

Purpose:

This will remove potential race conditions
It will make the NavHostController the single source of truth about the current destination
It will make it possible to deliver whole back stack in current destination information, which should make advanced back stack processing and NavAction verification possible.

Concerns:

What if the user presses back soft key while nav action processing is in progress? This will change back stack as well.

Needed for advanced NavIntent processing.


New Ember Yarn project cannot use Bootstrap 4

Thank you for taking the time to open an issue!
For general questions please use the #e-bootstrap channel on the Ember
community Slack.
For bug reports please include the following section in your issue details.
If you include instructions on how to reproduce the bug or a failing test case
it will make it easier for us to track down the issue you're having.

Output from ember bootstrap:info (available since 1.0.0-alpha.9):
Npm packages:
ember-bootstrap: ^1.0.0-rc.5 -> 1.0.0-rc.5
ember-cli: ~2.17.0 -> 2.17.0
bootstrap: n/a
bootstrap-sass: n/a
ember-cli-sass: ^7.1.2 -> 7.1.2
ember-cli-less: n/a
Bower packages:
bootstrap: n/a
bootstrap-sass: n/a
ember-bootstrap configuration:
bootstrapVersion: 3
importBootstrapFont: true
importBootstrapCSS: false

I am trying ti switch to to use bootstrap 4. Getting the following error
Error: ENOENT: no such file or directory, copyfile '/home/jcostanzo/.cache/yarn/v1/npm-bootstrap-sass-3.3.7-6596c7ab40f6637393323ab0bc80d064fc630498/assets/javascripts/bootstrap/affix.js' -> '/home/jcostanzo/Development/menu-manager/node_modules/bootstrap-sass/assets/javascripts/bootstrap/affix.js'
Steps to produce:

ember install ember-bootstrap
ember generate ember-bootstrap --bootstrap-version=4

Downgraded from rc5 to rc4 and it worked. So it seems to be something with rc5
Thoughts?

As written in Slack, have you tried cleaning up your node_modules and the yarn cache? I cannot think of any recent changes that could have caused this, sound more like a local error...


Makes body parameter type explicit

Feign has MethodMetadata.bodyType(), but never passed it to encoders.
Encoders that register type adapters need to do so based on the
interface desired as opposed to the implementation class. This change
breaks api compatibility for < 8.x, by requiring an additional arg
on Encoder.encode.
see https://github.com/square/retrofit/issues/713

NetflixOSS » feign » feign-pull-requests #39 SUCCESS
This pull request looks good


Tech debt: Remove use-metamask

use-metamask is obsolete since we use wagmi everywhere. Also the feature toggle can now be deleted.

See what it is connected still in masa-react @hide-on-bush-x @simodrws


Ingestion task not submitting when using s3-statsic firehose

When trying to submit an ingestion task using s3 firehose, getting error -
{"error":"Instantiation of [simple type, class org.apache.druid.indexing.common.task.IndexTask] value failed: null"}
Affected Version
Error encountered in version 0.14.2
Description
The ingestion spec for the task am submitting is as follows :
{
"type": "index",
"spec": {
"dataSchema": {
"dataSource": "reco_collab_data",
"parser": {
"type": "string",
"parseSpec": {
"format": "json",
"timestampSpec": {
"format": "yyyy-MM-dd'T'HH:mm:ss.SSS",
"column": "eventTime"
},
"dimensionsSpec": {
"dimensions": [
{
"type": "LONG",
"name": "product_id"
},
"experiment",
"variation",
{
"type": "LONG",
"name": "recoProductListSize"
}
]
}
}
},
"metricsSpec": [
{
"type": "count",
"name": "_count"
}
],
"granularitySpec": {
"segmentGranularity": "DAY",
"queryGranularity": "hour",
"rollup": true
}
},
"tuningConfig": {
"type": "index",
"maxRowsInMemory": "100000",
"maxRowsPerSegment": "5000000"
},
"ioconfig": {
"type": "index",
"firehose": {
"type": "static-s3",
"uris": [
"s3://hs-data-engineering/druidtest/testfile.log"
]
}
}
}
}
If I replace task type index with index_parallel I get error with ParallelIndexTask instead, but the same error occurs. Can someone tell if anything is wrong in my ingestion spec or something other missing ? For the record this is a production setup with 32 realtime tasks running always and 20-30 other batch tasks running throughout the day, so I don't think it's an issue with the cluster though could always have overlooked something.

Unfortunately that error message isn't very useful, but maybe it's this part of the spec:
"ioconfig": {

It needs to be ioConfig
Do you see any stack traces in the overlord log when this happens?


[QUESTION] Clarifying DeepSpeed-Inference

Some questions about DeepSpeed-Inference:

In the documentation for DeepSpeed-Inference (e.g. here), it is claimed that the functionality works specifically for large transformer models. Obviously, certain components, like the optimized kernels, only apply if your model is a standard HuggingFace-style transformer. Are the other features (e.g. inference-time model parallelism) more general? Can I use them on any PyTorch model, and without a HuggingFace pipeline?

Am I correct to assume that DeepSpeed-Inference model parallelism would not improve the memory performance of a memory-intensive model whose parameters account for only a small portion of its memory consumption (e.g. a relatively small model (~100M parameters) with extremely large intermediate activations)?

Assuming that the answer to (1) is yes, I've experimented a little with simple MLP's. DeepSpeed-Inference appears to run a full copy of the model in parallel on each GPU (rather than splitting it up between them, as I'd expect). Here's the code I used for those experiments:

import os

import torch
import deepspeed

torch.manual_seed(42)

local_rank = int(os.getenv('LOCAL_RANK', '0'))
world_size = int(os.getenv('WORLD_SIZE', '1'))

NUM_LAYERS = 9000
LAYER_DIM = 500

class Model(torch.nn.Module):
    def __init__(self):
        super().__init__()

        self.ml = torch.nn.ModuleList()
        for _ in range(NUM_LAYERS):
            self.ml.append(torch.nn.Linear(LAYER_DIM, LAYER_DIM))

    def forward(self, x):
        for l in self.ml:
            x = l(x)

        return x

model = Model().eval()

ds_engine = deepspeed.init_inference(
    model,
    mp_size=world_size,
    checkpoint=None,
    replace_method=None,
)
model = ds_engine.module

with torch.no_grad():
    example = torch.rand((500, 500)).to(f"cuda:{local_rank}")
    out = model(example)

if not torch.distributed.is_initialized() or torch.distributed.get_rank() == 0:
    print(out)

When I ran this successively with deepspeed --num_gpus 2 experiment.py, increasing NUM_LAYERS until I ran out of memory, I ran into the same upper bound as when I ran the same experiment on a single GPU using vanilla PyTorch, even though I'd expect the DeepSpeed-Inference version to be able to handle values approximately twice as large. Am I doing something wrong?

Hi @gahdritz
Thanks for using DeepSpeed-Inference, and raising these interesting questions.
I try to answer your questions below:

Yes, we have tried to show the inference module in DeepSpeed mostly on HF models. Except the kernels, which requires some modification for supporting different transformer-type models, the rest can be used with different PyTorch models. On the other hand, you need to provide an injection-policy when using the model-parallelism as what you need to use in your specific example. That being said, I am preparing a tutorial on how to specify this so that you can understand what this policy exactly means.
I am not sure if I completely understand your question. But, if you mean that you use a very large batch size, then you can use parallelism across batch (which is better than model-parallelism since you have smaller model). Or if the sequence-length is large, then yes you are right that model-parallelism cannot help you since the attention computation remains the same.
The main reason you did not see any improvement with DeepSpeed is that you require to pass in a policy, so that we can inject parallelism properly (like in this example). For this, you need to modify your example a bit:

class mlp(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.m1 = torch.nn.Linear(LAYER_DIM, LAYER_DIM)
        self.m2 = torch.nn.Linear(LAYER_DIM, LAYER_DIM)

    def forward(self, x):
            x = self.m2(self.m1(x))

class Model(torch.nn.Module):
    def __init__(self):
        super().__init__()

        self.ml = torch.nn.ModuleList()
        for _ in range(NUM_LAYERS//2):
            self.ml.append(mlp())
        if (2 * (NUM_LAYERS//2)) < NUM_LAYERS:
            self.ml.append(torch.nn.Linear(LAYER_DIM, LAYER_DIM))

    def forward(self, x):
        for l in self.ml:
            x = l(x)

        return x

model = Model().eval()

ds_engine = deepspeed.init_inference(
    model,
    mp_size=world_size,
    checkpoint=None,
    replace_method=None,
    injection_policy={ml: ('mlp.m2')},
)
model = ds_engine.module

Thanks! This answers all of my questions. I look forward to that policy guide.


Remove playdoh and add peep.

Please add python-memcached in requirements.

@johngian updated.

It looks like there are some missing dependencies in prod.txt:
Installing collected packages: ipaddress, fake-factory, rcssmin, rjsmin
Successfully installed fake-factory-0.5.7 ipaddress-1.0.16 rcssmin-1.0.6 rjsmin-1.0.12

Same for prod (with some overlaps because dev inherits prod) :
Successfully built rcssmin rjsmin tornado PyYAML argh watchdog pathtools simplegeneric pexpect backports.ssl-match-hostname
Installing collected packages: rcssmin, rjsmin, babel, snowballstemmer, imagesize, alabaster, backports.ssl-match-hostname, singledispatch, certifi, backports-abc, tornado, PyYAML, argh, pathtools, watchdog, livereload, mccabe, pyflakes, decorator, ipython-genutils, traitlets, pathlib2, pickleshare, simplegeneric, ptyprocess, pexpect
Successfully installed PyYAML-3.11 alabaster-0.7.7 argh-0.26.1 babel-2.3.4 backports-abc-0.4 backports.ssl-match-hostname-<IP_ADDRESS> certifi-2016.2.28 decorator-4.0.9 imagesize-0.7.1 ipython-genutils-0.1.0 livereload-2.4.1 mccabe-0.4.0 pathlib2-2.1.0 pathtools-0.1.2 pexpect-4.0.1 pickleshare-0.7.2 ptyprocess-0.5.1 pyflakes-1.0.0 rcssmin-1.0.6 rjsmin-1.0.12 simplegeneric-0.8.1 singledispatch-<IP_ADDRESS> snowballstemmer-1.2.1 tornado-4.3 traitlets-4.2.1 watchdog-0.8.3

r+wc

thanks!


Coral-Hive: Add UDF name in the log of ArtifactsResolver

Sometimes, the translation on some views takes very long due to the issues like:
	impossible to acquire lock for com.fasterxml.jackson.core#jackson-annotations;2.7.8

22/06/05 10:23:49 WARN functions.ArtifactsResolver: Unable to fetch dependencies: [unresolved dependency: com.fasterxml.jackson.core#jackson-annotations;2.7.8: configuration not found in com.fasterxml.jackson.core#jackson-annotations;2.7.8: 'master'. It was required from com.fasterxml.jackson.core#jackson-databind;2.7.8 compile]

This PR adds the UDF name to the log for debugging. We may register the affected UDFs to StaticHiveFunctionRegistry for such an issue.

Thanks for the PR @ljfgem !
Could you enhance the description to include a testing section to verify that this change is logging the UDF name properly?

Thanks for the PR @ljfgem ! Could you enhance the description to include a testing section to verify that this change is logging the UDF name properly?

Thanks for review, didn't include it because this issue doesn't happen consistently, but I am sure it could help to log the UDF name properly.

Sure, sounds good. This PR is enhancing logs and should not cause issues.
LGTM.


Update the binder_context, when there is function expression in the projection's target_list

Search before asking

[X] I have searched the EvaDB issues and found no similar feature requests.

Description
Right now, the binder_context is not getting updated for the function expression in the projection's target_list. So the following query will not work:
SELECT HomeSaleForecast(3) ORDER BY price;

CREATE TABLE AS SELECT HomeSaleForecast(3) FROM postgres.home_sale;

Use case
No response
Are you willing to submit a PR?

[ ] Yes I'd like to help by submitting a PR!

This will resolves the #1119

5 long integration test failed. circle ci
FAILED test/integration_tests/long/test_select_executor.py::SelectExecutorTest::test_sort_on_nonprojected_column - evadb.executor.executor_utils.ExecutorError: ['myvideo.id']
FAILED test/integration_tests/long/test_similarity.py::SimilarityTests::test_end_to_end_index_scan_should_work_correctly_on_image_dataset - evadb.executor.executor_utils.ExecutorError: ['testsimilarityimagedataset.data']
FAILED test/integration_tests/long/test_similarity.py::SimilarityTests::test_should_do_vector_index_scan - AssertionError: False is not true
FAILED test/integration_tests/long/test_single_document_similarity.py::SingleDocumentSimilarityTests::test_single_pdf_should_work - AssertionError: False is not true
FAILED test/integration_tests/long/interfaces/relational/test_relational_api.py::RelationalAPI::test_create_index - evadb.executor.executor_utils.ExecutorError: ['meme_images.data']

@gaurav274 A gentle ping.

Investigating in #1147


Lifecycle events are not firing of child template in tabs

Short description of the problem:
Lifecycle events are not firing of child template in tabs. I created a hello template and added in one of the tabs root page. Lifecycle events of root page are firing but not of child template.
What behavior are you expecting?
Lifecycle events of child template should fire
Steps to reproduce:
I created new ionic2-app-base project and added this minimal commit https://github.com/aggarwalankush/ionic2-issues/commit/a9489e2026f165230cb3c9eb81f0b5980c279c06

Which Ionic Version? 1.x or 2.x
Ionic 2 rc3
Run ionic info from terminal/cmd prompt: (paste output below)
Your system information:
Cordova CLI: 6.3.1
Ionic Framework Version: 2.0.0-rc.3
Ionic CLI Version: 2.1.8
Ionic App Lib Version: 2.1.4
Ionic App Scripts Version: 0.0.45
ios-deploy version: 1.9.0
ios-sim version: 5.0.8
OS: OS X El Capitan
Node Version: v6.7.0
Xcode version: Xcode 8.1 Build version 8T61a

hello is not a page, so it does not receive page lifecycle events. That's the expected behaviour.
You could inject ViewController, and subscribe to lifecycle observers:
http://ionicframework.com/docs/v2/api/navigation/ViewController/#instance-members

@manucorporat Agree, but instead of having a child template in tab root page, if I push a page having child template, then lifecycle events of child template also executes.
this.navCtrl.push(PageHavingHelloChildTemplate) will fire hello template lifecycle. if hello is not a page, then this also should not happen.

I am not sure if I understand, sorry
this.navCtrl.push(PageHavingHelloChildTemplate)

^in this case, you are pushing another angular 2 component as an ionic page so It will receive the lifecycle.


Make viewable subannotations "stick"

currently once you start looking at VOTs in the detail viewer, you have to reselect them each time you go to the next phone. It would be nice to be able to keep it activated at all times (unless deactivated).

I think the default should be: subannotations on and the VOT selected, so you can immediately see the blue selection across both the wave and the spectrogram. Right now it's a bit difficult to see what part of the spectrogram has been selected by autoVOT.


Emojis aren't rendered correctly in Algolia results

For example, do a search on "Our packages have been downloaded one million times"

Inspecting the search results, it seems like the actual import of posts haven't properly passed emojis, or maybe Algolia doesn't support emojis at all?

I'm playing about with your blog so I did a quick test. It seems that Algolia does support emojis. As can been seen in the screen shot.

I tried making a new database with a non-unicode character set to see if there was a problem with character storage and translation but could not replicate the issue. It seems as though the laravel database.php config takes care of that with:
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',

which sets the correct character set for the tables during migration even if the database was created with a different character set.
I commented out the lines in database.php and created a new non-unicode database. I ran the migrations and database seed. I then tried to add an emoji and the database threw an error.
Finally, the response from murze.be. We can see that Algolia does not return the Emoji.

Without access to your index on Algolia it seems as if somehow the initial import of the data did not parse the emojis correctly even though they are clearly stored correctly in your database as they can be seen in the articles.
I hope this helps 🤷♂

I've cleared out the algolia index and rebuilt it with php artisan scout:import "App\Models\Post"
Unfortunately the problem persist, will investigate this further soon.


server side delegations support in validation and snapshot generation

Signed-off-by: David Lawrence<EMAIL_ADDRESS>(github: endophage)

@diogomonica @cyli @riyazdf no longer WIP, PTAL.

Could we add another test that updates both "targets" and "targets/level1", but the update to targets deletes "targets/level1"?  In which case I think there should only be 1 update?

Other than the above, LGTM!  :D

LGTM


Run demo.py in centos, torch1.4.0-cpu, memory keep increasing and then overflow

I have trained it on my GPU, now I run it on the server with centOS pytorch1.4.0-cpu. After each image was detected, the memory don’t decrease and after several rounds  the memory overflow.This never happen on PC Windows. Is there anyone meet the same problem like me?

If the imager’s sizes are the same, memory will not increase. However, if their sizes are different, memory will increase.

在ubuntu上测试也是如此


fix: Updating the primary action on clicking app card in mobile view

Description
Updating the primary action for app card click on mobile.
Fixes #32084
Automation
/ok-to-test<EMAIL_ADDRESS>:mag: Cypress test results

[!TIP]
Tests running at: https://github.com/appsmithorg/appsmith/actions/runs/8448397082
Commit: 2424f0f6a1f891f0ff74162ebdb718e6a433a411
Workflow: PR Automation test suite
Tags: ``

Summary by CodeRabbit

New Features

Enhanced the application card to support launching the mobile app directly with specific URL parameters for a more tailored user experience.

/build-deploy-preview skip-tests=true

Tested on the above DP - clicking on the app card launches the app


Could it support GPU Acceleration?

It would be more efficiency with GPU Acceleration. Is it possible to support GPU Acceleration?

It's possible only if you build (compile) Tesseract (text recognition library) with OpenCL (GPU Acceleration library) support, but:

Important note: OpenCL support in Tesseract is still considered experimental. There are some major bugs associated with it (see issue 837). Building Tesseract with OpenCL support is not recommended (for any version of Tesseract), unless you are a developer and want to improve the OpenCL code.

So, if you want to try, go here for the instructions.

I've created a fork of this library that uses the newer PaddleOCR library which does support GPU: https://github.com/oliverfei/videocr-PaddleOCR


Get the bare name of a raw Ident

Currently, all to string conversions on Ident return the ident verbatim, e.g. r#move as r#move. It would be very useful to have a way to get the bare name, e.g. move.
This would be useful e.g. for creating wrappers in pyo3:
fn function_wrapper_ident(name: &syn::Ident) -> syn::Ident {
    syn::Ident::new(&format!("__pyo3_get_function_{}", name), Span::call_site())
}

This version errors when given r#move with "__pyo3_get_function_r#move" is not a valid identifier. It's possible to work around this by replacing r# with an empty string, but I'd prefer it if syn provided a method for that.

It seems that using mashup! { m["method"] = __pyo3_get_function_ $function_name; } in mashup also doesn't work with raw identifiers, but I couldn't get a proper macro expansion from rustc to confirm this.

Ah yeah we hit this in serde_derive as well and ended up doing:
let suffix = ident.to_string().trim_left_matches("r#").to_owned();
let dummy_const = Ident::new(&format!("_IMPL_SERIALIZE_FOR_{}", suffix), Span::call_site());

The syn::Ident type is just copied from the compiler's proc_macro::Ident so this would need to be implemented here first: https://github.com/rust-lang/rust/blob/master/src/libproc_macro/lib.rs before we would pick it up in Syn.


Adding Tests

I was thinking if we can add tests to the python library for Imgur API. To do this though, there are many options to consider. Since this an web API, results retrieved from requests will not always be the same. Some options to overcome this:

Creating a test database from which to test the API calls. Imgur may not need to provide this
Mocking up requests on which to run the API calls against? I know there is a mock library to Python that is used for that. I have never used it but if this a way to go I would like to give it a try
Create test albums/images etc. using the POST requests, test them and check if result is as expected. Then test GET/DELETE request functions on them and check if result is as required. My concern with this is that not all functions will be tested
Let me know what you think :)

@ueg1990 2) seems like the best route :+1:

PRAW uses betamax for its integration tests of different requests.


Use Weaver.js

Is it possible to use Weaver with the dagre layout? I don't see it as an option in the code, and I would like to be able to offload the work into a thread to not block the main thread.

Not possible. Can't control dagre internals

On Aug 28, 2017, at 15:41, Josh Hardy<EMAIL_ADDRESS>wrote:
Is it possible to use Weaver with the dagre layout? I don't see it as an option in the code, and I would like to be able to offload the work into a thread to not block the main thread.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Is there any way to not block the main thread then? Allow for some sort of progress bar?

Not without real threads


Nest default sort order

NEST/Elasticsearch.Net version: 5.2.0
Elasticsearch version: 5.2.0
Description of the problem including expected versus actual behavior:
Does NEST define a default sort order?
I'm sending the same request and after certain number of requests I get a randomized order. The documents have the same _score = 1 and i didn't specify any sort. This got me worried as i'd like perform to paging on my results.
Steps to reproduce:
I had a simple query for test
new BoolQuery {
     Must = new QueryContainer[] { new MatchAllQuery()
}

if I run the search for consecutive number of times, I get randomized order.

The default sort order for Elasticsearch is _score descending which means that documents with equal score may appear in different order on subsequent requests. You can add additional sorting with the client
client.Search<MyDocument>(s => s
    .Query(q => q
        .MatchAll()
    )
    .Sort(so => so
        .Descending("_score")
        .Ascending("name")
    )
);


ph4 run slows down significantly at increased times?

Hi,
I'm trying to run some simulations with ph4 on a cluster of 2000 stars, with a timestep of 0.2 (bridged with a static MW potential imported from galpy). It seems that the ph4 code is slowly down significantly as the n-body time of simulation increases: I had noticed this a little in some initial runs, but after some print statements to debug this seems very clear. When t is around 1 nbody time, the ph4 code seems to be progress 0.2 nbody times roughly every 3 seconds, yet when it gets to around 20 nbody times, there now is about 20-30 seconds between each step of 0.2 nbody times.
Does anyone have any idea why this might be happening? I'm not sure if there's some issue with my code — I'm hoping that's the case, rather than this being an actual problem with the ph4 code ... Please let me know!

ph4 uses a dynamic timestep. As you get a denser star cluster, it will take shorter internal time steps and therefore run slower. This is especially true if you form any binaries.

@rieder is 100% correct here. It is likely that unless you have something to deal with dynamical binaries (like the Multiples interface) that you will continue to see this ramp up through the mass segregation phase. There are a few examples on how to Bridge Multiples+PH4 with GalPy or the like in the textbook towards the back. If you need extra help, hit me up directly at<EMAIL_ADDRESS>This is my bread-and-butter.
~ Joe G.

You could also try to use the new petar gravity code (Wang et al. 2020), which uses algorithmic regularisation to deal with binaries. The AMUSE interface for it is available now.

Ok, thanks so much for the help; that explains a lot. We are not particularly focused on the core/densest region of the GC, insofar as a timestep that's too large doesn't allow the core region to affect the outer region that is being stripped off (we're more focused on streams formations). As such, is there a way (and does it make sense?) to set a larger timestep by hand to speed up our runs?
Thanks for pointing out the petar code. Is its precision / energy error good compared to a code like ph4? All we're really trying to do right now is evolve a GC (N=2000 right now, but higher N would be better!) in the MW potential to see if and how streams begin to form — would you recommend petar for this?
Thank you both so much!

Yes, I would certainly recommend petar for that.

you should be perfectly able to use ph4 for that. but without the code it
is hard to guess what you are doing...
On Fri, Oct 9, 2020, 22:06 Steven Rieder<EMAIL_ADDRESS>wrote:

Yes, I would certainly recommend petar for that.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/amusecode/amuse/issues/685#issuecomment-706379267,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABCPFTDEOCF3LM6EAGIMB5DSJ5ULZANCNFSM4SJUTE3A
.

Hello,
I've been starting to test things out with petar recently again, and I was wondering if someone could explain how petar timestepping works. I'm curious because I see that the timestep parameter is read-only, and while evolving a N=20000 GC for a short time (upto 1 n-body time, with snapshots at times like 0.05, 0.1, 0.3 n-body times, etc), the GC with a concentration parameter W of 5 ran a lot faster than one with a concentration parameter of 10.
Is it some sort of dynamic timestep based on GC density? Could I get a bit more clarification on what factors are taken into account when computing the timestep (and whether there is anything the user (me) needs to fine-tune—perhaps dt_soft, the "tree time step")?
Thank you for your help!


Exceeding memcached max key length with httplib2.Http()

The Gmail API recommends enabling cacheing with httplib2.Http() for better performance. The problem I'm running into is that the Google API Python client seems to use the entire API query as the key, which means that long queries fail. E.g. if you are searching Gmail with a list of 10 - 20 different keywords then the memcached max key length will be exceeded and the query will fail.
This might be by design, although in the case of the GMail API only the discovery query is cacheable anyway, so it seems like the full search queries shouldn't be necessary for that to work.

Can you share a small sample to reproduce this? Is the client library just generating a query string that's too long for httplib2 to cache?

Correct. The key length limit for memcached is 250 characters, so the querystring the client library generates is just too long for that. If all the info in the querystring is needed then maybe the hash of the querystring could be used as the key. The line that's failing is:
threads = gmail_client.users().threads().list(userId='me', maxResults=15, pageToken=pageToken, q=query).execute()
Where the 'query' variable is a string that's maybe 100 characters long.

Are there any drawbacks to just disabling caching (other than the obvious)?

Nope, cacheing is disabled for now and it works fine.

Okay, so it seems like the bug itself is on httplib2. Does anything need to be done in this library?

Ahh ok, I guess not then. Thanks!

Feel free to file this bug (and your proposed solution) over at https://github.com/jcgregorio/httplib2


Add unit testing for client util

Feature request:
Refer to https://github.com/dragonflyoss/client/tree/main/dragonfly-client-util.

Use case:

UI Example:

https://github.com/dragonflyoss/client/pull/826
https://github.com/dragonflyoss/client/pull/827
done!


CsvSource parser configuration

While reading CsvSource files, I get Array out of Bounds error on rows with data longer than the default column width set to 4096:
com.univocity.parsers.common.TextParsingException: Length of parsed input (4097) exceeds the maximum number of characters defined in your parser settings (4096).
Hint: Number of characters processed may have exceeded limit of 4096 characters per column. Use settings.setMaxCharsPerColumn(int) to define the maximum number of characters a column can have

How can the csv parser configuration be changed to allow for longer column widths?

I can add that option to the CSV Source.

Fixed in master. Which version are you using?

Thanks for doing this! I just updated my deps to latest master.

Let me know if there are issues/docs with help wanted; I am using eel-parquet inside an AWS Lambda container w/ Scala ofc ;) and it works so well!

I wrote eel initially for use at Barclays and then Deutsche Bank. So it's nice to hear of other users outside of those two banks.


Function Packager: files or folders beginning with a dot(.) do not add to latest-build.zip

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
14.17.3
Amplify CLI Version
5.3
What operating system are you using?
Mac
Amplify Categories
auth, storage, function, api, analytics
Amplify Commands
push
Describe the bug
When running amplify push, I see that any file or folder that begins with a dot (.) does not get copied to the functions dist/latest-build.zip.  My project utilizes dot folders, and a node module I use also contains a .config file that doesn't get added to the zip, resulting in Lambda Erroring when called.
Expected behavior
Any file in the src directory gets added to the latest-build.zip
Reproduction steps

Add a .test file to a function
Do amplify push
Open function's dist/latest-build.zip
See that the file is not there

GraphQL schema(s)

# Put schemas below this line

Log output

# Put your logs below this line

Additional information
No response

I also just started seeing this issue, which wasn't happening last week. I am running Amplify CLI version 5.2.1, node.js 16.6.1, on a Mac.

Hey @mgarabedian and @dismantl :wave: thanks for raising this! I was able to reproduce this on the latest version of the CLI and confirmed if I downgraded the CLI to an older version (pre 5.0.0, I used 4.52.0 in my reproduction) it was zipping the dotfiles. Marking this as a bug

No problem. Any idea when this will be fixed? I cannot deploy certain Lambda functions now, and am stuck here. Thanks


Add an "always" Ansible Job Run Mode to the Policy addon

The Ansible integration in the Policy addon only supports the modes once , manual, and disabled. The once mode is limiting since after a policy violation occurs, the Ansible job runs once and then it is disabled. One has to manually renable it for the Ansible job to run again on the next policy violation.
Ideally, there should be a new mode called something like always, which runs on every new policy violation. Consider a situation where a policy applies to two clusters. If the policy violation occurs on the first cluster, the Ansible automation would run. Then later on, if there is a policy violation that occurs on the second cluster, the Ansible automation would run again. The Ansible automation would never run again for either of these clusters until the policy violation is resolved and then violated again.

/assign mprahl I'll take on writing the design document.

Closing this issue as complete.


How to override LinkGenerator?

It is necessary to change the Click Handler to my own version

Just implement your own:
public class MyLinkGenerator : LinkGenerator
{
    ...
}

And then assign it:
view.Generators["a"] = new MyLinkGenerator();

also see: https://github.com/MyToolkit/MyToolkit/wiki/HtmlView


Add permit_weak_imports directive

Issue Homebrew/homebrew-core#3727 suggested we set -no_weak_imports for new versions of Xcode to ensure that e.g. building on 10.11 against the 10.12 SDK doesn't result in a situation where autotools thinks symbols exist that don't actually exist on the current platform.
Further discussion in golang/go#16770 revealed that some packages require weak imports to build normally.
This PR allows formulas to opt out of -no_weak_imports.

Reading https://github.com/golang/go/issues/16770#issuecomment-240581950 I'm not sure this is the right solution.
I think we should strip -mmacosx-version-min=10.6 from Go's CFLAGS because everything else we do is build around building bottles that only work on a specific version and it's that that's causing the conflict.
Alternatively, we only apply -no_weak_imports on Xcode 8 on OS X 10.11 (or, in general, only when using the SDK that doesn't match the OS version). @ilovezfs remind me: we have other issues that only affect that case, right?
CC @achivetta for thoughts on this.

@MikeMcQuaid Yeah, my SDKROOT bug currently affects
ansible
baresip
h2o
mitmproxy
passenger
rrdtool
uwsgi
vim
wolfssl

I'm not thrilled with this -no_weak_imports idea. It seems like we're anticipating a need to cater to build systems that may make bad assumptions about macOS. I'd rather see us actually hit those bugs and then report them upstream rather than trying to side-step a problem that may or may not exist, and is basically always going to be an upstream bug when it happens. The right place for formulae that would need this is most likely :skull: since it means their upstream is no longer actively supporting macOS.

@ilovezfs to be more explicit: does that bug also only affect e.g. using the OS X 10.X+1 SDK on OS X 10.X? If so, it feels like we do want to start special-casing that situation (which we previously decided to not do).

to be more explicit: does that bug also only affect e.g. using the OS X 10.X+1 SDK on OS X 10.X

Yes.

Yeh, I think we may want various options that apply only to that case, then, rather than fixing it on a formula-by-formula basis.

@MikeMcQuaid set depends_on :macos => :el_capitan for every formula, and when Sierra ships, change them to depends_on :macos => :sierra. Problem solved. :smiling_imp:

@tdsmith In that case I think it makes sense to just set this flag for 10.11 using the 10.12 SDK (i.e. Xcode 8) and have a general block/method for handling the other various quirks that need set in this situation (although this can be punted to another PR).

To be more explicit: I'm definitely 👎 on disabling this on a formula-by-formula basis and we can consider just disabling this flag on OS X 10.12 entirely for now if what I've described is too much work to do in the near future.

I think removing mmacosx-version-min would be a mistake in this case, presuming our reading of the upstream situation with Go is correct.
Whilst we don't build our Go-using formulae to run as far back as 10.6, I think it'd be the wrong thing to do to make anyone who wants to develop backwards-compatible executables with go outside of Homebrew reinstall go from an alternative source. That feels pretty hostile & likely in the end would result in us having to introduce a special :go dependency.

Just for expectation-setting, I don't have any immediate plans to work on a different approach; I just thought this sounded like a chance to play with ENV internals. :)
I'm not really compelled by the idea that build failures against future SDKs are upstream bugs but I that's probably a separate discussion vs. this PR.

I don't have a strong opinion on the right approach here; my goal was just to make sure you had the tools to solve the broken packages before macOS GM'ed.  But let me see if I can sort out the various cases.
There are three versions at play here:

The SDK version, which is determined by which Xcode is installed.
The OS version the user is running.
The deployment target set by the code, or the OS version if none.

OS == SDK == deployment is the common case of building random autoconf-based open source software on the latest macOS.  In this case, -no_weak_imports should be a noop.
OS == deployment < SDK is the case where you are on OS X 10.11 with Xcode 8.  When building software that wasn't written with macOS-style weak linking and a specific deployment target in mind (likely including almost everything that uses autoconf), this is what -no_weak_imports will fix (or at least move from a runtime failure to a build time failure).
deployment < OS == SDK and deployment < OS < SDK are situations where the developer almost certainly wanted the old weak-linking behavior.  Go falls into this bucket and we shouldn't add -no_weak_imports here.
Okay, so a few thoughts on how to proceed:

If there's a way to detect when the project is passing -mmacos-version-min= and avoid adding -no_weak_imports, that's almost certainly a good thing to do.  If not, you'll need an opt-out for such projects.
Only applying the flag when OS < SDK will likely result in fewer broken builds for users, but also hides issues that only occur when in that situation.  e.g. if that was the current behavior, we wouldn't know that Go was broken in that situation.

Hope that helps!  (Also, CC @jeremyhu in case he has other thoughts.)

If there's a way to detect when the project is passing -mmacos-version-min= and avoid adding -no_weak_imports, that's almost certainly a good thing to do. If not, you'll need an opt-out for such projects.

@achivetta Thanks, that's really helpful. There is a way to do this but only on a single compiler invocation e.g. if you pass clang -mmacos-version-min=10.6 -Wl,no_weak_imports we can filter out the -Wl,no_weak_imports but if e.g. clang -mmacos-version-min=10.6 was used by one call and clang -Wl,no_weak_imports by another we would not be able to filter that out as we don't track state between compiler invocations. Do you think it would be sufficient to filter just within a single invocation?

aside: -mmacos-version-min can also be specified as the MACOSX_DEPLOYMENT_TARGET environment variable (see man clang) but I don't think that alone changes the feasibility of the superenv approach

@MikeMcQuaid The problem with a per-compiler-invocation approach is that you need the -no_weak_imports flag at link time, but the deployment target takes effect at compile time.  I'm not sure how it would work out for e.g. Go which I suspect does have different deployment targets per compilation unit.

Ideally and if we could auto-detect if a formula properly supports compiling against a macOS SDK and can properly deal with the situation that OS/deployment version < SDK version, we could consider turning off some of the adjustments made by our superenv. But since this is unlikely to be possible in practice, I think this PR solves this nicely on a formula-by-formula basis for those (relatively rare) cases where some of our adjustments don't really make sense.

Thanks everyone. Think it's worth 🚢ing this as-is for now and we'll rethink later if we need to start whitelisting too many formulae.


Split meshes can also be an optimization

Nice tool you got there, but for my purposes I need something that can split a gltf/glb mesh Into multiple meshes (with max_Vertices_Count and max_Faces_Count), as the desired game engine only supports maximum indices of 32.767.
Would it be possible to add such a feature or is it too complicated?
My current workaround is loading the model in Blender and reducing complexity :/

Mesh splitting isn't too complicated so it could be implemented I think. Are you sure the limit is 32767 vertices as opposed to 65535?

I'm sure, the rever engineered Data says Face Indices are signed shorts :/ when I go above the value the model gets out of hand.

I would probably look into https://github.com/donmccurdy/glTF-Transform for this kind of involved processing - I don't know OTOH if it supports mesh splitting, but it tends to be better when you want to execute a series of specific processing commands, whereas gltfpack tends to be better at processing scenes optimally with little configuration.

Thanks for the hint I'll check it out!

gltfpack doesn't do coordinate system transforms; similarly to above, this seems to be a task better suited for glTF-Transform (unsure if it already supports it or not).

Thank for all 👌

Looking at this again, this is probably specific enough that glTF-Transform would be a better fit.


Change mouse pointer when mousing over expand sub-grid icon

Hi
 I've got a really nice grid working with a sub-grid.  Currently the only issue I have is that the mouse pointer doesn't change to a finger when mousing over the expand ('+') or un-expand ('-') icon for the subgrid.  How can I do the the 'angular' way - I'm currently trying to accomplish it with jQuery with no success yet.

Thanks
Dave

Figured out a css only solution ...
div.ui-grid-cell-contents i.ui-grid-icon-plus-squared {
cursor: pointer;
}


chore(java): remove java since maven is a hassle

Fixes #

Codecov Report

Merging #14 (28c9b87) into main (0eadcfd) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 28c9b87 differs from pull request most recent head b2a8655. Consider uploading reports for the commit b2a8655 to get more accurate results

@@            Coverage Diff            @@
##              main       #14   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            2         2
  Lines           54        54
  Branches        12        12
=========================================
  Hits            54        54

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Issuing credits to mashbean

Issue 50000 credits to @mashbean

 Follow up
On Tuesday, 08-Oct-24 14:09:12 UTC by Gov4Git dev
Issued 50000 credits to member @mashbean.


Add scCATCH

Here is another cell type identification and classification method (scCATCH). It is a cluster-based automatic annotation R package on cell types of clusters from single-cell RNA sequencing data. Please add it to your list, thanks!

Thanks!


[SW] Angular 5: Service Worker Uncaught (in promise) DOMException: Quota exceeded.

Prerequiste: test on chrome last version and chromium last version
I simply install the service worker package and deploy the default ngsw-config.json just add a API call and register the cache strategy inside the ngsw-config.json
"dataGroups": [{
      "name": "api-server",
      "urls": [
        "https://dog.ceo/api/**"
      ],
      "cacheConfig": {
        "strategy": "freshness",
        "maxSize": 100,
        "maxAge": "3d",
        "timeout": "10s"
      }
    }]

An I have the following error:
Uncaught (in promise) DOMException: Quota exceeded.
So, it seems the ngsw-worker.js doesn't work very well
Have ever encountered this problem?
Thanks for all

Error message like Quota exceeded usually mean that you're trying to use a webservice too much. It means they have a quota for how many requests you can do per a period of time (a quota) and you made more requests than that.
It's not something that we can address in the Angular CLI repo since it's completely unrelated to us. You should contact the owner of that service if you need a bigger quota.

This is not related to a remote service.
In my case this happens with the standard ngsw-config on Chrome 63 / Ubuntu.
ng new ServiceWorker1 --service-worker
cd ServiceWorker1
ng build --prod

Everything DOES work in firefox and it also works when I switch to Chrome's incognito mode.


Remove inspector_scan.ex

https://github.com/aws-beam/aws-codegen/pull/101 will remove this file anyway tonight but since I'm on it and would like to merge https://github.com/aws-beam/aws-elixir/pull/170 I'll remove it manually for now to avoid having to merge https://github.com/aws-beam/aws-elixir/pull/170 tomorrow instead 👌


No git tags for each version

Looking at setup.py the current version is 2.0.2, and the history would let us track back and find specific version bumps - for example https://github.com/PacificBiosciences/pypeFLOW/commit/cefc4ac03418e60200ce68aff05c49d3e54cd36e#diff-2eeaed663bd0d25b7e608891384b7298 jumped from 1.1.0 to 2.0.0
However, without explicit tags, it hard to be sure that commit itself represented the 2.0.0 release, or if it was a slightly later commit.

Only one version tag exists right now,

v0.1.0 - https://github.com/PacificBiosciences/pypeFLOW/tree/v0.1.0

Missing entries:

v0.1.1 - https://github.com/PacificBiosciences/pypeFLOW/commit/ff635f4605baac5f66cb000e552ddb3dcf98d34a#diff-2eeaed663bd0d25b7e608891384b7298
v1.0.0 - https://github.com/PacificBiosciences/pypeFLOW/commit/79c77235eb10e7b36a98ada891e1e6ef454aed58#diff-2eeaed663bd0d25b7e608891384b7298
v1.1.0 - https://github.com/PacificBiosciences/pypeFLOW/commit/67d1b7216116c782e9935912dc20764fd3174bfa#diff-2eeaed663bd0d25b7e608891384b7298
v2.0.0 - https://github.com/PacificBiosciences/pypeFLOW/commit/cefc4ac03418e60200ce68aff05c49d3e54cd36e#diff-2eeaed663bd0d25b7e608891384b7298
v2.0.1 - https://github.com/PacificBiosciences/pypeFLOW/commit/78c975bf53263797da4c6edeec913713a1367b73#diff-2eeaed663bd0d25b7e608891384b7298
v2.0.2 - https://github.com/PacificBiosciences/pypeFLOW/commit/374533b1dc9eee960b18c88f9395b3f3af5fef1e#diff-2eeaed663bd0d25b7e608891384b7298

I don't know who set up the pypeflow Bioconda recipe. We plan to deploy all of Falcon to Bioconda fairly soon, but we are not supporting what's there yet.

The vast majority of BioConda recipes are created and maintained without help from the package authors, although your input is of course welcome. BioConda already has Falcon, all be it an older version. Part of the trouble here is that the PacBio repositories have not taken full advantage of git in their release process, making it harder than it should be to pull out a specific release from the commit history.
However, this is wider issue than just BioConda packaging, it affect all packing efforts such as Linux distributions or homebrew. Also, this is an impediment to reproducible science as installing specific versions can be important for retracing past analyses.
Please consider re-opening this issue, and adding release tags to this git repository.


Removelife from death requirement from naxx

Closed by https://github.com/AzerothWarsLR/WarcraftLegacies/commit/581b5c39ae79f28e26ecede3430b9599796dec47.


Fix for null pointer dereference in jmem_heap_free_block

Fixes #2435.
JerryScript-DCO-1.0-Signed-off-by: Daniel Balla<EMAIL_ADDRESS>

Hi,
I'm the original poster of the issue you are fixing here.
While this change does prevent segfault, undefined behavior is still triggered at jerryscript/jerry-core/ecma/builtin-objects/typedarray/ecma-builtin-typedarray-prototype.c:598
This is because behavior of memcpy is undefined if the second argument is NULL.
This can be verified by using UBSan (undefined behavior sanitizer).
To do this, build jerryscript with
python tools/build.py --profile=es2015-subset --jerry-libc=off --lto=off --compile-flag="-fsanitize=undefined"
Running the example provided by me shows the following message:
jerryscript/jerry-core/ecma/builtin-objects/typedarray/ecma-builtin-typedarray-prototype.c:598:7: runtime error: null pointer passed as argument 2, which is declared to never be null /usr/include/x86_64-linux-gnu/bits/string3.h:53:10: runtime error: null pointer passed as argument 2, which is declared to never be null.

The implementation of filter (from jerryscript/jerry-core/ecma/builtin-objects/typedarray/ecma-builtin-typedarray-prototype.c) calls memcpy (in line 598) with the second argument (pass_value_list_p) being NULL.
Current implementation (without this commit) in JMEM_FINALIZE_LOCAL_ARRAY checks if var_name (pass_value_list_p in this case) is NULL. However, this check is optimized out, because of the call to memcpy. This is because gcc assumes, that that the second argument of memcpy is never NULL.
The default build for jerryscript enbales gcc optimisation -fdelete-null-pointer-checks, which removes the check from JMEM_FINALIZE_LOCAL_ARRAY  -- because of the call to memcpy.

I've added an early return to resolve this issue, and the test case as well.

This version no longer triggers undefined behavior.


Follow-up on Shared ref

Continuation of the previous PR. API change: default ctor is now state unitialized and static member make_ref is mandatory for build an initialized ref.
It is more clear and removes ambiguity within constructors.

wow :-)


Items get deleted when player gets absorbed (keepCon: false)

From what Foxyas and I seemed to uncover, this seems to be due to the method of killing the player after they get assimilated

I was reading through my server's log  while trying to replicate and it never stated that the player had died

I can get a dedicated sever to test this, just message me on discord when I am online

Tested, and resolved.


How to access and change selectColumns core method inside react-handsontable?

I want to know that, how to access selectColumns menthod inside react-handsontable. I already tried to assign refs to hot table component <HotTable refs="hot"/> but still i cant access selectColumns method.
console.log(this.refs.hot.hotInstance.selectColumns);

// output - returns undefined
Thanks for reading this question.

Hi @sagar-gavhane
here's a working demo https://codepen.io/anon/pen/zjQadN?editors=0010
lines 71-73 define the instance refference

Wow, Great example to access core methods.
Thanks a lot @AMBudnik

@Sagar Any other approaches? It's not so convenient if we include HotTable in other components and can't render it in such way.
If we speak about this rows from example
const exampleComponent = ReactDOM.render(
<ExampleComponent />, document.getElementById('example')
);
window.hotInstance = exampleComponent.refs.hot.hotInstance;

it's okay, when we render it as a separate component. But if I import it in App component and then render App component via ReactDom.render, I have no refs, just empty object


chat.langchain.com is not working

chat.langchain.com is not working, i am not getting a response from it. @hwchase17

@bizpruthviraj are you still seeing issues?


Sagemaker Batch Transform

Hi, I have few questions regarding Sagemaker Batch Transform Jobs.
Currently I have a deployed model (for object detection in an image) and an endpoint with which i am able to invoke and get inferences. Now, I am getting inferences for each image at a time.
I am wondering whether it is possible to get inferences for a batch of images at a time using Batch Transform job. If it is possible, can anyone please provide instructions of how to do this? I searched for some examples, but didnt find any relavant ones.
Thanks in advance!
Harathi

In general, I don't feel like bringing your own models to the batch transform mode is very well documented. I'd like to see a Tensorflow example using the script_mode python sdk (no docker included).

If you already have a model that works, you can simply use it in SageMaker Batch Transform by creating a transform job, passing in the input and output location in S3. When the job completes, it'll upload the inference results to the output location you specified.
SageMaker has a bunch of image-classification sample notebooks which could be helpful, e.g. https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-lst-format.ipynb

@ryan/@aloha Thanks for the suggestions. Sorry for the late intimation. I am able to create batch transform job successfully.

@Harathi123, If you can mention the steps to create a batch transform job that would be helpfull.
I not using AWS-Sagemakers inbuilt models. I have my own code as a Jupyter Notebook and trying to run that code for batch inferences.
In the AWS samples, I see a sample file for R deployment, nothing on how to deploy the Jupyter instance.
I believe need to create a docker image and then train and deploy.
Thanks in Advance.


Reapply "vsr: liveness, nack prepares from before log view"

Fixes the crash: https://github.com/tigerbeetle/tigerbeetle/issues/1529
There, a replica nacked op=0, the root op, which lead to the op being truncated.
The problem is that, when we compute nacks, in the case 3:
                // Case 3: We don't have a prepare at all, and that's not due to a fault.
                if (journal_header == null and !faulty) {
                    nacks.set(i);
                }

we didn't account for the possibility that the op was legitimately evicted from journal during repair.
This makes me think that there's one more reason to tighten up our repair_min: https://github.com/tigerbeetle/tigerbeetle/issues/1378:
The primary actually can't reason that checkpointed prepares are available on backups! In an idle culster, the checkpointed prepares might slowly get corrupted & cleared, and backups won't attempt to repair them. So, we could accumulate more than f faults.

https://github.com/tigerbeetle/tigerbeetle/issues/1545 is a legit issue with this PR. That should be solved by
https://github.com/tigerbeetle/tigerbeetle/pull/1552
Want to land that change separately, as it makes sense in isolation as well

Dropped https://github.com/tigerbeetle/tigerbeetle/commit/56e315035b4039ff630e5293d3974327bc80aec6 --- now that SV headers are guaranteed to survive into DVC, I think we should no longer worry about cleaning up the slot for the root prepare for this particular issue.
Let's see if the vopr finds anything problematic here.

Let's see if the vopr finds anything problematic here.

Did you mean to re-add the vopr label to this PR?

Ah, absolutely, thanks dj!


chore(release): release

Codecov Report

Merging #53 (185078b) into master (c1422d5) will decrease coverage by 7.54%.
The diff coverage is 98.00%.

@@             Coverage Diff             @@
##            master      #53      +/-   ##
===========================================
- Coverage   100.00%   92.45%   -7.55%
===========================================
  Files           20       18       -2
  Lines          226      212      -14
  Branches        22       22
===========================================
- Hits           226      196      -30
- Misses           0       14      +14
- Partials         0        2       +2

Impacted Files
Coverage Δ

packages/generator/src/lib/index.ts
100.00% <ø> (ø)

packages/parser/src/common/is-primitive.ts
100.00% <ø> (ø)

...ackages/parser/src/handlers/array-value-handler.ts
73.68% <0.00%> (ø)

...ages/parser/src/handlers/callback-value-handler.ts
100.00% <ø> (ø)

packages/parser/src/handlers/enum-value-handler.ts
100.00% <ø> (ø)

...arser/src/handlers/object-literal-value-handler.ts
100.00% <ø> (ø)

.../parser/src/handlers/primitive-handler-abstract.ts
100.00% <ø> (ø)

...ges/parser/src/handlers/primitive-value-handler.ts
100.00% <ø> (ø)

...ckages/reflect/src/lib/property-decorator-value.ts
23.07% <ø> (ø)

packages/reflect/src/lib/property.ts
100.00% <ø> (ø)

... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a529236...185078b. Read the comment docs.


⚠ GitLab Server has degraded performance

In 612965c, GitLab Server (https://gitlab01.its-telekom.eu) experienced degraded performance:

HTTP code: 200
Response time: 6896 ms

Resolved: GitLab Server performance has improved in e72a5c4.


Exception with the Gnokii library

Generate the xml file
$ h2xml gnokii.h -I ~/projects/Gnokyy/extra/gnokii/include/ -I ~/projects/Gnokyy/extra/gnokii/include/gnokii/ -o ../../../gnokii/gnokii.xml
creating xml output file ...
running: gccxml /tmp/tmpvUnUGQ.cpp -fxml=../../../gnokii/gnokii.xml -I /home/vampas/projects/Gnokyy/extra/gnokii/include/ -I /home/vampas/projects/Gnokyy/extra/gnokii/include/gnokii/

So far so good, but:
$ xml2cython.py gnokii.h ../../../gnokii/gnokii.xml
Ignoring <ctypeslib.codegen.typedesc.Structure object at 0xb721de0c> _IO_FILE
Item not handled for cy_generate <ctypeslib.codegen.typedesc.ArrayType object at 0x9e0d28c>
Item not handled for cy_generate <ctypeslib.codegen.typedesc.ArrayType object at 0x9e0d28c>
Item not handled for cy_generate <ctypeslib.codegen.typedesc.ArrayType object at 0x9e0d28c>
Item not handled for cy_generate <ctypeslib.codegen.typedesc.ArrayType object at 0x9e0de0c>
Item not handled for cy_generate <ctypeslib.codegen.typedesc.ArrayType object at 0x9e0de0c>
Item not handled for cy_generate <ctypeslib.codegen.typedesc.ArrayType object at 0x9e0d28c>
Item not handled for cy_generate <ctypeslib.codegen.typedesc.ArrayType object at 0x9e0d28c>
Item not handled for cy_generate <ctypeslib.codegen.typedesc.ArrayType object at 0x9dfc42c>
Item not handled for cy_generate <ctypeslib.codegen.typedesc.ArrayType object at 0x9dfc42c>
Not handled:  <ctypeslib.codegen.typedesc.Enumeration object at 0xb72247ac>
Traceback (most recent call last):
  File "/home/vampas/projects/DotSOS/.working-dir/installed/1.0b1-dev/bin/xml2cython.py", line 165, in <module>
    sys.exit(main())
  File "/home/vampas/projects/DotSOS/.working-dir/installed/1.0b1-dev/bin/xml2cython.py", line 148, in main
    ffilter=ffilter, funcs_list=funcs)
  File "/home/vampas/projects/DotSOS/.working-dir/installed/1.0b1-dev/bin/xml2cython.py", line 71, in generate_main
    generate_cython(output, gen, anoenumvals)
  File "/home/vampas/projects/DotSOS/.working-dir/installed/1.0b1-dev/lib/python2.7/site-packages/cycodegenlib/cycodegen.py", line 12, in generate_cython
    cython_code = [cy_generate(i) for i in genitems]
  File "/home/vampas/projects/DotSOS/.working-dir/installed/1.0b1-dev/lib/python2.7/site-packages/cycodegenlib/cycodegen.py", line 76, in cy_generate
    return cy_generate_structure(item)
  File "/home/vampas/projects/DotSOS/.working-dir/installed/1.0b1-dev/lib/python2.7/site-packages/cycodegenlib/cycodegen.py", line 41, in cy_generate_structure
    output.append("\t" + (generic_named_decl(m.typ) % m.name))
  File "/home/vampas/projects/DotSOS/.working-dir/installed/1.0b1-dev/lib/python2.7/site-packages/cycodegenlib/cytypes.py", line 66, in generic_named_decl
    return pointer_named_decl(tp)
  File "/home/vampas/projects/DotSOS/.working-dir/installed/1.0b1-dev/lib/python2.7/site-packages/cycodegenlib/cytypes.py", line 55, in pointer_named_decl
    return generic_decl(tp.typ.returns) + '(*%s)' + '(%s)' % ", ".join(args)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

What can I do now?

Hi s0undt3ch,
Unfortunately, I don't have much time to spend on cython-codegen. Given that cython-codegen is really an hack, I would not spend too much time on it if you cannot make it work. I am still hoping to get something similar to cython-codegen but based on clang working, which should be much easier to code (and more stable/reliable).

Ok, Thanks anyway.


BUG / FR / documentation: prism loading - possibility to load other languages; other prism components;

When adding the plugin and trying extending languages, only a few are available. Later on other languages will be accessible.
This is IMO a bug.

Adding new languages - init should be called after all other languages are loaded by the plugin (or prismjs)

Features I miss (some might just need documentation to be added):

default language fallback (text) - at least for MD
possibility to add aliases for languages
possibility to tell which languages get's loaded
also which components are loaded (like line highlight, line numbering, etc... although some might need extra plugin code)

    eleventyConfig.addPlugin(syntaxHighlight, {
        init: function({ Prism }) {

            Prism.languages.console = Prism.languages.extend('markup', {});
            Prism.languages.terminal = Prism.languages.extend('markup', {});
            console.log('init', Object.keys(Prism.languages))
        }
    });

init [
  'extend',   'insertBefore',
  'DFS',      'markup',
  'html',     'mathml',
  'svg',      'xml',
  'ssml',     'atom',
  'rss',      'css',
  'clike',    'javascript',
  'js',       'console',
  'terminal'
]

Later on languages include more: ruby,rb,yaml,yml,markdown,md,liquid,bash,shell,java,python,py
But still missing: cs, sql, ...

Came here for similar. I'm attempting to alias nunjucks to jinja2 or twig or django, and this seemed the appropriate way. However if I log the keys of Prism.languages a variable amount are output each time.

This would also great for using css-extras, since it is not itself a language but rather enhances css. In other words, you cannot autoload css-extras, and so you'd want to be able to specify it manually in your config.


Stop Typing Activity

How do we send a "stop typing" activity?
If it is not possible, what will it take to make it possible?
This is particularly necessary when the framework is being used for live agent escalation.

Can you clarify what the specific functionality looks like that you would like to see implemented, and if there is a bug preventing that? To note, if it's "clear the typing indicator when the next message is recieved" this can be done in webchat but the functionality might not match if you are using a different channel.

When interacting with a human via facebook messenger or most live chats I have encountered, the typing indicator tends to track whether the other party is actively typing or not. You will see it start and stop if the person at the other end begins typing something then (for example) switches to another window.
The request here is simply to support the ability to notify channels that typing has stopped without having to send a message.
I assume we could work around this on a per-channel basis using channel data - I have seen examples of how it is done via facebook, and I can imagine how we might use the backchannel with the webchat control. But this really seems like functionality that would be better implemented as part of the framework.

It looks like the webchat control shows typing for 3 seconds after getting a typing activity - so the idea is that we need to regularly send typing activities to keep the typing indicator showing. I misunderstood how it works - I thought it would show the typing indicator until it received another activity.
Is the behavior documented anywhere? I might have missed it.

Looking to find a proper doc for v4, but it is mentioned in the middleware itself here.  As for v3, it is right here in the api docs.

Yes, the Typing indicator is ephemeral and intended to last 3 seconds. Most channels standardized on 3 seconds before we built this protocol. See requirement A6000 here: https://github.com/Microsoft/BotBuilder/blob/master/specs/botframework-activity/botframework-activity.md


keeping availability group logins in sync automatically

Slack Community Channel: https://sqlcommunity.slack.com/messages/C1M2WEASG/ (get an invite: https://dbatools.io/slack/)
Documentation within the module: Get-Help <command>
Database Administrator on Stack Exchange network, dbatools tag: https://dba.stackexchange.com/questions/tagged/dbatools
StackOverflow dbatools tag: https://stackoverflow.com/questions/tagged/dbatools
For some reason the code fails to pick SQL ID (SQL authenticated Id) and transfer it to secondary replicas , however windows ID is getting transferred, also if i feed the AG IP on $AvailabilityGroupName and it fails for my SQL agent service account.
When i change the $AvailabilityGroupName to name instead of IP the windows ID gets trasnferred but SQL ID doesnt.
This is not uniform across but facing in some of my win 2012 R2 and now on Win 2016 servers
Any idea ?

please post the command you're using and the resulting failures, this question lacks every bit of detail needed to help you

WARNING: [06:53:44][Get-DbaAvailabilityGroup] Failure. | Login failed for user
'my windows sqlagent service account '.
Error
while syncing logins for Availability Group 'AGlistenername: Cannot bind
argument to parameter 'SqlInstance' because it is null.'
+ CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,SyncLoginsToReplica.ps1

this SQL agetn service account has sysadmin rights on DB and admin rights on server

SQL agent job runs the CMD exec command below and the .ps1 script is taken from dbatool blog only
Powershell -file "D:\Microsoft SQL Server\MSSQL12.MSSQLSERVER\AG\SyncLoginsToReplica.ps1"
define the AG name
$AvailabilityGroupName ='AGlistenername'
internal variables
$ClientName = 'AG Login Sync helper'
$primaryInstance = $null
$secondaryInstances = @{}

try {
# connect to the AG listener, get the name of the primary and all secondaries
$replicas = Get-DbaAgReplica -SqlInstance $AvailabilityGroupName
$primaryInstance = $replicas | Where Role -eq Primary | select -ExpandProperty name
$secondaryInstances = $replicas | Where Role -ne Primary | select -ExpandProperty name
# create a connection object to the primary
$primaryInstanceConnection = Connect-DbaInstance $primaryInstance -ClientName $ClientName
# loop through each secondary replica and sync the logins
$secondaryInstances | ForEach-Object {
$secondaryInstanceConnection = Connect-DbaInstance $_ -ClientName $ClientName
Copy-DbaLogin -Source $primaryInstanceConnection -Destination $secondaryInstanceConnection -ExcludeSystemLogins
}
}
catch {
$msg = $_.Exception.Message
Write-Error "Error while syncing logins for Availability Group '$($AvailabilityGroupName): $msg'"
}

You should rightfully focus on this one, the rest is just a result of not getting any data:
WARNING: [06:53:44][Get-DbaAvailabilityGroup] Failure. | Login failed for user 'my windows sqlagent service account '.

Double check that your SQL agent runs under the account that has proper permissions on the remote servers.
Try to run the script manually by opening Powershell using runas under the Sql Agent account.
Check your target server logs, login failures are logged by default. The error message is usually informative enough to find the root cause.

Point 2 and 3 helped to nail the issue, yes it has it on security logs, for some reason this issue crops on SQL2014 /Windows 2012R2 but i dont see the same on Windows 2016 /SQL2016.
Again thanks Kirill for the assistance.

Glad it worked for you.
Closing the issue.


Return the soapHeader

@srushti
We need to return the Response Soap Header (not http response headers)
So that we can get information like wsUsernameToken
thoughts?

whoops meant to PR this to my fork


Minor typo fix in vhosts page

This PR fixes typos in the vhosts page

@agileshaw Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@agileshaw Thank you for signing the Contributor License Agreement!


Users not saved

I'm trying to understand how it works. I have persistence, whitelist, keys and users set to true.
I've added a hash to whitelist, tried to announce to the tracker, to make it work I have to add a key through the api and then says working and it adds a record in the torrents table, yet nothing is persisted in the keys table.
I tried to add a used through the API, says "OK", but also not persisted in the DB.
Can someone help me understand how it works? How to persist users. Now after I add a key and use it, if I stop the client, the key is lost.

I'm trying to understand how it works. I have persistence, whitelist, keys and users set to true. I've added a hash to whitelist, tried to announce to the tracker, to make it work I have to add a key through the api and then says working and it adds a record in the torrents table, yet nothing is persisted in the keys table. I tried to add a used through the API, says "OK", but also not persisted in the DB.
Can someone help me understand how it works? How to persist users. Now after I add a key and use it, if I stop the client, the key is lost.

I'll look at it tomorrow.
Could be something is not "saving" when persistency is enabled.
I've not looked at this part of the code for some time, since I've been busy with the public tracking system.

Thank you @Power2All
These are the steps I did after adding a whitelist hash.
My guess is I'm doing something wrong, but I can't seem to find out what.

Add seeder with announce https://tracker.domain.com:6969/announce/2234567890123456789012345678901234567890 ( 'Invalid Key' )
Add key using API: POST /api/keys/2234567890123456789012345678901234567890/0?token=mytoken ( Now in the client status 'Working' )
Did the same with a leecher https://tracker.domain.com:6969/announce/1234567890123456789012345678901234567890
Trying to add a user using POST https://tracker.domain.com:8080/api/user?token=mytoken and JSON data

{
  "uuid":"11scbeca840-e9f7-44d5-a1e4-51e9aef864a7",
  "key":"1234567890123456789012345678901234567890",
  "uploaded":0,
  "downloaded":0,
  "completed":0,
  "updated":1686302997,
  "active":1
}

Get response
{
    "status": "ok"
}

Then if I list the users /api/users?token=mytoken response is

{
    "status": "json parse error"
}

This is in the console
2024-02-06 14:37:37.000367710 [INFO ][torrust_actix] [KEYS] Checking now for old keys, and remove them.
2024-02-06 14:37:37.000406821 [INFO ][torrust_actix] [KEYS] Keys cleaned up.
2024-02-06 14:37:37.000410662 [INFO ][torrust_actix] [STATS] Torrents: 1 - Updates: 0 - Shadow 0: - Seeds: 1 - Peers: 1 - Completed: 0
2024-02-06 14:37:37.000446561 [INFO ][torrust_actix] [STATS] Whitelists: 1 - Blacklists: 0 - Keys: 0
2024-02-06 14:37:37.000472682 [INFO ][torrust_actix] [STATS TCP IPv4] Connect: 8 - API: 5 - Announce: 3 - Scrape: 0
2024-02-06 14:37:37.000481342 [INFO ][torrust_actix] [STATS TCP IPv6] Connect: 0 - API: 0 - Announce: 0 - Scrape: 0
2024-02-06 14:37:37.000488396 [INFO ][torrust_actix] [STATS UDP IPv4] Connect: 0 - Announce: 0 - Scrape: 0
2024-02-06 14:37:37.000496987 [INFO ][torrust_actix] [STATS UDP IPv6] Connect: 0 - Announce: 0 - Scrape: 0
2024-02-06 14:37:37.017203426 [INFO ][torrust_actix] [SAVING] Starting persistence saving procedure.
2024-02-06 14:37:37.017223960 [INFO ][torrust_actix] [SAVING] Moving Updates to Shadow...
2024-02-06 14:37:37.017236671 [INFO ][torrust_actix] [SAVING] Saving data from Shadow to database...
2024-02-06 14:37:37.017848276 [INFO ][torrust_actix] [SAVING] Clearing shadow, saving procedure finishing...
2024-02-06 14:37:37.017868809 [INFO ][torrust_actix] [SAVING] Torrents saved.
2024-02-06 14:37:37.017878307 [INFO ][torrust_actix] [SAVING] Saving data from Whitelist to database...
2024-02-06 14:37:37.018239529 [INFO ][torrust_actix] [SAVING] Whitelists saved.
2024-02-06 14:37:37.018251263 [INFO ][torrust_actix] [SAVING] Saving data from Keys to database...
2024-02-06 14:37:37.018444375 [INFO ][torrust_actix] [SAVING] Keys saved.
2024-02-06 14:37:37.018453385 [INFO ][torrust_actix] [SAVING] Saving data from Users to database...
2024-02-06 14:37:37.018638536 [INFO ][torrust_actix] [SAVING] Users saved.
2024-02-06 14:37:37.018647615 [INFO ][torrust_actix] [SAVING] Saving persistent data procedure done.

Now I changed it to MySQL, but first I tried SQlite file. It created whitelist, keys and torrents table by itself, but no users table.
After I changed to MySQL, no tables were created, in the console was trowing errors when trying to save torrents and whitelist, I created manually the tables and it worked.
In both SQLite and MySQL I did not have a user table, but also never seen an error in the console.
And this is the config I used
log_level = "info"
log_console_interval = 60
statistics_enabled = true
global_check_interval = 10
db_driver = "mysql"
db_path = "mysql://root:password@localhost:3306/tracker"
persistence = true
persistence_interval = 60
api_key = "mytoken"
whitelist = true
blacklist = false
keys = true
keys_cleanup_interval = 60
users = true
maintenance_mode_enabled = false
interval = 1800
interval_minimum = 1800
peer_timeout = 2700
peers_returned = 200
interval_cleanup = 900
cleanup_chunks = 100000

[[udp_server]]
enabled = false
bind_address = "<IP_ADDRESS>:6969"

[[http_server]]
enabled = true
bind_address = "<IP_ADDRESS>:6969"
ssl = true
ssl_key = "/home/user/torrust-actix/target/release/privkey.pem"
ssl_cert = "/home/user/torrust-actix/target/release/cert.pem"

[[api_server]]
enabled = true
bind_address = "<IP_ADDRESS>:8080"
ssl = true
ssl_key = "/home/user/torrust-actix/target/release/privkey.pem"
ssl_cert = "/home/user/torrust-actix/target/release/cert.pem"

[db_structure]
db_torrents = "torrents"
table_torrents_info_hash = "info_hash"
table_torrents_completed = "completed"
db_whitelist = "whitelist"
table_whitelist_info_hash = "info_hash"
db_blacklist = "blacklist"
table_blacklist_info_hash = "info_hash"
db_keys = "keys"
table_keys_hash = "hash"
table_keys_timeout = "timeout"
db_users = "users"
table_users_uuid = "uuid"
table_users_key = "key"
table_users_uploaded = "uploaded"
table_users_downloaded = "downloaded"
table_users_completed = "completed"
table_users_updated = "updated"
table_users_active = "active"

Well, good question.
I'll have to look into that myself, but MySQL you need to create the tables yourself.
In the bin folder there is a MySQL table structure you can use to create them.
Errors I'm not sure, I've not bothered to look if it gave errors at all.
Probably the error is purely MySQL, but the code itself didn't trigger an error.

This is being fixed on version v4.0.0
Currently still WIP.

Fixed in v4.0.0
Use this now as the base.