Jedis connection timeout when conductor is running on heavy load

The following image shows the failed HTTP.

Different workflows are failing on different task
Common pattern:

All failed task in all workflow is HTTP.
All Got 409 on the first try even though the target server creates a new record.
3) The poll count is 2 (seems like conductor task send two HTTP requests on the same HTTP task)
The failed task took a long time to finish.

This problem is not related to timeout as timeout was set to a very long duration of 10 min.
Conductor log error/warnings

@manan164 any suggestion on this. This is not only happening to HTTP but worker task as well. The conductor is unable to poll timely and connection timeout.

From the exception posted above, this seems to be failing connections to redis-server. Could you please verify the configurations of your redis server?

So in our case (I work with @simarpal) it appears to be caused by a combination of:

RDB & AOF (everysec) persistence being enabled in Redis
Our custom workflow archiver module not clearing out old workflows, so
Redis uses large amounts of memory for storage due to old data
Writing RDB to the disk while under heavy write load due to AOF (everysec) causes severe latency spikes (>2s) which triggers the redis connection timeout.

To resolve this, we'll likely move to Conductor's contributed workflow archival module (so workflows get cleared out of redis faster). However @apanicker-nflx I was wondering, why does the indexDAO.searchForArchivableWorkflows method now have this:
.must(QueryBuilders.rangeQuery("endTime").lt(LocalDate.now().minusDays(archiveTtlDays).toString()).gte(LocalDate.now().minusDays(archiveTtlDays).minusDays(1).toString()))

in the query, it used to just have:
.must(QueryBuilders.rangeQuery("endTime").lt(LocalDate.now().minusDays(archiveTtlDays).toString()))

This narrows the search window down to 1 day. If the archival module was down for >1 day this means that some workflows might never get archived. So why the new limit on the query?

Also, on a semi-related note: is it possible to have multiple implementors for WorkflowStatusListener? If we plan to use conductor's archive module, and another module of our own (unrelated to archiving), will we need an intermediary layer that calls onWorkflowCompleted for both these modules?

@indiv0 The observations about the searchArchivableWorkflows method are correct, the limit was introduced to constrain the bound of the ES query. This was done as an experimental task but was never removed. Please feel free to refactor/redo this method.

Is it safe to set archiveTtlDays to -1 (i.e. same-day archival) if we're using synchronous indexing? Could that cause issues due to premature workflow archival? That is, could this result in us archiving workflows before they've been re-indexed, and would that be a problem?

In the case of synchronous indexing, you can use immediate archival upon completion. The latency for loading such workflows would be slightly higher since fetching data from ES is slower than fetching data from redis (in your case).

is it possible to have multiple implementors for WorkflowStatusListener? If we plan to use conductor's archive module, and another module of our own (unrelated to archiving), will we need an intermediary layer that calls onWorkflowCompleted for both these modules?

In the current setup, you would need an additional intermediary layer.

Thanks for the information!


Spotify Connect keeps dropping and music stops

The last week or so I haven't been able to get Spotify Connect to work reliably. The Pi shows up as a spotify connect device. It connects and will start playing a song, but then after a few seconds the music stops and the Pi will disappear from the devices list. Then a few seconds later it will show up again and you can reconnect. But every time you start to play music it stops after a few seconds. This behaviour has only started in the past week or two. It was working fine about a month ago when I first set up Raspotify.
I'm using a RPi3B+, wired network connection, and a hifiberry shield. Everything was freshly installed and updated about a month ago. As far as I can tell the network is fine. The Pi is still reachable the whole time.
Any ideas? did Spotify change something again? Or is it just me?

I also experience the Spotify disappearing and reappearing in the spotify connect menu, but i dont experience it disconnecting after a couple of seconds.
However when I edited /etc/default/raspotify to add a new OPTION that pointed to a bad script (ie one with a critical error) then I would experience spotify disconnecting after about 1 second.
I'd recommend sharing your /etc/default/raspotify, and checking what you might be calling that is causing this issue. off the top of my head, maybe the hardware number of the hifiberry got changed during an update?

I think I figured it out. I was using a 4gb SD card and a minimal debian buster install. I knew it was going to be a tight install but I thought it would be ok. Turns out I was running low on disk space. I cloned to a larger 16gb card and expanded the filesystem and now it appears to be working without dropping.

Turns out this is not solved as I had suspected. It turns out that it is infact raspotify that is filling up the drive. After switching to a larger SD card it was fine for a while but has managed to consume all of the free space once again.
/var/cache/raspotify/files/ is getting filled with a bunch of folders and files usually less than 100mb.

Any ideas?

Looks like it's related to this: https://github.com/dtcooper/raspotify/issues/144


python: make sure tests fail when the test cases include file types magika doesn't support

@reyammer The python package needs a new release, the one on Pypi doesn't work with py3.12 and doesn't have the typing fixes for 3.8/3.9

@gaby yep, working on it right now.
And closing this, fixed in https://github.com/google/magika/commit/91cfb0aa83abd5ae260308a27b7e844ca95eee12


Error with iCloud Birthday calendar

It was working fine and keep getting this error now...
Whoops! There was an uncaught exception...
{ [Error: EACCES: permission denied, open '/home/pi/MagicMirror/modules/MMM-iCloud-Client/contacts.db']
errno: -13,
code: 'EACCES',
syscall: 'open',
path: '/home/pi/MagicMirror/modules/MMM-iCloud-Client/contacts.db' }
MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues

This seems like a problem with the MMM-iCloud-Client module. I suggest you open an issue in that repository.


A release would be nice

Since two weeks have passed since I provided this PR a patch version bump would be nice :P

sorry, emberconf + moving has kept me busy. Will release tonight. If i don't by tomorrow please herras me.

Haha. I hope the recordings will be released soon. I'm curious :)

@MajorBreakfast they are out!

Nice. I usually rely on ember weekly for ember news and the last mail didn't include them yet. So they cannot have been out for long :P

https://www.youtube.com/playlist?list=PLE7tQUdRKcyacwiUPs0CjPYt6tJub4xXU
i think the first ones were out march 6. Regardless I hope you enjoy.

I watched your talk this morning. Was quite enjoyable :)

sorry that it took so long

Thx anyways :)


TimeZone

Umbrella issue tracking TimeZone

TimeZone was moved with https://github.com/apple/swift-foundation/pull/10


build breaks with webpack.js

When building project that imports html-colors using Webpack.js, build fails at
https://github.com/radiovisual/html-colors/blob/7b7be29ffd77f0a736bb826f0586bc898cbfc2a5/index.js#L3
Webpack is expecting js module.
To fix that,  .json extension is needed to be specified.
const htmlColors = require('./html-colors.json');

This was fixed in the now published version 0.0.5 👍 Thanks for reporting!


longQC.py error

Hi, I am trying to run longQC.py with the following command. However, it is showing following error. Please suggest.
python longQC.py sampleqc -x pb-rs2 -o /longqc/ merge.fastq.gz
error:
Traceback (most recent call last):
File "/media/bmaurice/Data2/Hybrid_assembly_virus/DMV10_IBV/longqc/LongQC/longQC.py", line 20, in
import pandas as pd
ModuleNotFoundError: No module named 'pandas'

Hi @sekhwal ,
Is a common error when you don't have pandas. try to run pip3 install pandas

many thanks for @Sanrrone!
Let me close this issue for now, and kindly open when further input needed.


Update mention of Address Book to be consistent with CareConnect

Currently, terms like "Address Book", "Address App" etc. are still used in various places across the app (such as in the app title, name of the storage file on disk etc.)
We may want to consider renaming these to better reflect the purpose of our app which is an address book specifically for managing social work cases
Alternative considered
Or we can just leave it as it is, since Address Book technically also captures what we're doing

fixed with commit 7ada1d8671a2d22999b4bc7c93428faf40f98559


Simulating tx on block 0 fails

That's not critical, but it could be interesting to understand why.
I think it can be debugged with pallet tests, if we don't walk a few block as we usually do before running our tests
Originally posted by @EvolveArt in https://github.com/keep-starknet-strange/madara/pull/1416#discussion_r1483091811

repository archived in favor of https://github.com/madara-alliance/madara


added and updated game messages to be more consistent

Codecov Report

Merging #31 into develop will decrease coverage by 0.80%.
The diff coverage is 12.12%.

@@             Coverage Diff             @@
##           develop      #31      +/-   ##
===========================================
- Coverage    50.69%   49.88%   -0.81%
===========================================
  Files           16       16
  Lines          434      441       +7
===========================================
  Hits           220      220
- Misses         214      221       +7

Impacted Files
Coverage Δ

euchrecli/command/play.py
10.94% <8.00%> (-0.51%)
:arrow_down:

euchrecli/abstract/human.py
20.83% <14.28%> (-0.45%)
:arrow_down:

euchrecli/abstract/player.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5890c15...fd9a594. Read the comment docs.


yarn: 1.7.0 -> 1.8.0

Motivation for this change
Version update.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

~[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)~
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg  build yarn yarn2nix

Success on aarch64-linux (full log)
Attempted: yarn, yarn2nix
Partial log (click to expand)
Linked yarn2nix
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/sdwyfrc9a91fh94h64cg3xgchc4dw4bv-yarn2nix-1.0.0
strip is /nix/store/qg2agrqkf240s656d207zqhipl0bc2id-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/sdwyfrc9a91fh94h64cg3xgchc4dw4bv-yarn2nix-1.0.0/bin
patching script interpreter paths in /nix/store/sdwyfrc9a91fh94h64cg3xgchc4dw4bv-yarn2nix-1.0.0
/nix/store/sdwyfrc9a91fh94h64cg3xgchc4dw4bv-yarn2nix-1.0.0/node_modules/yarn2nix/bin/yarn2nix.js: interpreter directive changed from "/usr/bin/env node" to "/nix/store/138777ah3x4h0bvnr0wjkgmi1x8hzidr-nodejs-8.11.2/bin/node"
checking for references to /build in /nix/store/sdwyfrc9a91fh94h64cg3xgchc4dw4bv-yarn2nix-1.0.0...
/nix/store/slbvxwcwrfvxgch9mzg7131fs866b3av-yarn-1.8.0
/nix/store/sdwyfrc9a91fh94h64cg3xgchc4dw4bv-yarn2nix-1.0.0

Success on x86_64-linux (full log)
Attempted: yarn, yarn2nix
Partial log (click to expand)
Linked yarn2nix
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/b9xk6m3pbk72dbpv0sm46jhf8n9xj7nz-yarn2nix-1.0.0
strip is /nix/store/21ymadblbmsbb2bk4q7gl4kjasp8zmgd-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/b9xk6m3pbk72dbpv0sm46jhf8n9xj7nz-yarn2nix-1.0.0/bin
patching script interpreter paths in /nix/store/b9xk6m3pbk72dbpv0sm46jhf8n9xj7nz-yarn2nix-1.0.0
/nix/store/b9xk6m3pbk72dbpv0sm46jhf8n9xj7nz-yarn2nix-1.0.0/node_modules/yarn2nix/bin/yarn2nix.js: interpreter directive changed from "/usr/bin/env node" to "/nix/store/a3wjipkj72fgnq9lrswcdcchcr8z6qqh-nodejs-8.11.2/bin/node"
checking for references to /build in /nix/store/b9xk6m3pbk72dbpv0sm46jhf8n9xj7nz-yarn2nix-1.0.0...
/nix/store/2aky77k54n1baqc91vxlynvr4y7phlhh-yarn-1.8.0
/nix/store/b9xk6m3pbk72dbpv0sm46jhf8n9xj7nz-yarn2nix-1.0.0

Success on x86_64-darwin (full log)
Attempted: yarn, yarn2nix
Partial log (click to expand)
installing
Processing  yarn2nix
Linked yarn2nix
post-installation fixup
strip is /nix/store/yyak5sjv68n9vdgnkd2cb5djk1l9sqj5-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/wfr006cgslviy462v2xd5l3zl8m65qzj-yarn2nix-1.0.0/bin
patching script interpreter paths in /nix/store/wfr006cgslviy462v2xd5l3zl8m65qzj-yarn2nix-1.0.0
/nix/store/wfr006cgslviy462v2xd5l3zl8m65qzj-yarn2nix-1.0.0/node_modules/yarn2nix/bin/yarn2nix.js: interpreter directive changed from "/usr/bin/env node" to "/nix/store/24mslvs65h5gxv4lkmsscvpng2y630ak-nodejs-8.11.2/bin/node"
/nix/store/8h50ncjmpkwillz37pqbm256wisrjgr0-yarn-1.8.0
/nix/store/wfr006cgslviy462v2xd5l3zl8m65qzj-yarn2nix-1.0.0


Update nix to 0.25

Any particular reason for this change?

In fd I noticed that we now have both 0.24 and 0.25 of nix in the dependency graph. argmax requries 0.24.* and ctrlc (at least version 3.2.3) requires 0.25.0.
I suppose another way to avoid the duplicate dependency would be to change the dependency to something like
version = ">=0.24.2,<0.26"

also, nix probably isn't a very big dependency, so it might not matter too much.


Documentation> "Hyperlink Props" link> Link "Hyperlink Props" is navigating to incorrect location

Describe the bug
Link "Hyperlink Props" is navigating to heading "Inverse" instead of heading "Search Props"
To Reproduce
Steps to reproduce the behavior:
1.Go to https://react-magma.cengage.com/version/2.5.7/patterns/header/#call_to_action
2.Click on "Hyperlink Props" link
3. Verify that link is navigating to heading "Inverse"
Expected behavior
Link "Hyperlink Props" should navigate to correct location.
Screenshots:
https://somup.com/c3n21bZWJ1
Desktop:
OS: [Win 10]
Browser [chrome]
Version [Chrome version: Version  98.0.4758.82 (Official Build) (64-bit)]

The issue is fixed.
See Screen-cast : https://www.screenpresso.com/=JkUte
Hence, Closing the ticket.
Thanks!


Upload a file button gets no focus

How would you expect it to work?

I would expect the button to receive tabbed focus and a yellow border appear like all other form elements. The enter key or space bar may open up the file selection dialog.
How does it work currently?

Skips tabbed focus completely.
Feel free to suggest a fix...

Use javascript to set a focussed class on the button wrapper when the hidden input has focus.
Because the input is using display: none we won't be able to get focus so perhaps using opacity: 0 may work, or text-indent: -999px. I have not tried any of these approaches yet.
Explain the steps to reproduce the bug

Tabbing through the DOM.
Tell us about your environment

Version used:
Latest version
Browser Name and version:
Firefox 48.0.1
Operating System and version (desktop or mobile):
Mac OSX El Captain
Link to your project:
http://tfl-prosecutor.herokuapp.com/send-data-files

Perhaps a better solution is here:
http://tympanus.net/codrops/2015/09/15/styling-customizing-file-inputs-smart-way/

Hey @trevorsaint, looks like these do have some problems with regards to accessibility and browser support, recommend using the default inputs until we have a fix for the inputs :)
Thanks for raising!

Hi back @nickcolley I have it now working in my version and did not require that much work to be honest. Firefox has a little issue with focus but have a JS solution to add a focused class. It works well. Take a peak: http://tfl-prosecutor.herokuapp.com/send-data-files

Hey @trevorsaint,
Looks better but there's still a few issues:

The input is not functioning as expected with JavaScript turned off, while it will technically work there's not enough feedback to the user.

In legacy browsers (for example IE8) the input does not function with the keyboard etc

In other legacy browsers (e.g. IE7-6) the input does not function at all, so we should fallback to a native input here.

I've not been able to properly test this in any assistive technology yet for example screenreaders

There could be more issues I'm not aware of at the current time, that's why I recommend using the native upload input until we can get this properly tested.

Hi @nickcolley Yes I know more needs doing :).
I think perhaps .js element would be good here so If no-js we would have an alternative being the standard file upload. I will spend some time looking into it also. Thanks Nick.

@nickcolley Improved :)
http://tfl-prosecutor.herokuapp.com/send-data-files

Great, working well without JS! 🚀
Just to illustrate why older browser keyboard support is important here...
With your current revision IE8 works with a cursor but keyboard access is broken.
This is important when you consider the people using JAWs on IE8 (http://webaim.org/projects/screenreadersurvey6/#browsercombos)
So you could consider falling back like you have done without JavaScript under IE9, this would enable fix the issues for anyone using IE6-8 (including screenreader users)
If you'd like to contribute to govuk_elements feel free to do a pull request, no worries if not 😄

Hi @nickcolley I will do just that :).

We've done something similar on passports renewals for our photo upload. We visually hide the file input and style the label as a button and set role="button".
DAC (Digital Accessibility Centre) have tested it and it was mostly fine, apart from with older versions of Dragon Naturally Speaking that don't understand the button role. If memory serves, I think JAWS may read out two file inputs but this is partly to do with the way IE handles the input and filename in the first place.
We auto-submit the form once a file has been selected (so don't need to show the filename). This has tested well with our users.
It's working in IE8 and above, for older browsers we show show the defaults. Have a look at https://www.passport.service.gov.uk/photo/upload (you will  get thrown back to the start of the journey but it's only about five pages in).

For now, we're going to revert what we have back to the native controls (as @nickcolley mentioned above), until there's time to find a better custom solution.


Error: Cannot find module 'nexmo' & error TS2307: Cannot find module nexmo

Hello, I used the node nexmo library and followed the instructions from this page:
https://dashboard.nexmo.com/getting-started/verify?firstLogin=true?
I use the nestjs framework with typescript.
Everything works fine as long as I'm in development, but when I try to build on the server I get these errors:
Error: Cannot find module 'nexmo'
error TS2307: Cannot find module nexmo
I think the problem is something either with require vs import or something with the tsconfig.json.
I'm trying to understand the problem.
I read about some configurations in the tsconfig.json like:
"moduleResolution": "classic" vs "moduleResolution": "node" and
"module": "commonjs" vs "module": "AMD"
I tried every possible scenario but I'm still getting errors.
Can you please help me?
Update:
This is my tsconfig.json file
{
"compilerOptions": {
"module": "commonjs",
"declaration": true,
"removeComments": true,
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
"target": "es2017",
"sourceMap": true,
"outDir": "./dist",
"baseUrl": "./",
"incremental": true,
"esModuleInterop": true
},
"exclude": ["node_modules", "dist"]
}

Can you check if you are on version 2.9.0 or 2.9.1?

Can you check if you are on version 2.9.0 or 2.9.1?

From package.json: "nexmo": "^2.9.1",

I'm going to take a look this morning and try to replicate to see what I can figure out.

I'm going to take a look this morning and try to replicate to see what I can figure out.

Thank you!
Do you need me to send you some of my code?
My now tsconfig.json for example? Or something else?

Update:
I made a question today in stack overflow too: https://stackoverflow.com/questions/63487624/error-cannot-find-module-nexmo-error-ts2307-cannot-find-module-nexmo

That would probably help a great deal - is there a way I can clone your repo? I've never used NestJS, so I'm starting from scratch.

Hopefully we find a solution - I've either broken our typescript implementation, or there is another solution. I would think that using a packaged library wouldn't cause this problem, so it's probably me.

You can also test this out on your end - install<EMAIL_ADDRESS>and let me know if this goes away.

<EMAIL_ADDRESS>
I will try right away! Should I first npm uninstall the previous nexmo I installed yesterday?

May as well, but I don't think you'd need to.

So what I did now:

npm uninstall nexmo
npm install<EMAIL_ADDRESS>Build in production failed with: Error: Cannot find module 'nexmo'
Added in tsconfig.json -> "moduleResolution": "node" and builded in production, failed with: Error: Cannot find module 'nexmo'
Changed "moduleResolution": "node" to "moduleResolution": "classic" and builded for production, failed with: src/main.ts(1,29): error TS2307: Cannot find module '@nestjs/core'.
src/main.ts(4,25): error TS2307: Cannot find module 'dotenv'.

Should I keep asking the library into my service like this:
const Nexmo = require('nexmo');
Or should I change it to something like this:
import Nexmo from 'nexmo' ?

Ok - so seems to be something related with how our library interacts with NestJS specifically.
You can try using import Nexmo from 'nexmo' and see if that makes any difference - but I'm not sure it will.
You might also try referencing the d.ts file with
/// <reference path="../node_modules/nexmo/typings/index.d.ts" />

Reference it like this gives me this error:
nest-ffs-backend/node_modules/nexmo/typings/index.d.ts' is not a module.ts(2306)

It would be exactly as written with the triple slashes and everything, at the top of the page.

import { Nexmo } from "../../../node_modules/nexmo/typings/index.d.ts"

import { Nexmo } from "../../../node_modules/nexmo/typings/index"

/// <reference path="../node_modules/nexmo/typings/index.d.ts" />
import Nexmo from 'nexmo'

In development it seems it doen't gives errors. But in production it gives this:

Ok - it's just not able to find the module for some reason. I'll see what I can figure out on my end.

As I wrote in stack overflow:
A friend of mine told me to check something about installing typings but NestJs already uses @types which from what I read is a more updated version of typings.
Can it be something like this? I don't know what typings are and I haven't used them so far.
So I don't know if installing them, may cause problems with NestJs. I saw the folder into node_modules/nexmo/ is called typings can it be an old and a new way of making and reading a module - library?

Can you post your tsconfig.json for me as well?

{
"compilerOptions": {
"module": "commonjs",
"declaration": true,
"removeComments": true,
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
"target": "es2017",
"sourceMap": true,
"outDir": "./dist",
"baseUrl": "./",
"incremental": true,
"esModuleInterop": true
},
"exclude": ["node_modules", "dist"]
}

Ok - tried a few things from a fresh install, and here is what I got to work:
//tsconfig.json
{
  "compilerOptions": {
    "module": "commonjs",
    "declaration": true,
    "removeComments": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "allowSyntheticDefaultImports": true,
    "esModuleInterop": true, <- this seems to be the important one here
    "target": "es2017",
    "sourceMap": true,
    "outDir": "./dist",
    "baseUrl": "./",
    "incremental": true
  }
}

//app.controller.ts
import { Controller, Get } from '@nestjs/common';
import { AppService } from './app.service';
import Nexmo from 'nexmo';

const nexmo = new Nexmo({
  apiKey: '',
  apiSecret: '',
});

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Get()
  getHello(): string {
    console.log(nexmo.verify);
    return this.appService.getHello();
  }
}

I then ran
~/G/A/test-nest> master > nest build
~/G/A/test-nest >master >node ./dist/main
[Nest] 21347   - 08/19/2020, 11:36:27 AM   [NestFactory] Starting Nest application...
[Nest] 21347   - 08/19/2020, 11:36:27 AM   [InstanceLoader] AppModule dependencies initialized +18ms
[Nest] 21347   - 08/19/2020, 11:36:27 AM   [RoutesResolver] AppController {}: +7ms
[Nest] 21347   - 08/19/2020, 11:36:27 AM   [RouterExplorer] Mapped {, GET} route +3ms
[Nest] 21347   - 08/19/2020, 11:36:27 AM   [NestApplication] Nest application successfully started +3ms

Is this "nexmo": "^2.8.0" or "nexmo": "^2.9.1" ?

  "dependencies": {
    "@nestjs/common": "^7.0.0",
    "@nestjs/core": "^7.0.0",
    "@nestjs/platform-express": "^7.0.0",
    "nexmo": "^2.9.1",
    "reflect-metadata": "^0.1.13",
    "rimraf": "^3.0.2",
    "rxjs": "^6.5.4"
  },

Where did you ran:

nest build
node ./dist/main

Inside VSCode's terminal?

Correct - that was in a terminal.

If I do it in the terminal of vscode, it works for me too.
What I meant when I said it doesn't work in production, is that when I'm building it up on the server live it makes the errors I told you.
I'm using Docker for the build.
Update:
Solved!
I just noticed that for some reason the RUN npm install inside the Dockerfile wasn't creating the nexmo folder in the node_modules.
I copied the nexmo folder from my local node_modules into the server and it worked.
So it was a docker problem that I haven't encountered before and not something wrong with nexmo.
Thank you for your time!


Add godatadriven/dbt-bouncer

Thanks for the PR! Just remove the /<package> and it'll be good to go; owners' packages are automatically discovered.

Thanks for the PR! Just remove the /<package> and it'll be good to go; owners' packages are automatically discovered.

Done!


V2 Linting and Pre-Commit Hooks

PR Objectives:

[x] Remove pylint, leaving just flake8 as a code checker, both in the CI and locally.
[x] Add  pre-commit configuration to project root, with black and flake8 hooks, for autoformatting and checking when committing, to find and fix issues before the build process.
[x] Updates development documentation with local pre-commit usage.

(each in a commit to ease on the review)
Will be happy if the reviewer will use the documentation and test the logics locally to eliminate environment-related issues.

Looks good. 👍
Small remark: we need to consider adding a stage to the pipeline so we fail the build if the user did not run the pre-commit script.
I don't think it is top priority but it's nice to have a ticket about it for the future.

It's already there, flake8 is currently running in the build

Looks good. 👍
Small remark: we need to consider adding a stage to the pipeline so we fail the build if the user did not run the pre-commit script.
I don't think it is top priority but it's nice to have a ticket about it for the future.

It's already there, flake8 is currently running in the build

@omri374 let's extend this, in another PR, using flake8 plugins. It's actually becoming preferred since we can have more control on what we check for, and as of now, run it as part of the pre-commit as well.

@omri374 let's extend this, in another PR, using flake8 plugins. It's actually becoming preferred since we can have more control on what we check for, and as of now, run it as part of the pre-commit as well.


Allow folder-specific configuration

Closing this one for now since it doesn't work well enough with boolean flags.


Add P Ratings To Array

Ensure people that have the P0 if they also have another rating.

:tada: This PR is included in version 6.4.10 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


More flexible topology rewiring for glob schemas

Consider a process (e.g. multibody physics) with a ports schema like this:
{
    'agents': {
        '*': {
            'a': {...},
            'b': {...},
        },
    },
}

Now imagine that we want to use this process with an agent with a store hierarchy like this:
{
    'a': {...}
    'boundary': {
        'b': {...},
    },
}

We don't currently have a way to wire the environment process to this agent.

Proposed topology syntax for this situation:
{
    'agents': {
        '_path': ('agents',),
        ('*', 'b'): ('agents', '*', 'boundary', 'b'),
    }
}

I'll work on this!

'agents' is that root path, so I think it should be more like this. 'agents' is removed from the rewired path.:
{
    'agents': {
        '_path': ('agents',),
        ('*', 'b'): ('*', 'boundary', 'b'),
    }
}

I'm not sure this is a feature we want to support.... I think for various
reasons this could end up being a problem. Star schemas are different in a
number of ways, conceptually and implementation wise, and are really meant
to stand as a unified group of related things. I think if you want
functionality like this you are better off just modeling the different
elements directly and avoiding the star schema altogether. Happy to go into
this further.... star schemas are really a convenience that are not
strictly required (anything modeled as a star schema could have just been a
regular schema with a little more explicit handling), sometimes I wonder if
the added complexity is really worth it. A feature like this really crosses
that threshold for me, but like I say, happy to discuss.
On Sat, Mar 19, 2022, 4:59 PM Eran Agmon @.***> wrote:

'agents' is that root path, so I think it should be more like this.
'agents' is removed from the rewired path.:
{
'agents': {
'_path': ('agents',),
('', 'b'): ('', 'boundary', 'b'),
}
}
—
Reply to this email directly, view it on GitHub
https://github.com/vivarium-collective/vivarium-core/issues/168#issuecomment-1073132373,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAACG5YEZYEPYHVR4BGW7Q3VAZS7LANCNFSM5PIS2BPQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

The reason for this would be to rewire one process's glob ports schema to match another's. The * glob does something that can't be done otherwise -- it can access n substore simultaneously as the simulation grows. You can't do that with a fixed topology in advance.


marge dependency is vulnerable

Environment
- OS: Ubuntu 22.04
- Node: 17.6.0
- cypress-mochawesome-reporter: 2.3.0
- cypress: 9.3.1

What happened?
I noticed on the npm audit the next output
Inefficient Regular Expression Complexity in validator.js - https://github.com/advisories/GHSA-qgmg-gppg-76g5
No fix available
node_modules/validator
mochawesome-report-generator  1.1.1 - 5.2.0
Depends on vulnerable versions of validator
Depends on vulnerable versions of yargs
node_modules/mochawesome-report-generator
cypress-mochawesome-reporter  *
Depends on vulnerable versions of mochawesome-report-generator
node_modules/cypress-mochawesome-reporter
mochawesome  2.1.0 - 6.3.1
Depends on vulnerable versions of mochawesome-report-generator
node_modules/mochawesome
Config file
{
  "viewportHeight": 1080,
  "viewportWidth": 1920,
  "component": {
    "viewportHeight": 1024,
    "viewportWidth": 1280
  },
  "ignoreTestFiles": "**/*.md",
  "reporter": "cypress-mochawesome-reporter",
  "reporterOptions": {
    "reportDir": "cypress/reports",
    "charts": true,
    "reportPageTitle": "STC App Team Checklist",
    "embeddedScreenshots": true,
    "inlineAssets": true
  },
  "video": false
}

Relevant log output
No response
Anything else?
No response

Fixed in v3.0.0


[Question] How to create a link to another Endpoint

I have two endpoints :
one which create a resource
    /// <summary>
    /// Creates new appointment
    /// </summary>
    [HttpPost("appointments")]
    [AllowAnonymous]
    public class CreateAppoinmentEndpoint : Endpoint<NewAppointmentInfo, Browsable<AppointmentInfo>>
    {
        private readonly IUnitOfWorkFactory _unitOfWorkFactory;
        private readonly LinkGenerator _linkGenerator;

        /// <summary>
        /// Builds a new <see cref="CreateAppoinmentEndpoint"/>
        /// </summary>
        /// <param name="unitOfWorkFactory"></param>
        /// <param name="linkGenerator"></param>
        public CreateAppoinmentEndpoint(IUnitOfWorkFactory unitOfWorkFactory, LinkGenerator linkGenerator)
        {
            _unitOfWorkFactory = unitOfWorkFactory;
            _linkGenerator = linkGenerator;
        }

        ///<inheritdoc/>
        public override async Task HandleAsync(NewAppointmentInfo req, CancellationToken ct)
        {
            using IUnitOfWork unitOfWork = _unitOfWorkFactory.NewUnitOfWork();

            Appointment newAppointment = new(AppointmentId.New(), req.Subject, req.Location, req.StartDate.ToInstant(), req.EndDate.ToInstant());
            foreach (AttendeeInfo attendee in req.Attendees)
            {
                newAppointment.AddAttendee(new Attendee(AttendeeId.New(), attendee.Name, attendee.Email, attendee.PhoneNumber));
            }

            await unitOfWork.Repository<Appointment>().Create(newAppointment, ct);
            await unitOfWork.SaveChangesAsync(ct).ConfigureAwait(false);

            AppointmentInfo appointmentInfo = new()
            {
                Id = newAppointment.Id,
                Location = newAppointment.Location,
                StartDate = newAppointment.StartDate,
                EndDate = newAppointment.EndDate,
                Subject = newAppointment.Subject
            };

            Browsable<AppointmentInfo> browsable = new()
            {
                Resource = appointmentInfo,
                Links = new[]
                {
                    new Link
                    {
                        Href = _linkGenerator.GetUriByName(HttpContext, nameof(GetAppointmentByIdEndpoint), new { newAppointment.Id }),
                        Method = "GET",
                        Relations = new[] { LinkRelation.Self }
                    }
                                                }
            };

            await SendCreatedAtAsync<GetAppointmentByIdEndpoint>(new { newAppointment.Id }, browsable, cancellation: ct);
        }
    }
}

another that gets a resource by its id.
/// <summary>
/// Gets an appointment by its id
/// </summary>
public class GetAppointmentByIdEndpoint : Endpoint<AppointmentId, Browsable<AppointmentInfo>>
{
    private readonly IUnitOfWorkFactory _unitOfWorkFactory;
    private readonly LinkGenerator _linkGenerator;

    /// <summary>
    /// Builds a new <see cref="GetAppointmentByIdEndpoint"/> instance.
    /// </summary>
    /// <param name="unitOfWorkFactory"></param>
    /// <param name="linkGenerator"></param>
    public GetAppointmentByIdEndpoint(IUnitOfWorkFactory unitOfWorkFactory, LinkGenerator linkGenerator)
    {
        _unitOfWorkFactory = unitOfWorkFactory;
        _linkGenerator = linkGenerator;
    }

    /// <inheritdoc/>
    public override void Configure()
    {
        Get("/appointments/{@id}", x => new { Id = x.Value });
        Head("/appointments/{@id}", x => new { Id = x.Value });
        AllowAnonymous();
    }

    /// <inheritdoc/>
    public override async Task HandleAsync(AppointmentId req, CancellationToken ct)
    {
        using IUnitOfWork unitOfWork = _unitOfWorkFactory.NewUnitOfWork();
        Option<AppointmentInfo> mayBeAppointment = await unitOfWork.Repository<Appointment>()
                                                                   .SingleOrDefault(selector: appointment => new AppointmentInfo
                                                                   {
                                                                       Id = appointment.Id,
                                                                       Subject = appointment.Subject,
                                                                       Location = appointment.Location,
                                                                       EndDate = appointment.EndDate,
                                                                       StartDate = appointment.StartDate,
                                                                       Attendees = appointment.Attendees.Select(attendee => new AttendeeInfo
                                                                       {
                                                                           Id = attendee.Id,
                                                                           Email = attendee.Email,
                                                                           Name = attendee.Name,
                                                                           PhoneNumber = attendee.PhoneNumber,
                                                                       }),
                                                                   },
                                                                                predicate: (Appointment x) => x.Id == req,
                                                                                cancellationToken: ct).ConfigureAwait(false);

        await mayBeAppointment.Match(
            some: async appointment =>
            {
                Browsable<AppointmentInfo> browsable = new()
                {
                    Resource = appointment,
                    Links = new[]
                    {
                        new Link
                        {
                            Href = _linkGenerator.GetUriByRouteValues(HttpContext, nameof(GetAppointmentByIdEndpoint), new { Id = req }),
                            Method = "GET",
                            Relations = new [] { LinkRelation.Self }.ToHashSet()
                        },
                    }
                };
                await SendOkAsync(browsable, ct).ConfigureAwait(false);
            },
            none: async () => await SendNotFoundAsync(ct).ConfigureAwait(false))
                              .ConfigureAwait(false);
    }
}

What I would like to do, is being able to create the url to point to the GET /appointments/{id} so that I could give that url back (as a part of the data returned by the endpoint which create the resource).
Is there a way to construct the url to a endpoint using LinkGenerator and the current HttpContext ?

you can do it by specifying a name/operation id for the target endpoint like so:
public class Endpoint : EndpointWithoutRequest
{
    public LinkGenerator LinkGenerator { get; set; }

    public override void Configure()
    {
        Get("test");
        AllowAnonymous();
    }

    public override async Task HandleAsync(CancellationToken c)
    {
        var targetURL = LinkGenerator.GetUriByName(HttpContext, "THE_TARGET", new { id = 123 });
        await SendAsync(targetURL);
    }
}

public class Target : EndpointWithoutRequest
{
    public override void Configure()
    {
        Get("target/{id}");
        AllowAnonymous();
        Options(x => x.WithName("THE_TARGET"));
    }

    public override async Task HandleAsync(CancellationToken c)
    {
        await SendAsync(Route<int>("id"));
    }
}


Support out of tree issuers in ingress-shim

Enables supporting out of tree issuers with ingress annotations

fixes #2100
/assign
/hold

/hold cancel
/assign @munnerz

/unassign
/assign @munnerz

Should be good for another look
/unassign
/assign @munnerz

/lgtm


handle local source file URLs!

todo: swap HTML content URLs with saved source file relative paths.
as of last commit, the URLs that point to source files, reference the structure of the remote site. so, since the source files are being saved locally, we need to refer to this new structure. so we're remapping URLs.

page document works now. some lose ends with stray elements. especially hosted 3rd party scripts, styles, and img srcsets will need to be worked on. to be continued...


RE: /features/private-sourcemaps

I'm finding it difficult to debug which step I'm missing in my implementation of private source maps. It looks like I'm uploading the source map files successfully (the requests return ids and status 200s). I'm adding explicit URLs to the context. But it appears as though the association has not been successful
`"sourceMapEnabled": true,
  "sourceMapErrors": {
    "https://cdn.summitlearning.org/javascripts/19-7e4ad96c.js": "http.Get url=\"https://cdn.summitlearning.org/javascripts/19-7e4ad96c.js.map\": got 403 Forbidden, expected 200 OK",
    "https://cdn.summitlearning.org/javascripts/new_relic_prod.js": "http.Get url=\"https://cdn.summitlearning.org/javascripts/new_relic_prod.map\": got 403 Forbidden, expected 200 OK"
  },
  "sourceMaps": {
    "https://cdn.summitlearning.org/javascripts/19-7e4ad96c.js": "https://cdn.summitlearning.org/javascripts/19-7e4ad96c.js.map"
  }`

According to documentation, if I successfully upload a file with name="https://cdn.summitlearning.org/javascripts/19-7e4ad96c.js.map" to airbrake, it should be able to find the uploaded version of https://cdn.summitlearning.org/javascripts/19-7e4ad96c.js.map to connect it the error.
One note : it would be helpful to be able to get / see the successfully uploaded source maps, as well as their names.

I'm using the RestClient API to make the request, but essentially looks like this:
response = RestClient.post(
          "https://airbrake.io/api/v4/projects/#{ENV['AIRBRAKE_PROJECT_ID']}/sourcemaps",
          {
            multipart: true,
            file: File.new(file),
            name: https://cdn.summitlearning.org/javascripts/19-7e4ad96c.js.map,
          },
          {
            Authorization: "Bearer #{ENV['AIRBRAKE_API_KEY']}",
          }
        )

So the https://cdn.summitlearning.org/javascripts/19-7e4ad96c.js.map is actually invalid (because I'm not uploading the source map to s3), it's just the 'name' that I'm using to upload the file to airbrake directly, and also the name that I use in the airbrake context to directly associated the .js file with its corresponding .map file.
I think it's returning a 403 instead of a 404 because without the correct permissions requests to any sort of url with the cdn.summitlearning.org is erroring out with a 403. Is the 403 the reason why airbrake is not falling back to look for the private uploaded sourcemaps? If the request returned a 404 instead (e.g. if the hostname was localhost:3000) would it work?
Just trying to reason through why the private source maps are not being found. I already know that this URL (https://cdn.summitlearning.org/javascripts/19-7e4ad96c.js.map) does not / should not exist.

Is the 403 the reason why airbrake is not falling back to look for the private uploaded sourcemaps? If the request returned a 404 instead (e.g. if the hostname was localhost:3000) would it work?

Before doing any HTTP requests Airbrake checks if there is a file with such name in the private sourcemaps database. Only if such file is not found in local db Airbrake will attempt to make a HTTP request. So this should work if the sourcemap was successfully uploaded.
You can check what sourcemaps you have using following API:
GET https://getexceptional.airbrake.io/api/v4/projects/AIRBRAKE_PROJECT_ID/sourcemaps?key=AIRBRAKE_API_KEY


⚠ Bofrost.be has degraded performance

In 01a84b0, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 3603 ms

Resolved: Bofrost.be performance has improved in 73af860.


Out-of-source build is broken

When you try to configure out-of-source, you get:
$ $srcdir/autogen.sh
python3: can't open file 'autogen.sh/data/oic/oicgen.py': [Errno 2] No such file or directory

This oic generator being called out of the build system itself (namely being called from autogen.sh) seems not the right thing, the new build system does that - if we want to fix it before kconfig gets integrated we should move the oic generator to inside the build system itself.

Autogen.sh is to be executed in tree. Not just for oicgen, but autoconf as well. Some sources are generated and should be distributed in the "make dist". As well as Makefile.in
What you can execute out of tree is configure.

Yep, I know, what I mean is that the oic generator should be called from Makefile (like we for for all the other generators) and not Autogen.sh (the way it's today).

Can we close this now that kbuild got in?


If consumer and producer agree on the content type of a message coercion should be automatic

If a consumer and producer don't specify a content type (via spring.cloud.stream.bindings.*.contentType) then as long as they share a class definition for the payload (a domain object) then the message will flow. That's uncomfortable for 2 reasons: first it forces  them to share the domain class (or have an identical class with the same FQCN), and second it uses a default content type which is not interopable (Java serialization basically, not JSON or protobuf or whatever).
As soon as you specify the content type (e.g. as application/json for interop / human readable messages), the system breaks because JSON gets converted to a String and the receiver can't process it as a domain object any more (he has to manually convert the String to a domain object, going through 2 layers of conversion, from String to JSON to Object).
I propose that if the consumer and producer agree on the content type (e.g. application/json) then Spring Cloud Stream (or Spring Integration) should be able to handle the conversion and type coercion.
Marius has pointed out that you have a similar situation in REST endpoints where the solution is HttpMessageConverters that are mime-type as well as Java type aware. The payload is essentially bytes (or Strings) as far as the "dumb pipes" model is concerned (Spring Cloud Stream), but it carries content type headers, a bit like an HTTP request.

+1 for interop out of the box and not promoting shared classes (if supported at all, that should at least not be the simplest thing to do)

If a consumer does expect a JSON String, then a Converter with target type String should be registered for the application/json MIME type.

The problem is the other way around - the incoming data is String and it needs conversion to Vote in the downstream handler method. One way to solve it is to add contentType=application/x-java-object;type=demo.Vote but that feels a bit iffy. I guess we could register a Converter<String,Object> that assumes JSON, but then again, that feels a bit too opinionated without further consideration (it would assume that any String is JSON, which is maybe ok, if it wasn't for XML). So perhaps a solution that combines OOTB SI conversion with the presence of the original type (application/json) in the message header would be an option. Or perhaps we could have something like @EnableJsonConversion. I think that we need to explore the option space and try to figure out what's best there.

Not saying this is how spring-cloud-stream works currently but how it should - to see if we're on the same page... Let's just take the actual String as inbound payload case with 3 examples: JSON text, XML text, or plain ole text. Those inbound messages should carry contentType headers with MIME types application/json, application/xml (or text/xml), and text/plain respectively. The consumer binding properties may explicitly provide a target type, but if not, standard SI conversion should apply when the method-to-be-invoked has been determined. Does that make sense?

Those inbound messages should carry contentType headers

What exactly do you mean? I think we're back to my point that this could work with adapters where the concept of mime type applies, but not for others.
i.e.
 http | log
^    ^
1    2

At point 1, there is intrinsic content type by virtue of the Content-Type http header, and at point 2, this could be overridden by --outputType=xxx
But for the following case, this falls short:
tcp | log

So, maybe we should allow (an equivalent) of --inputType=xxx for a source?

Also, I think there is something we're missing when saying

Let's just take the actual String as inbound payload case with 3 examples: JSON text, XML text, or plain ole text.

There is no String as payload. There are only "bytes". ie [3c 3f 78 6d 6c 0a] with text/plain could be seen as a String, but with application/octet-stream it's bytes, with text/xml it could be the start of an xml document, etc. My point being: when it's on the wire, it does not have a type per se. It is given a type by

an accompanying "content type"  header (e.g. the Content-Type http header, external)
a header s-c-s sets, which may or may not be understood by the middleware the message flows in

Yes, good points. We should be clear in the distinction between cases where the "other side" of the producer->consumer relationship is a spring-cloud-stream module vs. when it's not (e.g. http or tcp). In the former case, the actual bytes on the wire might be sourced from a String via UTF-8 encoding, and in that case those details should be reflected in the headers of the Message. However, as you point out, part of the clarification of the role for MIME types includes mapping to/from other protocols (something SI can handle, e.g. for HTTP or RabbitMQ) as well as accepting raw messages with no context.

I think that the context of this issue is strictly producer->consumer, where it seems that we could do the type conversion in the handler based on the original content type of the inbound message, because we have enough information carried around (original content type, that is). Currently that is not supported, because we do not act on it outside the binder. While building on top of SI's conversion support, this is the kind of OOTB configuration that SCS could provide.
We could solve this problem by doing the coercion directly in the binder (by simply setting an appropriate input.contentType), but given the context above, it seems like we could do something cleverer.
Surely, we could apply the same concepts to other situations, where content type is available (like HTTP), and assume a default content type if no header is present (like FTP).

Then for modules such as FTP, the content type could be exposed as a configurable property (with a documented default)?

Sorry, I meant TCP not FTP, but yes.

Ping. Let's resurrect this one now that s-c-stream is nearing 1.0. I believe we need to do a better job here. Here's a summary of the current situation (correct me if I'm wrong):
On the producer I can put spring.cloud.stream.bindings.output.content-type=application/json and the message payload will be a JSON String (technically a byte array I know, but the conversion from a POJO to JSON is done by Spring and a header is attached to say what it did). This is good, and fine.
On the consumer, the JSON is not unpacked into an ojbect because there is no information in the headers to say what kind of object it is. This is good because we don't want brittleness and poor inerop caused by shared types on producer and consumer. So it's passed on downstream through the s-c-stream chain as a String (I don't know why it picks String, but that's the observation I made).
The message reaches a handler like this one:
@ServiceActivator(inputChannel = Sink.INPUT)
public void accept(Vote vote) {
...
}

So the application developer has bound the "input" channel to a strongly typed method. At this point the stars should align and something in Spring Cloud Stream should be able to figure out that the message can easily be converted to a Vote because it is plainly JSON (judging by the message headers), the target handler wants a POJO, and we have an ObjectMapper available.
Unfortunately, what happens is that the handler blows up saying
...
Caused by: org.springframework.expression.spel.SpelEvaluationException: EL1004E:(pos 8): Method call: Method accept(java.lang.String) cannot be found on demo.CounterApplication$$EnhancerBySpringCGLIB$$eaa35de6 type
	at org.springframework.expression.spel.ast.MethodReference.findAccessorForMethod(MethodReference.java:211) ~[spring-expression-4.2.4.RELEASE.jar:4.2.4.RELEASE]
	at org.springframework.expression.spel.ast.MethodReference.getValueInternal(MethodReference.java:125) ~[spring-expression-4.2.4.RELEASE.jar:4.2.4.RELEASE]
	at org.springframework.expression.spel.ast.MethodReference.access$000(MethodReference.java:49) ~[spring-expression-4.2.4.RELEASE.jar:4.2.4.RELEASE]
	at org.springframework.expression.spel.ast.MethodReference$MethodValueRef.getValue(MethodReference.java:347) ~[spring-expression-4.2.4.RELEASE.jar:4.2.4.RELEASE]
	at org.springframework.expression.spel.ast.CompoundExpression.getValueInternal(CompoundExpression.java:88) ~[spring-expression-4.2.4.RELEASE.jar:4.2.4.RELEASE]
	at org.springframework.expression.spel.ast.SpelNodeImpl.getTypedValue(SpelNodeImpl.java:131) ~[spring-expression-4.2.4.RELEASE.jar:4.2.4.RELEASE]
	at org.springframework.expression.spel.standard.SpelExpression.getValue(SpelExpression.java:330) ~[spring-expression-4.2.4.RELEASE.jar:4.2.4.RELEASE]
	at org.springframework.integration.util.AbstractExpressionEvaluator.evaluateExpression(AbstractExpressionEvaluator.java:166) ~[spring-integration-core-4.2.4.RELEASE.jar:na]
	at org.springframework.integration.util.MessagingMethodInvokerHelper.processInternal(MessagingMethodInvokerHelper.java:317) ~[spring-integration-core-4.2.4.RELEASE.jar:na]
...

You can fix it by declaring the content type differently to the default content type for the input channel:
spring.cloud.stream.bindings.intput.content-type=application/x-java-object;type=demo.Vote

This is mechanical (boiler plate) for the application developer. In an ideal world it should be possible to make this behaviour the default. It is also a little strange to put application/json in one end of the pipe and see it apparently come out as something completely different, viz application-x-java-object. So in the ideal world we would go a step further and make the declared content-type a conversion hint for data on the wire (so application/json on both sides would be redundant, but everything would work).
Do we have a proposal for a solution yet?

Even worse (if you ask me): if the user doesn't supply any content type on either side and there happens to be a class called demo.Vote on both sides, then everything will work, but only if the two classes have identical declarations. This out-of-the-box easy path serves to encourage the bad practice sharing classes between producer and consumer, and gives users a false sense of security in demo and getting started, plus it is terrible for interop because the payloads are transported as Serializable bytes by default.

In most cases, I think the problem is setting the contentType to text/plain instead of application/json for the JSON String during the serialization process at the producer. If the payload is of type String then the contentType is automatically set to use text/plain. I believe this problem can be fixed by setting the contentType that is set in the bindingProperties. This will make sure the expected contentType (application/json for example) is set when the deserialization process takes place at the consumer.
I am trying out some changes to propose an idea and will share it once ready.

This [change] (https://github.com/ilayaperumalg/spring-cloud-stream/commit/cee87ffb1cace5ebcdd60ea1cb6278b572f24025) tries to address one of the issues where the contentType is always set to text/plain even for the application/json contentType used at the producer. I guess the other thing to address is to deduce the expected type from the inbound adapter handler and perform automatic type conversion instead of doing
spring.cloud.stream.bindings.intput.content-type=application/x-java-object;type=demo.Vote

So there's a slightly orthogonal issue that the content type is ignored on the consumer. Should be easy to fix, but it won' solve the overall problem (if I read your comments correctly)? I looked at the code that does the conversion a bit because of an unrelated issue with spring boot devtools. I can see that org.springframework.cloud.stream.binder.MessageChannelBinderSupport.deserializePayload(byte[], MimeType) is the place where the initial deserialization happens. This is a couple of hops upstream of the @ServiceActivator but the information has to reach it somehow, so I guess it makes more sense for it to convert back to JSON, and then let something downstream deal with the JSON->POJO conversion? Would Spring Integration do that for us out of the box, or do we need to fix the dispatcher somehow? Could it be handled a layer above the default SpEL handler (so that faster, non-reflective handlers can be substituted and the same flow works)?

Making serialization the default strategy (i.e. absent any content type information) for transporting data is a legacy of Spring XD. Haven't looked at @ilayaperumalg 's change in detail, but in theory we could replace the default marshalling with a more portable format (e.g. JSON).
The other issue though is that the dispatching process in Spring Integration is based purely on type information (so indeed a String cannot be converted by default to a Vote). The question here is how easy it is to swap (or augment) the current method invocation strategy so that the passed arguments can be populated based on information found in the entire message (i.e. the String payload + contentType headers). I think we need some guidance from @garyrussell here.

I don't object (much) to putting contentType=application/json in my config. So if we could persuade Spring Integration that it knows how to convert JSON to Vote we are winning?

Seems there are hooks for it - http://docs.spring.io/spring-xd/docs/current/reference/html/#mime-types-and-java-types
It this issue only wrt to json, or do we have something to address as well if we want to send protocol buffer payloads across the wire.  That was the original motivation to add custom converters here - http://docs.spring.io/spring-xd/docs/current/reference/html/#Providing-a-New-Type-Converter

IMO it shouldn't matter what the actual content type is as long as there is a sensible strategy for conversion from/to the POJOs in the handlers. It should be up to the user what content type is transported in the bytes over the wire, and he should have a reasonable expectation that it will be unpacked to a supported type in the consumer without doing anything (beyond maybe including the right converter on the classpath). I.e. I don't think there is anything JSON specific.

This change provides one possible approach to specify the input message type programmatically (as the developer would know what the input type would be when programming).
While both commits on the branch depicts the underlying issue and rough approach to handle this, I will write a doc that can be discussed for designing a better solution.

Hi guys
Maybe is not the good thread but I think is related to.
For the StreamListener annotation worked the case Json->Pojo, but for @Filter annotation on a method for a input channel that have content-type: application/json the transform is failing from JSON to POJO. If the producer have the binding content-type: application/json and the consumer have binding content-type: application/x-java-object;type=x.x.POJO then is working.
 Any thoughts related to are much appreciated.
Thank you

@Filter is not a Spring Cloud Stream annotation. It does not support content negotiation.

The @Filter and other Messaging Annotations configuration, or even better - any POJO method invocation, will support content negotiation since Spring Integration 5.0: https://jira.spring.io/browse/INT-4225

We are producing AVRO message over Kafka from Spring Boot and trying to consume it with a NodeJS application, we find the AVRO data contains x-java-object as content type and isn't really a valid JSON for NodeJS for parse.
anyone facing similar problem?

@hamzaislam When producing messages into non Spring Cloud Stream applications,  you would need to set spring.cloud.stream.bindings.<channelName>.producer.headerMode=raw. You can find documentation for this under ProducerProperties headerMode.

@hamzaislam Also, it is recommended to create a new Github question/issue or ask the question on StackOverflow here as this current Github issue is already closed.


Updated packages. Added dependabot and gitlab workflow

I needed to update esbuild to be able to use ts-eager on M1.
Also added, to make updates like this less painful in the future:

dependabot.
github workflow with test and linter for pull requests.

ignore the massive amount of commits. Sorry.

@mhart Could you please look into this, also update esbuild and typescript to latest version?
Thanks a lot for this awesome library ✌

Any news on this?


ADM-880: [frontend] feat: generate chart - board metrics

Summary
generate chart - board metrics
Before
Description
Screenshots
If applicable, add screenshots to help explain behavior of your code.
After
Description
Screenshots
If applicable, add screenshots to help explain behavior of your code.
Note
Null

 Is this confirmed with @Yufan0226 ?

This is a bug just introduced, I will fix. thx.


Doesn't Apear chained_field value

Actual behavior
1- I Install Djnago Smart Selects by this commend pip install django-smart-selects
2- Added in settings => installed_apps + Add JQuery Permission line :
INSTALLED_APPS = [     'django.contrib.admin',     'django.contrib.auth',     'django.contrib.contenttypes',     'django.contrib.sessions',     'django.contrib.messages',     'django.contrib.staticfiles',     'MyApp',     'smart_selects', ] USE_DJANGO_JQUERY = True
3-  models.py  :
`class Topic(models.Model):
user = models.ForeignKey(CustomUser, on_delete=models.CASCADE)
title = models.CharField(max_length=400)
image = models.FileField(blank=True, null=True)
created_date = models.DateTimeField(auto_now=True)
class Reply(models.Model):
user = models.ForeignKey(CustomUser, on_delete=models.CASCADE)
topic = models.ForeignKey(Topic, on_delete=models.CASCADE)
reply = models.TextField()
created_date = models.DateTimeField(auto_now=True)
class Likes(models.Model):
user = models.ForeignKey(CustomUser, on_delete=models.CASCADE, null=True)
topic = models.ForeignKey(Topic, on_delete=models.CASCADE, null=True)
reply = ChainedForeignKey(
Reply,
chained_field="topic",
chained_model_field="topic",
show_all=False,
auto_choose=True)`
4- admin.py :
class LikesAdmin(admin.ModelAdmin): admin.site.register(models.Likes, LikesAdmin)
5- urls :
urlpatterns = [     url(r'^admin/', admin.site.urls),     url(r'^my-app/', include('MyApp.urls')),     url(r'^chaining/', include('smart_selects.urls')), ]
Result : Reply's field keep empty after selecting Topic field
Not working either in admin panel and templates
Did I Miss Any thing  ?

Any Reply ?

Can you please check if you have some error in the browser's console?

@GrilloLuca Yes, There is

bindfields.js:12

Uncaught ReferenceError: chainedfk is not defined
at initItem (bindfields.js:12)
at HTMLSelectElement. (bindfields.js:31)
at Function.each (jquery.min.js:2)
at HTMLDocument. (bindfields.js:30)
at i (jquery.min.js:2)
at Object.fireWith [as resolveWith] (jquery.min.js:2)
at Function.ready (jquery.min.js:2)
at HTMLDocument.J (jquery.min.js:2)

check this issue: https://github.com/digi604/django-smart-selects/issues/253
Solution: RafaDias/django-smart-selects@61f182f

That worked
I Hope Disruption Update as soon as possible
thx @GrilloLuca


[Question] Dictionary key transformation

Is there any way to transform dictionary key ? I need to transform JSON from [String: AnyObject] to [NSDate: AnyObject]

You should be able to create a custom transform to achieve this. Or alternatively you can do some post processing after you map to [String: AnyObject].
See the following transform as a starting place: https://github.com/Hearst-DD/ObjectMapper/blob/master/ObjectMapper/Transforms/DictionaryTransform.swift
Hope this helps


Set rule precedence

I have a scenario where it would be useful if possible to chain/set precedence on two rules.  For example, I am monitoring our load balancer logs to see if any of the "real" hosts that make up a load balanced address go down:
Rule a:
type: any
realert: minutes: 0
filter: query: query_string: query: query: '(msg:"ldb server up" AND vip:"www.fake.com http" AND server:"<IP_ADDRESS>")
command: notify_nagios
Rule b:
type: any
realert: minutes: 0
filter: query: query_string: query: query: '(msg:"ldb server down" AND vip:"www.fake.com http" AND server:"<IP_ADDRESS>")
command: notify_nagios
These are checking every minute and usually it works happily, until one of the "real" hosts is  rebooted and comes back up within a minute.  Then what happens is that both rules alert Nagios, and depending on the order the rules happen to run Nagios either shows the server as up (desired result) or it stays down (not so desired result).
Is there any way we can set a priority in the rules, so that the Up rule always runs AFTER the Down rule?

I wonder if alerts are run in parallel or if they are fired one at a time in alphabetical order (by filename).


#805 - Correct Canada Url in Template History

Description

Updating an Canada related URLs in the Template or TemplateHistory content attribute to our va.gov URL
Created downgrade as well

issue #805
How Has This Been Tested?

Deployed to Dev
Ran Downgrade
Note {{notification-api-url}}/service/{{service-id}}/template/{{template-id}}/version/1 uses TemplateHistory model. Used to route to test the update to TemplateHistory content attribute
Before new migration
Template

Template History

After new migration, Confirm URL updated
Template

Template History

After new downgrade, Confirm URL reverted
Template

Template History

Checklist

[x] I have assigned myself to this PR
[x] PR has an appropriate title: #9999 - What the thing does
[x] PR has a detailed description, including links to specific documentation
[x] I have added the appropriate labels to the PR.
[x] I did not remove any parts of the template, such as checkboxes even if they are not used
[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[n/a] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to any documentation
[x] My changes generate no new warnings
[n/a] I have added tests that prove my fix is effective or that my feature works. Testing guidelines
[x] I have ensured the latest main is merged into my branch and all checks are green prior to review
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules
[x] The ticket was moved into the DEV test column when I began testing this change

This has never been used. Should be okay to remove


How to export point cloud like in hf demo with inferred depths

Hi, thanks for your interests! Please refer to the following code to transfer a depth map to the point cloud:
https://huggingface.co/spaces/depth-anything/PromptDA/blob/main/app.py#L111-L120

Thanks for your reply. Can I use the odometry translation and quternion to merge all point clouds and do global reconstruction. Or is it better to do rgbd slam with the rgb and depth images?

Yeah, it’s possible to use visual odometry to merge point clouds as our prompting. This requires training a new model.
If you just want a global reconstruction with our iPhone model, you can take the camera poses from ARKit, which are provided by the Stray Scanner app.


[amazon-corretto] New EOL dates for Corretto 8 and 11

Amazon announced an Extension of EOL Dates for Amazon Corretto 8 and 11
https://aws.amazon.com/about-aws/whats-new/2024/10/extension-eol-dates-amazon-corretto-8-11/

@bootswithdefer thanks for the pr, gonna close in favor of #6000. 🙏


Neopixel and another modules

Hello, first: you are the best of all.
I want to know the modules in the esp32, because I try to use neopixel but I do not have the module installed, I try to create the module by copying the "Neopixel" code manually to be able to use it, but again it shows me the error "no module named board" (for example) , in the same way other modules do not exist in this project or I assume they are named differently, can you help me?
Thanks so much for your big work

Hi,
If you want to learn all about MicroPython on an esp32 development board, I suggest you do it on a standard esp32 board, flashing it with the generic esp32 firmware from the micropython.org site.
The firmwares provided here have been customised to include the camera driver. Few "standard" modules were not included to save space or because they were not considered necessary for a camera-based application.
Now you can import any python module you like. For neopixel you can try neopixel.
Upload "neopixel.py" to your board and do the following:
from machine import Pin
from neopixel import NeoPixel

neo = NeoPixel(Pin(4), 15) # on GPIO4 and 15LEDs
# and the rest!


Fix occt 7_8_0 module logic and update OCCT in CI

Fix incorrect OCCT module logic introduced in https://github.com/f3d-app/f3d/pull/1306
Update OCCT to V7_8_0 in CI

To backport to release branch: 536e00025df0827cd757e54d3aab0eb28ed95abb

@thierry-FreeBSD please take a look :)

@thierry-FreeBSD please take a look :)

Seems good.
Note: meanwhile a maintenance release of OCCT is available: https://github.com/Open-Cascade-SAS/OCCT/releases/tag/V7_8_1 .
I have not yet checked it, but it should not bring heavy changes.


Battlescape initial work, a display of a static map (58UFO8)

read map and tile information from 58UFO8 folder
display static battlescape map
buttons working: layer buttons (up/down, numbers), layer mode (up to current, all layers, only current), strategic view
available via ingame menu's button

Thanks once again.
I cleaned up a few unnecessary files, and added the new stuff to the linux CMakeLists.txt too


Correct "SSJ" to "SSH"

Corrected misspelling of SSH in the Caution note under CLI section.

@asudbring
Can you review the proposed changes?
Important: When the changes are ready for publication, adding a #sign-off comment is the best way to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off


When using google as provider: $provider->getUserDetails($token) throws Unknown Error

Facebook works fine but same code while Googel as provider is throwing  Ohh dear users.login.google Exception: Unknown Error
here is what I am trying to do:
$provider = new League\OAuth2\Client\Provider\Google(array(
			'clientId' => Config::get('sso.googleClientID'),
			'clientSecret' => Config::get('sso.googleClientSecret'),
			'redirectUri' => URL::to('/') . Config::get('sso.googleClientRedirect'),
			'scopes' => array('email','profile'),
			));

		$code = Input::get('code');
if (!isset($code)) {
			// If we don't have an authorization code then get one
			header('Location: ' . $provider->getAuthorizationUrl());
			exit;
		} else {
			// Try to get an access token (using the authorization code grant)
			$token = $provider->getAccessToken('authorization_code', [
				'code' => $code
				]);

			// Optional: Now you have a token you can look up a users profile data
			try {

				// We got an access token, let's now get the user's details

				/*
					Error: Ohh dear users.login.google Exception: Unknown Error.
				*/

				$userInfo = $provider->getUserDetails($token);
				//generate user
				$this->generateSocialUser($token, $userInfo,'google');
			} catch (Exception $e) {
				// Failed to get user details
				echo 'Ohh dear users.login.google ' . $e;
				exit;
			}

		}

same code works fine when login using facebook.
Thank you.

This changes fixed the issue: in /provider/google.php
public function urlUserDetails(\League\OAuth2\Client\Token\AccessToken $token)
   {
        return
            'https://www.googleapis.com/oauth2/v1/userinfo?alt=json&alt=json&access_token='.$token;
   }

public function userDetails($response, \League\OAuth2\Client\Token\AccessToken $token)
    {
        $response = (array) $response;

        $user = new User();
        $user->exchangeArray([
            'uid' => $response['id'],
            'email' => $response['email'],
            'verified' => $response['verified_email'],
            'name' => $response['name'],
            'firstname' => $response['given_name'],
            'lastName' => $response['family_name'],
            'urls' => $response['link'],
            'imageUrl' => $response['picture'],
            'gender' => $response['gender']
        ]);

        return $user;
    }

but how can it be resolved by default?

The current implementation works, but you have to enable Google+ API in the developer console first.

Thanks man. Which api to use google plus or otherone ? The google plus api has limit on number of requests.

@ideadx, Did enabling the Google+ API fix this issue for you?

@ramsey yes. enabling Google+ API fixes this but it has a limit on number of calls.

Thanks. For v1, we'll be splitting out the Google provider into its own package, and those who maintain it will be able to determine how to move forward with it. In particular, I've heard that Google has announced that they will be retiring Google+, so I'm not sure what that means for the Google+ API. We'll need to figure that out, once this is split out to it's own project.