copy short url

PR-Codex overview
The focus of this PR is to add the jekyll-shorts gem, implement short URLs with a copy button, and update the configuration for jekyll-shorts.
Detailed summary

Added gem "jekyll-shorts" to Gemfile
Updated configuration in _config.yml for jekyll-shorts
Implemented short URLs with a copy button in _layouts/post.html

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@rultor merge

@rultor merge

@h1alexbel OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@h1alexbel Done! FYI, the full log is here (took me 10min)


chore: update gcloud version

Updating to get the pub/sub changes. You can find the SHA here https://cloud.google.com/sdk/docs/install-sdk#linux for future updates.

This change is

That emulator log output looks different to how it was before, making me think the output may not be being flushed in the same way, which means we never get to witness the "Server address" line before we time out.

Current output:
I0000 00:00:1710352531.742451  242820 emulator_main.cc:40] Server address: localhost:8787

Old output:
Step #5:     17	2024-03-11 21:49:57  INFO  emulator_main.cc : 40 : Server address: localhost:8789

We believe the culprit is in google/cloud/spanner/ci/lib/spanner_emulator.sh
# Outputs the port number that the emulator chose to listen on.
function spanner_emulator::internal::read_emulator_port() {
  local -r logfile="$1"
  shift

  local emulator_port="0"
  local -r expected=" Server address: "
  for _ in $(seq 1 8); do
    if grep -q -s "${expected}" "${logfile}"; then
      # The port number is whatever is after the last ':'.
      emulator_port=$(grep "${expected}" "${logfile}" | awk -F: '{print $NF}')
      break
    fi
    sleep 1
  done
  echo "${emulator_port}"
}

I think it is specifically this line:
https://github.com/googleapis/google-cloud-cpp/blob/26b80a8c2650b7fca451406f54474da19618effa/google/cloud/spanner/ci/lib/spanner_emulator.sh#L36C23-L36C36
The : before Server address: is now gone.  I would try changing that line to:
  local -r expected=" Server address: "

Yes we agree @coryan. Running that now.


Start Using Graylog

Start using the Graylog to remove complexity from data exploration, compliance audits, and threat hunting.

This issue not covered in v5.0. To be analyzed

@ChakshuGautam @vijayatgh @htvenkatesh - please add your thoughts here.

Currently we are not using logs , we need to discuss on this

Added to Backlog


Access original source hash in a Dash object

Is this possible?
I have a created a dash class like so:
class Dash::Offer < Hashie::Dash
  include Hashie::Extensions::Dash::PropertyTranslation

  ############################################################
  # Properties
  ############################################################
  # Basic info
  property :id, required: true
  property :created_date, from: :timestamp, transform_with: proc { |value| value.to_datetime }
  property :status, transform_with: proc { |value| value.to_sym }
end

And I initialize an object like so:
hash = {
    id: "1234",
    timestamp:<PHONE_NUMBER>32,
    status: "active",
    unknown_property: "This is a property not accessible in the Dash object"
}
@offer = Dash::Offer.new(hash)

And now I'd like to access the property "unknown_property". Is there any way I can access the original source hash from the Dash object?
Something like this:
@offer.source_hash["unknown_property"]

Well, the whole point of a Dash that you can only access the declared properties :)
Looking at https://github.com/intridea/hashie/blob/master/lib/hashie/dash.rb#L118 it doesn't feel like you can reach into the underlying Hash's [], but I could be wrong.

@dblock Could I create a property on the dash class called "source", which took a hash as its value? And that way still have access to the original hash?

You could, but for starters Dash::Offer.new(hash) is failing with NoMethodError: The property 'unknown_property' is not defined for Offer., so this shouldn't even be possible.

Sure... but if I did something like this, it should work?
class Dash::Offer < Hashie::Dash
  include Hashie::Extensions::Dash::PropertyTranslation

  ############################################################
  # Properties
  ############################################################
  # Basic info
  property :id, required: true
  property :created_date, from: :timestamp, transform_with: proc { |value| value.to_datetime }
  property :status, transform_with: proc { |value| value.to_sym }
  property :source_hash
end

hash = {
    id: "1234",
    timestamp:<PHONE_NUMBER>32,
    status: "active",
    unknown_property: "This is a property not accessible in the Dash object"
}
@offer = Dash::Offer.new(hash.merge({ source_hash: hash }))

p @offer.source_hash["unknown_property"] // "This is a property not accessible in the Dash object"

The above works fine and is simple enough. Not sure why I didn't just do that from the start.
I'm using Dash to clean up the data from my database, for when I manipulate the data on my API. So sometimes I need access to the "original" data that I retrieve from my database.
Thanks for clearing things up for me.

Other than the fact that it's not very efficient I don't see any problems!


feat: Implements type inferrence for route params in V4

Mostly non breaking
Description
Created a new proposal for Itty Router v4 to support route param type inferrence. The only breaking change would be for users who are passing generics to their route handlers I believe (ie: Router().get<>) because the path now has to extends string for type inference to work.
Feedback welcome of course
Related Issue
https://github.com/kwhitley/itty-router/pull/150
Type of Change (select one and follow subtasks)

[ ] Documentation (README.md)
[ ] Maintenance or repo-level work (e.g. linting, build, tests, refactoring, etc.)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)

[ ] I have included test coverage

[x] Breaking change (fix or feature that would cause existing functionality/userland code to not work as expected)

[x] Explain why a breaking change is necessary: Type inference would never work without generic string params

[ ] This change requires (or is) a documentation update

[ ] I have added necessary local documentation (if appropriate)
[ ] I have added necessary itty.dev documentation (if appropriate)

I also saw yarn.lock was updated - but also I saw their are 2 conflicting lockfiles - npm and yarn - wasn't sure which one was the right one - might be worth setting a packageManager in the package.json

So I love trying to get inference on route params, but not at the price of a breaking change.  v4.x is new enough that yet another [breaking] change in the TS signatures this soon would probably be a sign of general instability in the platform.

So unless we can get this in without impacting (in a breaking manner) the existing types, I'm leaning towards closing this for now... 😞

Closing this for now... there are more heavily typed approaches to solving this, but itty aims to be the lightweight version of all the heavy approaches.


move list validation to produce an associated type

Change Summary
Heading towards https://github.com/pydantic/jiter/pull/63 still...
The goal here is to be able to split coupling between the Input trait definition and the actual types which can get produced by certain "validate" operations. This gives more flexibility in the lifetimes, because we can completely separate the Python and JSON types. I hope it might also give perf boosts by allowing code to optimize solely for the relevant types at hand.
The experiment is to do this just for validate_list first; if this works I'll do it for all uses of GenericIterable and GenericArgs.
Related issue number

Checklist

[ ] Unit tests for the changes exist
[ ] Documentation reflects the changes where applicable
[ ] Pydantic tests pass with this pydantic-core (except for expected changes)
[ ] My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Ok, so the alternative to this to support #1231 would be to make the Input trait have two lifetimes, Input<'data, 'py> and change GenericIterable to have an extra lifetime too, GenericIterable<'a, 'data, 'py>. That will pollute a lot of code with an additional lifetime which is mostly irrelevant.
I think having the localized complexity of this PR is better than polluting the whole codebase with lots of arbitrary lifetimes. So I'm going to merge this PR and then proceed to convert other relevant Input types to work in the same way.


Examine version needed to extract

Whilst reading the EPUB specification, I noticed this text in Section 4.2:

In the local file header table, OCF ZIP Containers MUST set the version needed to extract fields to the values 10, 20 or 45 in order to match the maximum version level needed by the given file (e.g., 20 if Deflate is needed, 45 if ZIP64 is needed). OCF Processors MUST treat any other values as being in error.

Currently, the library will set it to 20 when using stored or deflate compression, and 46 when using bzip2. We might want to specify 10 for using stored compression, and examine if the value is correct in other cases as well.
Also, if and when we will support writing ZIP64, we should also increase this number to at least 45 when it is used.

While reading and writing are we making sure the version information all checks out? I'd love to evaluate all the version identifiers given by the APPNOTE

Fixed in 2.1.1.


Error: System.InvalidCastException: Unable to cast object of type 'Microsoft.Xrm.Sdk.Messages.DeleteRequest' to type 'Microsoft.Xrm.Sdk.Messages.UpsertRequest'.

Hi all,
Since a few days now, we have been having issues with the pac powerpages upload command. Quite late in the process, the following error occurs:
2024-05-15 09:03:23.3524 | ERR | bolt.module.paportal.PortalDataImporter :: PAPortalUploadAction(): Failed
2024-05-15 09:03:23.4149 | ERR | bolt.module.paportal.PortalDataImporter :: System.InvalidCastException: Unable to cast object of type 'Microsoft.Xrm.Sdk.Messages.DeleteRequest' to type 'Microsoft.Xrm.Sdk.Messages.UpsertRequest'.
   at bolt.module.paportal.PortalDataImporter.<>c__DisplayClass31_0.<ExecuteRequestsAndHandleResponse>b__0(OrganizationRequestCollection requests)
   at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0`1.<ForWorker>b__1()
   at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
   at System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object <p0>)

I am attaching the log file, hopefully that helps.
pac-log.txt
I do not recall anything being changed on our end that could cause this. Also, we see this happening for multiple projects.
Looking forward to your replies! If more info is required, feel free to reach out.

Update: Fixed by downgrading to an earlier version (1.31.6 in my case).


Refactoring for Consistency

Is this pull request associated with an issue(s)?
No.
Description
This PR does some miscellaneous refactoring to make the codebase more consistent. Depends on #79 to make sure the documentation is still consistent after that PR's additions.
Note: This PR is only bumping the minor version, even though it contains API-breaking changes (moving assert.cmake), since we are not going to bump to v1.0.0 until the official release.
TODOs

[x] Make files named the same as their subdirectory (like map/map.cmake) primarily include other files containing public API functions instead of containing the public API code itself.
[x] Move utilities/assert.cmake to asserts/assert.cmake.
[x] Ensure that the documentation is consistent after #79.
[x] Check that CMaize unit tests and integration tests still pass.
[x] Check that CMakeTest unit tests still pass.

I verified that locally CMaize and CMakeTest still pass their unit tests with these changes.


OSD-7084 alert when cert will expire

this enables metric scraping by prometheus and includes updates to the local testing tooling

/label tide/merge-method-squash

/lgtm


It should be converted to number first before sort

Hi. Ashish. Your plugin is really nice. Thanks a lot. I'm using this plugin and found one bug. When I set thresholds with various values, it just didn't recognize well. Please see my pull request.

Thanks @donghopark for using it. I am glad that you found it useful.
I saw your modifications in the pull request. But can you please explain and write some cases which you tried so that i can reproduce the issue at my side?
Thanks
Ashish

You may try with below configuration
$scope.gaugeThreshold = {
'0': {color: '#228B22'},
'8': {color: "#AABC22"},
'21': {color: '#FFF54C'},
'31': {color: '#DF9111'},
'41': {color: '#FF0000'},
}
$scope.value = 22;
It should be third option to be selected, but once it go over 8, it always select the second option.

#46

Hi @donghopark
I tested your code, its working fine, with one simple change that you need to push in your pull request, so that build will pass.
change
.sort((a,b) => {Number(a) > Number(b)})

to
.sort(function(a,b) {return Number(a) > Number(b);}

Re-commited as you recommend


1INCH ticker is not recognized

Describe the bug
Bot does not recognize 1INCH ticker
To Reproduce
python3 pycryptobot.py --market 1INCHUSDT --verbose 1 --disablebullonly --disablebuynearhigh --disablebuyobv --disablebuyelderray --sellatresistan Invalid config.json: Binance market invalid: 1INCHUSDT
Expected behavior
Bot works as usual.
Desktop (please complete the following information):

OS: Debian 10
Version 2.1.0

Problem is related to this regex:
https://github.com/whittlem/pycryptobot/blob/61b1b216fd227c0a91571d75379ec0ac6fe6ec76/models/config/binance_parser.py#L8
It should be (currently no time to make a PR):
p = re.compile(r"^[0-9A-Z]{6,12}$")


finished account check function

finished account check function and add the function of httpapi, convenient to write testcase to call the main chain of some information.
And I have tested the httpapi function, it run well.
Usecase write in httpapi.README.md

I have cleaned up the code and unified log, and modified readme(en), and I tested startmonster, it worked well.


Update basic.vim

Add compatibility with terminal Vim

What's the use case for this key mapping?

Like @Arnie97, I am also unsure what the use-case is here. Note that you can include your keys and configs in the my_configs.vim Ref


Const generics have been added to stable

Const Generics have been added to stable rust since 1.51, so update the crate so it doesnt need nightly rust to build. Indicate a breaking change for people running older rustc

Published version 0.3.0. I will be working on updating downstream crates soonish, but currently working on https://github.com/vadixidav/hrc.


0.2.2 global log feature list

this is everything that wasn't finished in 0.2.0-0.2.1 (see #8)
Task TODO

[ ] Get Stack Trace as string method
[ ] Log to File Method
[ ] Log to File Settings (default to Always)

[ ] Log to File on Exit Notification implementation
[ ] Log to File interval implementation

[ ] add set-permission method for INFO logs
[ ] onExit prints all no-permission logging
[ ] tracking total logging calls
[ ] load user dict from res:// as json (once #12 is implemented)
[ ] support null as replacement for caller arg (add as default)
[ ] support for joining array log messages as an logtring
[ ] Asynchronous Logging Manager (logs send on idle console frames with timestamp)

no longer planned

[ ] Set Verbose Logging Permission (on a per script basis (uses self as key))

[ ] Errors always show, always push
[ ] Trace always show
[ ] Warn always show (or show with permission, exit log without?)
[ ] Info show with permission, exit log without

versioning update

add set-permission method for INFO logs > added permission logging for all

' Set Verbose Logging Permission' replaced with elevated logging behaviour in 94c3f75

e8a36e1 - 'Log to File interval implementation'
4692289 - completed 'Log to File Settings (default to Always)'

[ ] #15 moved to #17


Include webpack plugin

Versions:
angular-cli: 1.0.0-beta.15
node: 5.1.0
os: darwin x64
Is possible to install some plugin for webpack and include it?
One example is
// webpack.config.js
module: {
  loaders: [
    {
      test: /\.yaml$/,
      include: path.resolve('data'),
      loader: 'yaml',
    },
  ],
}

@daniele-zurico you might be interested in this issue https://github.com/angular/angular-cli/issues/1656

See https://github.com/angular/angular-cli/issues/1656.


[Feature Request]: Add Azure Bastion options for tunnelling

https://github.com/Azure/ResourceModules/blob/88ef83865142e28afa8937b35fcdedb414739111/modules/Microsoft.Network/bastionHosts/deploy.bicep#L178-L182
Azure Bastion deployed with standard SKU should have an option to enable tunneling, this is currently available on the current API version already in use by the module.

@MrMCake - Awaiting completion of #1778


Honor disallow in defer_js

See: https://github.com/pagespeed/mod_pagespeed/issues/1173

LGTM

So...  How do I indicate to mod_pagespeed that I don't want it to rewrite my javascript, but I do want it deferred?


How can I test trained model for my test images?

Thanks for your great work and codes.
I followed your commands:

I used pre-train phase.
used total-text dataset to improve training model.
test on total-text database to evaluate model.

now I want to evaluate model on my sample images. I run demo.py but it need mat file of my sample images, while I dont want to give .mat file. I want to test for images in which i don't have their gt files.
regards,John

Thanking for your attention to my work, pure inference needs dataset modification, which will be added to  this repo soon.

@johnSmith1990 pretrained models on my machine and pure inference script are released(see README.md) hope it can help you. I'll close this issue.


FreeRTOS timer callbacks occasionally executed twice (IDFGH-6590)

Environment

Development Kit: none
Kit version (for WroverKit/PicoKit/DevKitC): none
Module or chip used: ESP32-WROVER
IDF versions: v3.3, v4.4 (release branches as of today)
Build Systems: make for v3.3, idf.py for v4.4
Compiler versions: gcc version 5.2.0 (crosstool-ng-1.22.0-97-gc752ad5) for idf v3.3, gcc version 8.4.0 (crosstool-NG esp-2021r2) for idf v4.4
Operating System: Linux, macOS
Using an IDE?: No
Power Supply: USB

Problem Description
FreeRTOS single shot timer callbacks occasionally are called twice. This occurs especially with CPU/MEM load and Wifi network traffic, but it also has been seen with Wifi disabled running a PPP network via modem. The issue can be reproduced by building & launching the attached test projects and starting some flood pings to the module, see below for details.
Source project / reference:

https://github.com/openvehicles/Open-Vehicle-Monitoring-System-3/issues/666

Additional note: it's possible this isn't limited to single shot timers. I haven't tested periodic timers for this behaviour yet.
Expected Behavior
A single shot timer callback is never executed twice on timer expiry.
Actual Behavior
Single shot timer callbacks occasionally are executed twice on timer expiry.
Steps to reproduce

Build & run the test project(s)
Start some flood pings to the module
Monitor the output for "duplicate callback" messages

Code to reproduce this issue
See attached ZIP archive (contains two projects, one for idf 3.3 & one for 4.4):

test-timer-dupecall.zip

Debug Logs
See attached ZIP archive, files "testrun.log".
Example excerpt:
I (5486) app_task: running on core 1
I (15846) app_task: checkpoint tid=1000
E (17226) timer_callback: duplicate callback detected on tid=1126
E (22196) timer_callback: duplicate callback detected on tid=1595
I (26396) app_task: checkpoint tid=2000
E (26746) timer_callback: duplicate callback detected on tid=2032
E (28906) timer_callback: duplicate callback detected on tid=2238
E (30216) timer_callback: duplicate callback detected on tid=2359
…

Other items if possible
See attached ZIP archive for sdkconfig.

Additional info: the effect is also present with SPIRAM disabled (and 64K for cpuload_task to fit in internal RAM), and it's even more frequent when running in single core mode.
Regards,
Michael

The effect also affects periodic timers.
Extended test project including test results: test-timer-dupecall-2.zip
Detection method: there are 5 counting and 1 checkpoint timers, identified by their timer IDs. The counting timers are set to run every 1,2,3,4,5 ticks. The checkpoint timer is set to run every 120 ticks, it subtracts the expected counts and checks for remaining offsets.
Due to shuffling of the internal timer execution list, the counters can have a small offset on one checkpoint which will be equalized on the next. These get logged as warnings. If an offset appears to be stable, an error log line is written.
The checkpoint logs show the long term evolution of the offsets. Offsets should long term always return to zero if all timers are executed exactly as often as they should. If they execute more often than expected, offsets will accumulate to positive values, if less often to negative.
The test run shows these stabilized checkpoints:
I (125386) periodic_callback: checkpoint: offsets = 0, 0, 0, 0, 0
…
I (5525386) periodic_callback: checkpoint: offsets = 1, 0, 0, 0, 0
…
I (13325396) periodic_callback: checkpoint: offsets = 1, 0, 1, 0, 0
…
I (37325396) periodic_callback: checkpoint: offsets = 1, 0, 1, 1, 0
…

Conclusions:

Periodic timer callbacks are occasionally executed twice as well.
This happens with a much lower frequency than with the single shot callback dupes.

Regards,
Michael

Hi @dexterbg
Thanks for the report and sharing a very simple sample application that exhibits the problem
void app_task(void *userdata)
{
  ...
    while (xTimerChangePeriod(timer, 1, portMAX_DELAY) != pdPASS);  // [1]
    vTimerSetTimerID(timer, (void *) tid);   // [2]
    while (xTimerStart(timer, portMAX_DELAY) != pdPASS);   // [3]
  ...
}

I wonder why you keep changing the timer period [1] before starting it [3]? Isn't it possible (when the system is really heavy loaded) that the timer callback gets called before we update the timer id [2], i.e. with the older tid?
Assuming that [1] wakes the timer from its dormant state:
https://github.com/espressif/esp-idf/blob/8153bfe4125e6a608abccf1561fd10285016c90a/components/freertos/include/freertos/timers.h#L622-L623
https://github.com/espressif/esp-idf/blob/8153bfe4125e6a608abccf1561fd10285016c90a/components/freertos/timers.c#L879-L892

Hi David,
I seem to never have noticed that xTimerChangePeriod() actually also starts a stopped timer, although it's clearly described in the documentation (https://freertos.org/FreeRTOS-timers-xTimerChangePeriod.html).  I'm pretty sure that's the cause for the single shot callback dupes.
Any idea on why the cyclic timers also seem to get an occasional extra callback? Note we have no conclusive evidence for this happening in our main application, only some occasional effects that may be caused by this -- or by something else. So there may be something wrong with my test design here.
Thanks & regards,
Michael

Hi Michael,
Sorry for the late reply, no idea about those extra callback of periodic timers. I think it might be tricky to (dis)prove the duplicated callbacks of a cyclic timer. I would perhaps try to match timer's callback occurrences with kernel ticks?

@sudeep-mohanty Thanks for investigating this. I'll try v4.4 ASAP, but your quoted log's runtime was a magnitude too short for the effect to occur.
On my test runs, the first stable offset never occurred before about an hour runtime, with sometimes some hours between two offset jumps.

@dexterbg I redid the experiment for a (much) longer duration of about ~ 4 hrs and I still did not observe the timer callback being called more than once. Did you get a chance to test it at your end?
Logs -
I (14765825) app_task: checkpoint tid=1476000
I (14765835) periodic_callback: checkpoint: offsets = 0, 0, 0, 0, 0
I (14775825) app_task: checkpoint tid=1477000
I (14785825) app_task: checkpoint tid=1478000
I (14795825) app_task: checkpoint tid=1479000
I (14805825) app_task: checkpoint tid=1480000
I (14815825) app_task: checkpoint tid=1481000
I (14825825) app_task: checkpoint tid=1482000
I (14835825) app_task: checkpoint tid=1483000
I (14845825) app_task: checkpoint tid=1484000
I (14855825) app_task: checkpoint tid=1485000
I (14865825) app_task: checkpoint tid=1486000
I (14875825) app_task: checkpoint tid=1487000
I (14885825) app_task: checkpoint tid=1488000
I (14885835) periodic_callback: checkpoint: offsets = 0, 0, 0, 0, 0

Environment

IDF versions: v3.3, v4.4 (release branches as of today)

@dexterbg
It's difficult to know the exactly commit with above description.
Please share "git describe --tags" output so others can verify if the problem is reproducable on exactly the same commit.
PS. please always use "git describe --tags" for IDF versions when report issues.

I can confirm to also not be able to reproduce this with the current v4.4 release & toolchain.
esp-idf: v4.4.1-581-gee2029c38a
toolchain: crosstool-NG esp-2021r2-patch3 8.4.0

I've been running the test using different build configurations on an ESP32/R1, so I'm pretty sure this has been resolved.
So I guess this issue can be closed.
Thanks everyone involved!

Thanks for confirming @dexterbg!

@sudeep-mohanty
Which commit fixed this issue? (I'd like to know if it's also fixed in other v4.x branches)

Sorry, I cannot tell if I actually did the periodic callback test on the 4.4 branch. My test logs archived only show the periodic test on v3.3.4-848-g1ff5e24b1b, the only test log I've kept from the idf4 test was with v4.4-beta1-284-gd83021a6e8-dirty, but that was for the single shot test, which was flawed by me.
So the fix for the periodic callbacks may have been in a much earlier idf commit, or may have been in the new toolchain as well.


How to use the hyperkey?

It it possible to use hyper key modifier?

Not within keyd, because the Hyper key is an X11 concept. See #370, #220, #592 for more details and possible approaches for what you're trying to do :)


feat: extract & expose toString for stringifying types

Closes #1160
This essentially moves all the type model toString implementations into a separate module and exposes them for users, as spec'd in the above issue. toString is implemented on the base model now, so that method is still kept around for backward compatibility currently.
Currently it should have parity, and has a start for stringifying reflections in a similar manner (just call signature for now).
I've also added tests, which don't cover everything, but these weren't actually tested before at all. Progress!

Heads up - as a part of library mode work I think I've resolved the issues with wrapping. https://github.com/TypeStrong/typedoc/blob/library-mode/src/lib/models/types/abstract.ts#L59
This still won't quite work for your JSON use case, but library mode also brings us much closer to being able to reinflate JSON into the original models.

@Gerrit0 cool, so your library-mode branch subsumes the functionality of this PR? If that's the case, I'd be up for working on JSON deserialization once the dust settles. Is that something you consider in scope for the type models in typedoc?

It doesn't contain everything that this does insofar as it won't work for JSON models... I think deserialization will meet all the requirements from here.
I absolutely want to support deserialization of the full project. Tracking in #910 :)

At this point, I don't think I'm going to ever merge this -- it's a nice idea, but until TypeDoc itself can be run without Node APIs, I don't want to maintain multiple entry points to typedoc.
TypeDoc 0.24 did add support for deserializing a JSON project, so at this point I'm recommending using TypeDoc's deserialization API, and then calling toString on the created type..


bug: Nuxters dialog does not finish loading

🐞 Description
I wanted to press the Add your Nuxter card on Github button and display the Nuxters Card for GitHub.
However, the image does not appear and the loading never ends.

🔗 Linked Issue
related #65 .

@Atinux an idea for this error ?

https://nuxters.nuxt.com/shinGangan

@shinGangan https://github.com/nuxt-modules/og-image/pull/201 ;-)

@maximepvrt
Oh, thanks for opening PR. 🙏


Loading fixtures produces an auth_user error

This is in the readme: python manage.py loaddata fixtures/neilTaskRankingData.json
But results in:
Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/Library/Python/2.7/site-packages/django/core/management/init.py", line 338, in execute_from_command_line
utility.execute()
File "/Library/Python/2.7/site-packages/django/core/management/init.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv
self.execute(*args, **cmd_options)
File "/Library/Python/2.7/site-packages/django/core/management/base.py", line 441, in execute
output = self.handle(*args, **options)
File "/Library/Python/2.7/site-packages/django/core/management/commands/loaddata.py", line 60, in handle
self.loaddata(fixture_labels)
File "/Library/Python/2.7/site-packages/django/core/management/commands/loaddata.py", line 96, in loaddata
connection.check_constraints(table_names=table_names)
File "/Library/Python/2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 221, in check_constraints
self.cursor().execute('SET CONSTRAINTS ALL IMMEDIATE')
File "/Library/Python/2.7/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/Library/Python/2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/Library/Python/2.7/site-packages/django/db/utils.py", line 97, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/Library/Python/2.7/site-packages/django/db/backends/utils.py", line 62, in execute
return self.cursor.execute(sql)
django.db.utils.IntegrityError: Problem installing fixtures: insert or update on table "crowdsourcing_userprofile" violates foreign key constraint "crowdsourcing_userprof_user_id_7592ae1f4a72d702_fk_auth_user_id"
DETAIL:  Key (user_id)=(2) is not present in table "auth_user".
Did something change about the model?

No, just the json file needs to be adjusted

+1

The updated file should runs perfectly fine
manage.py loaddata fixtures/neilTaskRankingData.json
Installed 942 object(s) from 1 fixture(s)
In case you want to add requester  add below. Otherwise use the updated file
    {
        "fields": {
            "profile":2
        },
        "model": "crowdsourcing.requester",
        "pk": 2
    },
     {
        "fields": {
            "profile":3
        },
        "model": "crowdsourcing.requester",
        "pk": 3
    }


False positive while code return wrapped pointer

False positive while code return wrapped pointer. See following code:
https://github.com/master-q/practice-infer/tree/b0d810f3a326dda8d32ea0770e577a434b14635f/wrap
The code causes following result on Infer:
$ make verify
infer -- gcc `pkg-config glib-2.0 --cflags` -c main.c
Infer version v0.9.4-e757d92
Copyright 2009 - present Facebook. All Rights Reserved.
Starting analysis (Infer version v0.9.4-e757d92)

legend:
  "F" analyzing a file
  "." analyzing a procedure

F....
Analyzed 1 file

Found 1 issue

main.c:15: error: MEMORY_LEAK
   memory dynamically allocated to new_p by call to g_malloc() at line 14, column 15 is not reachable after line 15, column 2
  13.   struct foo* create(void) {
  14.           int *new_p = g_malloc(sizeof(int));
  15. >         return wrap(new_p);
  16.   }
  17.

Summary of the reports

  MEMORY_LEAK: 1

But the new_p is freed by delete() function.

Here Infer could not infer specs for wrap and create since the calls g_malloc(sizeof(...)) are causing PRECONDITION_NOT_MET. Maybe sizeof(...) is evaluated to 0, as mentioned by @ddino in #497. Replacing g_malloc with malloc removed those FPs (see code below).
include <stdlib.h>

struct foo {
  int *p;
};

struct foo* wrap(int *p) {
  struct foo* f = malloc(sizeof(struct foo));
  if (!f) exit(1);
  f->p = p;
  return f;
}

struct foo* create(void) {
  int *new_p = malloc(sizeof(int));
  if (!new_p) exit(1);
  return wrap(new_p);
}

void delete(struct foo *f) {
  free(f->p);
  free(f);
}

int main() {
  struct foo *f;
  f = create();
  delete(f);
}

I'm trying to fix the problem with g_malloc in #497, hopefully we can fix both problem.
At the moment not clear if both problems comes from  the same issue.

Looks like this is no longer an issue. With infer 0.15.0 I get a memory leak, but in main(), which I think is expected.


modal directive?

I am wanting to use semantic's modal module in a way similar to the AngularUI-bootstrap modal directive. Unfortunately my Angular-foo is not yet strong enough to write it myself. Is anyone working on more directives for semantic-ui and angular?

@micha-LEAP, I own this project so far. I will try to ship a solution for modal directive as soon as I can.

Any news on this @m0t0r?
Also https://github.com/Semantic-Org/Semantic-UI-Angular/issues/23

@RomainLanz not much, but will return to this issue as soon as I can. If you are willing to create this directive we are happy with that :+1:

@m0t0r I'm interested to start work on it. Can I?

@matheuspoleza, can you ping me on gitter ?

@m0t0r of course

closing all issues and PR for a while. Restarting project roadmap

also looking for the modal directive, which is why I'm still tied to angular-ui-bootstrap.
I'm glad to contribute.


Can not launch new job instance on MariaDB 10.4.12 (Column 'INTERNAL_POSITION' cannot be null)

When starting a new job instance I get the following exception (MariaDB 10.4.12):
Caused by: com.enioka.jqm.jdbc.DatabaseException: INSERT INTO JQM_JOB_INSTANCE (ID, DATE_ENQUEUE, EMAIL, APPLICATION, KEYWORD1, KEYWORD2, KEYWORD3, MODULE, INTERNAL_POSITION, PARENT, PROGRESS, SESSION_KEY, STATUS, USERNAME, JOBDEF, QUEUE, HIGHLANDER, FROM_SCHEDULE, DATE_NOT_BEFORE, PRIORITY, INSTRUCTION) VALUES(?, CURRENT_TIMESTAMP(3), ?, ?, ?, ?, ?, ?, ROUND(UNIX_TIMESTAMP(CURTIME(4)) * 1000), ?, 0, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
    at com.enioka.jqm.jdbc.DbConn.runUpdate(DbConn.java:129)
    at com.enioka.jqm.model.JobInstance.enqueue(JobInstance.java:570)
    at com.enioka.jqm.api.JdbcClient.enqueueWithCnx(JdbcClient.java:329)
    ... 56 more
Caused by: java.sql.SQLIntegrityConstraintViolationException: (conn=123) Column 'INTERNAL_POSITION' cannot be null

This error is caused by 'ROUND(UNIX_TIMESTAMP(CURTIME(4))' which returns NULL on that database version. Older versions of MariaDB are working (I've tested it with MariaDb 10.1).
10.4.12-MariaDB-1:10.4.12+maria~bionic:
select ROUND(UNIX_TIMESTAMP(CURTIME(4)) * 1000);
returns: NULL
10.1.44-MariaDB-1~bionic:
select ROUND(UNIX_TIMESTAMP(CURTIME(4)) * 1000);
returns:<PHONE_NUMBER>681
I don't know which MariaDB version causes this breaking change, but the following statement works on the new version:
The following statement works on 10.4.12-MariaDB-1:10.4.12+maria~bionic:
select ROUND(UNIX_TIMESTAMP(NOW(4)) * 1000);
returns:<PHONE_NUMBER>681

@marcanpilami

select UNIX_TIMESTAMP(CURTIME(4)); works in 10.3, gives NULL in 10.4.
select CURTIME(4); and select CURTIME(4)+0; gives same result in both.
It seems to be the consequence of https://jira.mariadb.org/browse/MDEV-20695 slated for resolution in a future 10.4 release.
We will work around it in the meantime.

@marcanpilami thank you for such a short turnaround. Could you please cut a new 2.2.x release with this fix?

@marcanpilami thank you for such a short turnaround. Could you please cut a new 2.2.x release with this fix?


Source env improvement2

I noticed that in case of nested config all the other attributes don't matter (you can't env it, nor default.) so I do early return to make code "work"

I have no idea what's going on with the incoming config where the default doesn't default ;_;

bors r+

bors r+


On Close, write dots to temporary layers

One per instance, using settings in the instance custom parameters

It might be better to have Pendot Designer as a plugin that could be opened and remain open, and can write to layers that remain in the project, and would be updated if they already exist. I can easily see this workflow:

Open tool
Toggle UI control to either use instance custom parameters or ones set in UI fields
Set layer name suffix (with reasonable default)
Press Apply
Look through layers and glyphs to review
Tweak parameters either in instance or in the still open plugin UI
Press Apply, review again, tweak...
Close UI when done

In general, closing a filter or plugin should not be a substitute for an Apply button.

There are two other related issues:
The modal nature of the UI means that once you activate it you can't move around in the preview window. So you need to set up what you want to see in the window, then activate the plugin, but then you can't see anything else. You need to close the plugin and repeat, which is a pain.
Currently when you activate the plugin it immediately applies whatever settings were last used. Even for only 26 letters that can take 3-10 seconds, then every time you change a single parameter it recalculates again. The live nature of this is cool if you're looking only at a single glyph, but if you've selected 20+ glyphs it gets in the way. It would be better if the plugin didn't apply anything until you hit an Apply button. Some delay, especially for a large number of selected glyphs, is expected but it should happen only when you want it to.

The modal nature of the UI means that once you activate it you can't move around in the preview window. So you need to set up what you want to see in the window, then activate the plugin, but then you can't see anything else. You need to close the plugin and repeat, which is a pain.

This sadly appears to be the nature of GlyphsApp filters; the other filters (try Noodler) behaves the same. You should be able to zoom in and out, though. Apparently we can have a non-modal interface with a vanilla-based plugin instead of a filter, and maybe this is where we need to go because we are starting to do a lot of things that don't really fit the "filter" model anyway.

OK, this is (I think) done, at least from the UI side. (Build side to follow.) Now we are both a reporter and a script, I'm going to add a readme to help with installation etc. - and @jvgaultney, you will need to make changes to your installation to get it working again. I'll document all that.


Tweak legacy package repository deprecation article

Fixups for #42811 / https://blog.k8s.io/2023/08/31/legacy-package-repository-deprecation/

bring it forward by half an hour
wrap lines
mark the blog article as evergreen (it will never get marked as out of date)

This one's important.
/approve

I've previewed this locally. I'll merge manually so comms can go out.


Start and Load Game Option

Hello @gennariarmando ...
Please Add Start New Game Option at Game EXE Start

I think this mod is designed to load the first saved file as a continuation for the story mod, while if the saved file in slot 2 the mod will not see it in the start screen, and it will take you to a new game where you can load the saved game from there.


Docker Machine should support trusting DTR instances and custom CAs at create time

@metcalfc pointed out to me that hack scripts like this: https://gist.github.com/metcalfc/50060709ee9d22ec620e are being used to trust companies' custom self-signed certs in boot2docker so that they can pull from their internal registry.
@nicokabar has a similar issue of wanting to automatically trust a given Docker Trusted Registry at create time.
I'd say supporting such use cases is critical for enablement of customer success, which definitely is a high priority for us.
So, my proposal (just spitballing here, haven't thought out the technical specifics):

--engine-trust-registry flag which would bootstrap trust for a given DTR instance as part of the provisioning based on the URL (e.g. --engine-trust-registry registry.mycompany.com)
--virtualbox-trust-ca which would allow trust of self-signed CAs within the created boot2docker instance.

@docker/machine-maintainers WDYT

cc @nicolaka

See also: https://github.com/docker/machine/issues/1872

Would like to see this. Lots of issues related to this: https://github.com/boot2docker/boot2docker/issues/347, https://github.com/docker/machine/issues/1799, https://github.com/docker/machine/issues/1963, https://github.com/docker/machine/issues/1872

Hi, I'm +1 to adding --engine-trust-registry, I'm handling this case by ssh and copying the ca to the appropriated places.


Previous issue about this: #2890

Cloudflare is now hosting an alternative service to replace polyfill(.)io
https://blog.cloudflare.com/automatically-replacing-polyfill-io-links-with-cloudflares-mirror-for-a-safer-internet/
https://cdnjs.cloudflare.com/polyfill/

Cloudflare had a polyfill[.]io mirror for a while: https://blog.cloudflare.com/polyfill-io-now-available-on-cdnjs-reduce-your-supply-chain-risk.
What's new is that we are automatically rewriting the insecure polyfill[.]io scripts to our mirror: https://blog.cloudflare.com/automatically-replacing-polyfill-io-links-with-cloudflares-mirror-for-a-safer-internet/.

@lovetingyuan the repo is now flagged

what's the status of this?

up!

What's new is that we are automatically rewriting the insecure polyfill[.]io scripts to our mirror

How? Are they modifying my files and databases?

@krystian3w  if your website is using Cloudflare's cdn and you enabled the polyfill rewriting feature, it will automatically rewrite your scripts tag to point to Cloudflare's fork.

https://www.bleepingcomputer.com/news/security/polyfillio-bootcdn-bootcss-staticfile-attack-traced-to-1-operator/


No second controller

Hi,
awesome project, thanks!
The hand icon with the number one on it seems to imply that you can pair other mobile devices ('wii style'). But this is not the case, unless I did something wrong?
Best,
Dirk

Only one controller is supported at the moment. I can see why the number on the cursor icon can be misleading; I should've gone with a plain icon.

I'm closing this issue because the goal of this project was to show that a smartphone can turn into a 3D controller with a web app and the goal has been accomplished. There are no plans for further development of this project.


How to get sensor values ?

Hi, I just got a Sphero Bolt and I'm learning to use it with your library. Sadly all sensors return "None". Can you tell me where is the mistake ? Here is my code :
import time
from spherov2 import scanner
from spherov2.sphero_edu import SpheroEduAPI
from spherov2.types import Color

toy = scanner.find_toy()
print(toy)
with SpheroEduAPI(toy) as droid:
    droid.reset_aim()
    droid.set_main_led(Color(r=0, g=0, b=255))
    print(droid.get_acceleration())
    print(droid.get_luminosity())
    droid.set_speed(60)
    time.sleep(0.5)
    droid.set_speed(0)
    print(droid.get_location())

Ill look into this shortly. Stuck on the other issues so I'll get you an answer asap

So i found out what's going on here, however i also found a much better way to translate the APK and am going to attempt to update code to do so, but there are calls missing with params not currently available in the library so updates will be coming with this in the forefront of my mind. Am currently working on.

Okey, I'm waiting for your next update. My students will love these ! Thanks for all

great....exciting stuff, thank you!

Hey @superfashi I'm kindve a noob at this right now. Investigating still, but curious if you have insights? it appears that sensor_control for the Bolt should return StreamingControl rather than StreamingControl, however in utils.py in add_listeners I'm hitting false for hasattr(toy, 'sensor_control'). Thats as far as I've gotten so far. Learning these python nuances but I'm hitting a wall at every turn LOL

Let's use emails for this. Mine can be found under my homepage.

Hey All,
I just cracked the issue here, will be getting a fix up soon, most likely by EOD today. Sorry for the delay, hope it's not too late for your uses, let me know if there are any other scenarios you may find to not work after I submit.

Pull request submitted, should be available once approved


sub-issue should keep epic info

to describe what an issue is about, the topic of work or component, we typically use this syntax:
[RequestView] more detailed title // this is a component
[shortcuts] more detailed title // this is a topic

I'm referring to the content of the square brackets as epic just to give it a name and because we called it like this in Jira.
It would be useful to prepopulate the sub-issue title with [...]  populated with the same epic as the parent issue.

:+1:


Base adapter is choosing the wrong view to recycle.

I'm having some problems with the cell reuse functionality of the base adapter.  It's getting confused when I reload the list.
What's happening is I'm getting false positive matches.  For example, my header is getting chosen to present a data item.
I think this is caused from the cell type auto-generation logic.
I'm actually thinking it might not be possible to auto generate the view types.
The problem is:  the adapter needs to know all possible cell types that could ever go into the list.  Not just the ones that are there now.  (getItemViewCount never gets called more than once!).
So let's say i have a list of things:   [HeaderCell, UserCell, UserCell].  No problem, the right thing happens.
But if the adapter starts off life like this:  [NoUsersAddedYetCell], the adapter say, "Got it!  1 Cell only".   When you swap the underlying adapter data, hilarity ensues.
For now, the workaround I'm going to use is to manually unhook the adapter and recreate another one when the dataset changes.  It's slow and you lose your scroll position, but the views are correct.

In those rare situations where you know there will be a new view type we can add some kind of syntax to add to our internal collection: so getViewTypeCount  and getItemViewType behave as planned.
Essentially the auto-generation logic can be in place as-is for 90% of lists, and those who may dynamically load more will be required to identify their needs ahead of time.

Here's an example.
In this version, it actually crashes when you scroll.
https://github.com/skellock/bluepotion/blob/reuse-shenanignas/app/screens/example_custom_table_cells_screen.rb
Steps To Reproduce:

Load the example app
click "Open Table With Custom Cells"
click any red cell to inject the poison!
scroll
read cryptic error message Java.lang.ArrayIndexOutOfBoundsException: length=2; index=2

This is now solved simply with one line:

@twerth this is a key item we should document at some point.


page renders before sagaTask is done

When we navigate through the pages "next-redux-saga" isn't waiting full loading of getInitialProps. This raises the problem of a "blinking page". When I click link (on gif it will be githab pages) on already loaded page - it show the page instantly, but data will load in 1-2 sec after that. It's not correct in most cases, bcs I wanna show the page with already loaded data. If we will load load page from server - everything is okay. I think we need await task end on client side too.

Hi @JerryCauser, thanks for submitting this issue & PR.
I have the page loading immediately on purpose, to provide instant feedback to the user. Data that is loading asynchronously on the client should use a spinner or loading message to notify the client.
I'm not against providing an alternative way to handle routing with async data, however, I don't think this should replace the existing default behavior. Can you adjust your PR to support the existing default behavior, while allowing the ability to opt-in to this alternative rendering model? One way to do this would be using an named export for an alternative component.  Please make sure there are tests to support this as well.
Thoughts?

I'm not against providing an alternative way to handle routing with async data, however, I don't think this should replace the existing default behavior.

Yeah, of course. Backwards compatibility in priority. I accent on that trouble because redux-observable, redux-thunk and redux-promise-middleware (and all "promise like" middlewares) have same scenario of lifecycle with awaiting on client side as I decrived before.

Can you adjust your PR to support the existing default behavior, while allowing the ability to opt-in to this alternative rendering model?
Yeah, of course. I have also 2 ideas in addition to yours.

Make @decorator withReduxSaga smarter and give him possibility to get async/sync flag as first argument. Something like that:

//async client side
class MyComponent extends React.Component {/**some code**/}
export const wrappedComponent = withReduxSaga(MyComponent)

//sync client side
class MyComponent extends React.Component {/**some code**/}
export const wrappedComponent = withReduxSaga({sync: true})(MyComponent)

//with es6-decorators
//async client side
@ withReduxSaga
class MyComponent extends React.Component {/**some code**/}

//sync client side
@ withReduxSaga({sync: true})
class MyComponent extends React.Component {/**some code**/}

Let's leave the decorator alone. And let component describe itself by static props sync:

//async client side
@ withReduxSaga
class MyComponent extends React.Component {/**some code**/}

//sync client side
@ withReduxSaga
class MyComponent extends React.Component {
  static sync = true //sync by default undefined, whats mean false for backwards compatibility
  /**some code**/
}

Which one look better for you? I like idea with tuning @decorator, but idea with second decorator (withReduxSagaSync for example) is nice too.

Please make sure there are tests to support this as well.

I think here I will need your advice which test I should add. But anyway it's not the problem.

I accent on that trouble because redux-observable, redux-thunk and redux-promise-middleware (and all "promise like" middlewares) have same scenario of lifecycle with awaiting on client side as I described before.

Thanks again for pointing this out.  I did some more research on how Next.js works out of the box and in their data-fetch example, it does wait for getInitialProps to finish loading before transitioning to the new route.  If that is how the other projects you mentioned work as well, I think it's better to break the API for next-redux-saga to be consistent with the community.
That being said, what do you think about releasing a new major version (2.x) with sync as the default?  Passing {async: true} would provide backwards compatibility.

// sync client side
class MyComponent extends React.Component {/**some code**/}
export const wrappedComponent = withReduxSaga(MyComponent)

// async client side
class MyComponent extends React.Component {/**some code**/}
export const wrappedComponent = withReduxSaga({async: true})(MyComponent)

// with es6-decorators
// sync client side
@ withReduxSaga
class MyComponent extends React.Component {/**some code**/}

// async client side
@ withReduxSaga({async: true})
class MyComponent extends React.Component {/**some code**/}

I think here I will need your advice which test I should add. But anyway it's not the problem.

No problem, we can work through the tests together.


fix(threads): add atomic.fence

atomic.fence was added to the threads proposal in
https://github.com/WebAssembly/threads/pull/141, and supported in wabt
in
https://github.com/WebAssembly/wabt/commit/d041025854ba2b00b3df9f308914c8aba05dcda9#diff-25d902c24283ab8cfbac54dfa101ad31.

Not sure how to add tests, I followed the instructions in the README, added a .wasm file to test/ but it didn't seem to run. When I moved it to test/fixtures, it ran and failed, and npm test update gave me this:
$ npm test update

><EMAIL_ADDRESS>test /usr/local/google/home/zhin/ssd2/wasmparser
> jest "update"

No tests found, exiting with code 1
Run with `--passWithNoTests` to exit with code 0
In /usr/local/google/home/zhin/ssd2/wasmparser
  41 files checked.
  testMatch: **/__tests__/**/*.[jt]s?(x), **/?(*.)+(spec|test).[tj]s?(x) - 4 matches
  testPathIgnorePatterns: /node_modules/ - 41 matches
  testRegex:  - 0 matches
Pattern: update - 0 matches
npm ERR! Test failed.  See above for more details.

Perfect, thanks for pointing that out, added in ea031b4.

npm test update gave me this:

Created #74 to update instructions. Use npm test -- --updateSnapshot

I ended up having to update wabt to 1.0.15, because that's when the atomic.fence instruction was added to wabt, otherwise it was failing at the compatibility test.
And the update with wabt required me to fix one of the test in WasmDis.test.ts, which had a redefinition of function error.
(btw for the merge I will do a squash so it's one commit for this fix and change, lmk if you want me to do otherwise.)

@ngzhian thanks for fixing here. Can you roll the new wasmparser release into Chrome DevTools?

:tada: This PR is included in version 3.1.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add support for inverting axes

This PR adds support for almost all Element types to be used as adjoints in bokeh by correctly implementing axis inversions on the plotting classes.

This looks amazing!  Let's merge right away!  I can't tell why the tests are taking so long.

Opened the issue above to ensure that we add support for inverting the axes on Raster types sometime in the future as well. For now this is ready to merge as it massively improves our existing support for adjoining plots and inverting axes.

Looks good! It will be good to finish this off so it is supported everywhere but this is already a nice improvement. Merging.


Correct jsnext:main

According to jsforum/jsforum#5 jsnext:main should contain the transpiled version with the exception of ES6 modules.
Users can include the library without having to apply the babel transformations needed by this package

I recommend you also to add same module field which is supported and with rollup-plugin-node-resolve and with webpack.

@TrySound thanks for the comment, added

I'm comfortable with the suggested approach, just wondering if it could be accomplished via a single rollup.config.js.

Didn't know about targets. It's now everything in the same rollup file

sourceMap: true has the same effect of manually specifying the path.
Do you guys prefer if I squash all commits together?

@piuccio sourceMap can be 'inline', truely and falsy. So yes, replace with true and squash please.

Ah, I see. Cool.

done 😃

How certain is the ES5 requirement here?  I haven't finished reading through that thread, but my understanding so far is that ES2015 (without stage-xx or any one-off plugins) is assumed to be acceptable if the source is already using ES2015 modules.  Neither preact nor preact-compat use react-specific stuff, it's all just things from the es2015 preset (intentionally only using the spec'd stuff).  Thoughts?

I'm having problems because preact-compat includes code like this which isn't ES2015 so it's better to transpile it here rather in the clients.
As for the preset es2015, it might be unnecessary, but I guess it benefits lot of people that don't run babel on their node_modules folder but still expect the code to work in most browsers.

@developit Do you supports IE?

IE9+

@TrySound great question, let's see. My project has to work on IE9+

In this case module should have es5+modules entry point to prevent long-time vendor transpiling.

@piuccio Ahh - here I thought there was no JSX in preact-compat - IMO that's actually a bug, but I see your point.  One trade-off this makes though, is that for those transpiling jsnext:main / modules:main, you end up with multiple copies of each Babel helper.

well if you get rid of JSX in the source code, I might be able to use jsnext:main without this change.
Ultimately I want to tree shake preact and preact-compat. At the moment they're adding 8kb to my project and I'm assuming it can go down.

I will definitely get rid of the JSX.  Everything else in there should just be es2015 (via babel, buble, etc - as per the spec).  I was debating whether to do that via a transpile step, but really it's only used in one place so I'll probably just use hyperscript directly.

@piuccio Didn't mean to auto-close this issue, but I did just remove all the JSX and add modules:main.  Let me know if that lets you use jsnext:main!  The codebase should now work fine with bublé or babel --presets es2015 (no stage-xx)

@piuccio heads up, I've released the changes in 3.0.0. Take a look and let me know if it's working! 😊

Thanks, I'm now on holiday, will let you know next week

Looks like @megafirzen is running into the same sort of issue now that we're publishing a "modules" pointer.  I'll manually merge this PR and start publishing .es.js :)

Sorry for the delay, I managed to test it right now and I still have some troubles.
JSX is gone but there are a couple of arrow functions here and there (like here) which break my rollup when I use module: true.
So for the moment I'm stuck with jsnext and module set to false.

@piuccio How's arrow functions breaks rollup? Do you use uglify? Rollup uses acorn which can parse arrow functions and whole es2015.

Yes it breaks the uglify step

@piuccio Then enable babel or buble transpiller by yourself.

@TrySound yeah I might enable babel on node modules but it's a bit weird that I have to compile my dependencies. It certainly makes compiling slower. I might give it another go when I have more time

It's @developit decision. And I'm not against. Now we can use es6 in browsers. For minifying can be used babili. If you want to support older browsers. You should by yourself prepare that code. es6 is standard, not extension.

Closing this, I'm not sure using babel on node_modules is the best solution but at least it's a solution


Support Bitnami embedded SBOMs

What would you like to be added:
As part of anchore/grype#1609, Syft should pick up on sboms in containers located at /opt/bitnami because this is how Bitnami records what's in an image.
The SBOM cataloger would probably do this already, but is off by default.
There are a few open questions here:

How should packages discovered by other catalogers interact with these SBOMs? For example, the binary cataloger might find Redis or MariaDB executables.
What if someone is building something FROM a Bitnami image? How do we know we can trust the SBOM?
If we are special-casing Bitnami images, e.g. turning the SBOM cataloger on by default only for certain images or certain paths, how do we detect this situation and what configuration options are available?

Why is this needed:
This is primarily needed so that running grype on a Bitnami image (see anchore/grype#1609) is as accurate as possible.
Additional context:
There are a few open requests for more accurate Bitnami classification. Ideally this work might also fix those.

Is there another way to scan these artifacts? Are these container images in some differing format from OCI? If the only way to identify what is installed is by scanning an SBOM, there could probably just be a Bitnami cataloger that looks for specific SBOMs in these known bitnami locations, instead of enabling the SBOM cataloger itself. It's pretty easy to just pass a reader to the SBOM decoder. And then we'd probably want to have a way to prevent SBOMs from getting scanned twice if a user does enable the SBOM cataloger.

Two questions for investigation:

If we add a bitnami cataloger, and turn both it and the SBOM cataloger on, do we get duplicates?
Do we and should we surface all artifacts and relationships from the bitnami SPDX in the Syft output SBOM?

If we add a bitnami cataloger, and turn both it and the SBOM cataloger on, do we get duplicates?

I did an experiment to answer this.

Copy the SBOM cataloger to make a new bitnami cataloger, but change the glob list to be only "/opt/bitnami/**/*.spdx"
Wire the new cataloger up here: https://github.com/anchore/syft/blob/main/internal/task/package_tasks.go#L151
Run syft with sbom and bitnami on, with each on, and with neither on, and look at the packages returned:

❯ go run ./cmd/syft -q --select-catalogers "-sbom-cataloger,+bitnami-cataloger" bitnami/moodle:4.4 -o json |\
 jq -r '.artifacts[] | select(.foundBy == "bitnami-cataloger" or .foundBy == "sbom-cataloger") | .name' |\
 shasum
b07dd9b416f25edca5e143218ac6474360980fce  -

❯ go run ./cmd/syft -q --select-catalogers "+sbom-cataloger,+bitnami-cataloger" bitnami/moodle:4.4 -o json |\
 jq -r '.artifacts[] | select(.foundBy == "bitnami-cataloger" or .foundBy == "sbom-cataloger") | .name' |\
 shasum
b07dd9b416f25edca5e143218ac6474360980fce  -

❯ go run ./cmd/syft -q --select-catalogers "+sbom-cataloger,-bitnami-cataloger" bitnami/moodle:4.4 -o json |\
 jq -r '.artifacts[] | select(.foundBy == "bitnami-cataloger" or .foundBy == "sbom-cataloger") | .name' |\
 shasum
b07dd9b416f25edca5e143218ac6474360980fce  -

So I think the answer to question 1 is, "at least as it stands right now, Syft's existing deduplication logic works fine if both catalogers are on." Of course, in this experiment the catalogers are identical, but it's still a good sign on question 1 above.

I've attached the SBOM syft makes in my experiment:
go run ./cmd/syft -q --override-default-catalogers "bitnami-cataloger" bitnami/moodle:4.4 -o spdx >/tmp/from-syft-bitnami.spdx.txt

from-syft-bitnami.spdx.txt

@willmurphyscode I started working on this and I realized that packages detected by a new "Bitnami" cataloger are given the type UnknownPackage and, after reading the developing guide I wonder whether it makes sense to create a new "Bitnami" package type with the captured data from Bitnami. For instance, the revision information we include in Bitnami versions, see:

https://github.com/bitnami/go-version

I guess this could complicate how to manage duplicates reported by both sbom and bitnami catalogers but I guess we could use the PURL for that.

Hi @juan131  (cc @wagoodman),
Some thoughts here:

Are the packages represented in the bitnami SBOMs from different ecosystems? For example, is it a Go binary or a Python package or something? I think they are usually just native binaries, like Redis or MySQL executable files, but I'm not sure. If so, it might make sense to keep them in those packages. If the package is an existing package type, e.g. Go or Binary, it might make sense to put it there.
Are they binary packages? For example, if you have a compiled MySQL server executable, that sounds like a binary package. (Also, if the binary classifier and bitnami cataloger both find it, we should de-dupe in favor of bitnami).
We could put a query param in the PURL, like bitnami=true or source=bitnami or something, to inform grype matching. Also, repository_url=bitnami.com or something is within the PURL spec, and could tell Grype to search bitnami vulns for this package.
We're reluctant to introduce a new package type, because bitnami is really a vendor of the package, not kind of package.

In short:

The PURL should say that the package is from bitnami somehow so that Grype and other tools can use your vulnerability feed.
We should not make a new package type, because bitnami packages are from a specific vendor, not a specific kind of package, so this information should go somewhere in the PURL besides the type, e.g. a query parameter.
We will require a Grype change here, and it probably makes sense to pull in https://github.com/bitnami/go-version for the Grype version comparison

bitnami is a purl package type though: https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst#bitnami

Are the packages represented in the bitnami SBOMs from different ecosystems?

Yes. In the same SPDX file packages from different ecosystems can coexist. Take the examples below (taken from bitnami/kubectl image) of packages listed in the kubectl SBOM:

We have "bitnami" packages such as this one:

        {
            "SPDXID": "SPDXRef-kubectl",
            "name": "kubectl",
            "versionInfo": "1.31.1-1",
            "downloadLocation": "git+https://github.com/kubernetes/kubernetes#refs/tags/v1.31.1",
            "licenseConcluded": "Apache-2.0",
            "licenseDeclared": "Apache-2.0",
            "filesAnalyzed": false,
            "externalRefs": [
                {
                    "referenceCategory": "SECURITY",
                    "referenceType": "cpe23Type",
                    "referenceLocator": "cpe:2.3:*:kubectl:kubectl:1.31.1:*:*:*:*:*:*:*"
                },
                {
                    "referenceCategory": "PACKAGE-MANAGER",
                    "referenceType": "purl",
                    "referenceLocator": "pkg:bitnami/kubectl@1.31.1-1?arch=arm64&distro=debian-12"
                }
            ],
            "copyrightText": "NOASSERTION"
        }

And also "golang" packages such as this one:

        {
            "name": "github.com/MakeNowJust/heredoc",
            "SPDXID": "SPDXRef-Package-808f8a3a08f58be6",
            "versionInfo": "v1.0.0",
            "supplier": "NOASSERTION",
            "downloadLocation": "NONE",
            "filesAnalyzed": false,
            "sourceInfo": "opt/bitnami/kubectl/bin/kubectl",
            "licenseConcluded": "NONE",
            "licenseDeclared": "NONE",
            "externalRefs": [
                {
                    "referenceCategory": "PACKAGE-MANAGER",
                    "referenceType": "purl",
                    "referenceLocator"<EMAIL_ADDRESS>                }
            ],
            "primaryPackagePurpose": "LIBRARY",
            "copyrightText": "NOASSERTION"
        }

Then, relationships links them:
    "relationships": [
        {
            "spdxElementId": "SPDXRef-kubectl",
            "relationshipType": "CONTAINS",
            "relatedSpdxElement": "SPDXRef-Application-b66f42f85c68bc03-kubectl"
        },
        {
            "spdxElementId": "SPDXRef-Application-b66f42f85c68bc03-kubectl",
            "relatedSpdxElement": "SPDXRef-Package-808f8a3a08f58be6",
            "relationshipType": "DEPENDS_ON"
        }

Are they binary packages?

Yes, Bitnami packages can be simply compiled binaries (based on Golang, C, C++, etc.) but they can be also apps written in interpreted languages (e.g. PHP or Node.JS apps)

We could put a query param in the PURL

I don't think that's necessary. As @westonsteimel mentioned, they're recognized as a valid PURL package type.

I think that makes sense @willmurphyscode !! Regarding the 3rd point, when you talk about packages from Bitnami but not pkg:bitnami, what packages are you referring to?

By the way, I added support for the Bitnami pURL type at https://github.com/anchore/packageurl-go/pull/22

when you talk about packages from Bitnami but not pkg:bitnami, what packages are you referring to?

I thought you told us that there were packages in bitnami SPDX files that are have a different purl type:

And also "golang" packages such as this one:
...
<EMAIL_ADDRESS>
from the second example in https://github.com/anchore/syft/issues/3065#issuecomment-2414179292.
So what I was trying to talk about was: Packages that are declared in a Bitnami SPDX manifest and are therefore found by the new bitnami cataloger but, because the bitnami SDPX declares them with a different PURL type, they do not have package type bitnami. Heredoc in your post above is such a package.
@juan131 does that make sense?

I see your point @willmurphyscode
Following the same example about the golang package included in the Bitnami SBOM. I guess the same package will be reported twice:

Once by Bitnami cataloger (analyzing the SPDX file).
Once by Golang cataloger (analyzing go.mod).

I guess the ideal scenario is to have a mechanism to detect both packages are actually the same one (e.g. by comparing their pURL or similar). With this in mind, are we adding value by labeling these packages as "being from Bitnami"?

@willmurphyscode the Bitnami Vulnerability Database only has info about Bitnami packages
For instance, render-template is a component we include on several Bitnami images. If we inspect its SPDX file...
{
    "SPDXID": "SPDXRef-render-template",
    (...)
    "packages": [
        {
            "SPDXID": "SPDXRef-render-template",
            "name": "render-template",
            "versionInfo": "1.0.7-4",
            "downloadLocation": "https://github.com/bitnami/render-template/archive/refs/tags/v1.0.7.tar.gz",
            "licenseConcluded": "Apache-2.0",
            "licenseDeclared": "Apache-2.0",
            "filesAnalyzed": false,
            "externalRefs": [
                {
                    "referenceCategory": "SECURITY",
                    "referenceType": "cpe23Type",
                    "referenceLocator": "cpe:2.3:*:render-template:render-template:1.0.7:*:*:*:*:*:*:*"
                },
                {
                    "referenceCategory": "PACKAGE-MANAGER",
                    "referenceType": "purl",
                    "referenceLocator": "pkg:bitnami/render-template@1.0.7-4?arch=arm64&distro=debian-12"
                }
            ],
            "copyrightText": "NOASSERTION"
        },
        {
            "name": "opt/bitnami/common/bin/render-template",
            "SPDXID": "SPDXRef-Application-4b412cf3f25d2574-render-template",
            "downloadLocation": "NONE",
            "filesAnalyzed": false,
            "primaryPackagePurpose": "APPLICATION",
            "copyrightText": "NOASSERTION",
            "licenseConcluded": "NOASSERTION",
            "licenseDeclared": "NOASSERTION"
        },
        {
            "name": "github.com/aymerick/raymond",
            "SPDXID": "SPDXRef-Package-c77f44f540ae92a0",
            "versionInfo": "v2.0.2+incompatible",
            "supplier": "NOASSERTION",
            "downloadLocation": "NONE",
            "filesAnalyzed": false,
            "sourceInfo": "opt/bitnami/common/package found in: opt/bitnami/common/bin/render-template",
            "licenseConcluded": "NONE",
            "licenseDeclared": "NONE",
            "externalRefs": [
                {
                    "referenceCategory": "PACKAGE-MANAGER",
                    "referenceType": "purl",
                    "referenceLocator"<EMAIL_ADDRESS>                }
            ],
            "primaryPackagePurpose": "LIBRARY",
            "copyrightText": "NOASSERTION"
        },
        (...)
        {
            "name": "github.com/bitnami/render-template",
            "SPDXID": "SPDXRef-Package-8213648cad51225d",
            "supplier": "NOASSERTION",
            "downloadLocation": "NONE",
            "filesAnalyzed": false,
            "sourceInfo": "opt/bitnami/common/package found in: opt/bitnami/common/bin/render-template",
            "licenseConcluded": "NONE",
            "licenseDeclared": "NONE",
            "externalRefs": [
                {
                    "referenceCategory": "PACKAGE-MANAGER",
                    "referenceType": "purl",
                    "referenceLocator": "pkg:golang/github.com/bitnami/render-template"
                }
            ],
            "primaryPackagePurpose": "LIBRARY",
            "copyrightText": "NOASSERTION"
        },
    ],
    "relationships": [
        {
            "spdxElementId": "SPDXRef-render-template",
            "relationshipType": "CONTAINS",
            "relatedSpdxElement": "SPDXRef-Application-4b412cf3f25d2574-render-template"
        },
        {
            "spdxElementId": "SPDXRef-Application-4b412cf3f25d2574-render-template",
            "relatedSpdxElement": "SPDXRef-Package-8213648cad51225d",
            "relationshipType": "CONTAINS"
        },
        (...)
        {
            "spdxElementId": "SPDXRef-Package-8213648cad51225d",
            "relatedSpdxElement": "SPDXRef-Package-c77f44f540ae92a0",
            "relationshipType": "DEPENDS_ON"
        }
}

... we can notice a few things:

The "main" package (name render-template) is a Bitnami package (purl pkg:bitnami/render-template@1.0.7-4?arch=arm64&distro=debian-12)
There's a package (name opt/bitnami/common/bin/render-template) which represents the compiled binary.
There's a package (name github.com/bitnami/render-template) package which is the "main" Golang package (purl pkg:golang/github.com/bitnami/render-template) used in the compiled binary.
There are relationships that describe that render-template (Bitnami pkg) contains opt/bitnami/common/bin/render-template (compiled binary) which contains github.com/bitnami/render-template (golang package)
Other Golang packages (e.g. github.com/aymerick/raymond) are added as dependencies of the "main" Golang package.

If we take a look to the Bitnami Vulnerability Database components (see the link below) we will NOT find any info about the compiled binary nor the golang packages but exclusively about the Bitnami package: render-template.

https://github.com/bitnami/vulndb/tree/main/config/components

Bitnami SBOM might include non-bitnami packages that can't be detected with other cataloger (this is very unlikely).

There might be a specific case where this is likely: native binaries (e.g. ELF files) that were not installed by any package manager. Those are currently challenging to identify, so having bitnami weigh in on them makes sense. Especially if we can get high quality CPEs for Grype's binary matcher to compare against NVD's database.

We implement some mechanism that look for duplicates on packages reported by Bitnami cataloger.

Syft already does some de-duplication of packages. If the Bitnami cataloger raises up all these extra packages, are you seeing duplicates? In other words if you scan an image with render-template in it, do you get 2 artifacts for pkg:golang/github.com/bitnami/render-template, one from the Go cataloger and one from the bitnami cataloger? I suspect Syft's existing deduplication may be working here already. Would you mind testing this based on your current PR and letting us know?
Thanks!

Hi @willmurphyscode
With the changes I'm proposing at https://github.com/anchore/syft/pull/3341, there are no duplicates. However, this is because I'm only reporting Bitnami packages in the current implementation.
If we report every package in the Bitnami SBOM applying this patch...
diff --git a/syft/pkg/cataloger/bitnami/cataloger.go b/syft/pkg/cataloger/bitnami/cataloger.go
index bfa4d3c2..0e8e0616 100644
--- a/syft/pkg/cataloger/bitnami/cataloger.go
+++ b/syft/pkg/cataloger/bitnami/cataloger.go
@@ -44,13 +44,8 @@ func parseSBOM(_ context.Context, _ file.Resolver, _ *generic.Environment, reade

        var pkgs []pkg.Package
        for _, p := range s.Artifacts.Packages.Sorted() {
-               // We only want to report Bitnami packages
-               if !strings.HasPrefix(p.PURL, "pkg:bitnami") {
-                       continue
-               }
-
                p.FoundBy = catalogerName
-               p.Type = pkg.BitnamiPkg
+
                // replace all locations on the package with the location of the SBOM file.
                // Why not keep the original list of locations? Since the "locations" field is meant to capture
                // where there is evidence of this file, and the catalogers have not run against any file other than,
@@ -59,13 +54,16 @@ func parseSBOM(_ context.Context, _ file.Resolver, _ *generic.Environment, reade
                        reader.Location.WithAnnotation(pkg.EvidenceAnnotationKey, pkg.PrimaryEvidenceAnnotation),
                )

-               // Parse the Bitnami-specific metadata
-               metadata, err := parseBitnamiPURL(p.PURL)
-               if err != nil {
-                       return nil, nil, err
-               }
+               if strings.HasPrefix(p.PURL, "pkg:bitnami") {
+                       p.Type = pkg.BitnamiPkg
+                       // Parse the Bitnami-specific metadata
+                       metadata, err := parseBitnamiPURL(p.PURL)
+                       if err != nil {
+                               return nil, nil, err
+                       }

-               p.Metadata = metadata
+                       p.Metadata = metadata
+               }

                pkgs = append(pkgs, p)
        }

... Duplicates appear:
$ go run ./cmd/syft bitnami/apache -o json | jq '.artifacts[] | select(.purl | startswith("pkg:golang/github.com/jessevdk/go-flags"))'

{
  "id": "15bd1508bd27b64e",
  "name": "github.com/jessevdk/go-flags",
  "version": "v1.6.1",
  "type": "go-module",
  "foundBy": "bitnami-cataloger",
  "locations": [
    {
      "path": "/opt/bitnami/common/.spdx-render-template.spdx",
      "layerID": "sha256:6923ab12004885c8d94bdd17626e36e661ddc6f2b159cb48bbfe3681dda3dd0a",
      "accessPath": "/opt/bitnami/common/.spdx-render-template.spdx",
      "annotations": {
        "evidence": "primary"
      }
    }
  ],
  "licenses": [],
  "language": "go",
  "cpes": [
    {
      "cpe": "cpe:2.3:a:jessevdk:go-flags:v1.6.1:*:*:*:*:*:*:*",
      "source": "syft-generated"
    },
    {
      "cpe": "cpe:2.3:a:jessevdk:go_flags:v1.6.1:*:*:*:*:*:*:*",
      "source": "syft-generated"
    }
  ],
  "purl"<EMAIL_ADDRESS>  "metadataType": "go-module-buildinfo-entry",
  "metadata": {
    "goCompiledVersion": "",
    "architecture": ""
  }
}
{
  "id": "2e09194e80f282d7",
  "name": "github.com/jessevdk/go-flags",
  "version": "v1.6.1",
  "type": "go-module",
  "foundBy": "go-module-binary-cataloger",
  "locations": [
    {
      "path": "/opt/bitnami/common/bin/render-template",
      "layerID": "sha256:6923ab12004885c8d94bdd17626e36e661ddc6f2b159cb48bbfe3681dda3dd0a",
      "accessPath": "/opt/bitnami/common/bin/render-template",
      "annotations": {
        "evidence": "primary"
      }
    }
  ],
  "licenses": [],
  "language": "go",
  "cpes": [
    {
      "cpe": "cpe:2.3:a:jessevdk:go-flags:v1.6.1:*:*:*:*:*:*:*",
      "source": "syft-generated"
    },
    {
      "cpe": "cpe:2.3:a:jessevdk:go_flags:v1.6.1:*:*:*:*:*:*:*",
      "source": "syft-generated"
    }
  ],
  "purl"<EMAIL_ADDRESS>  "metadataType": "go-module-buildinfo-entry",
  "metadata": {
    "goCompiledVersion": "go1.22.7",
    "architecture": "arm64",
    "h1Digest": "h1:Cvu5U8UGrLay1rZfv/zP7iLpSHGUZ/Ou68T0iX1bBK4=",
    "mainModule": "github.com/bitnami/render-template"
  }
}

As you can see there are two packages with different "id" and "foundBy" values but almost identical in the rest of fields, except for metadata which is richer on the package reported by "go-module-binary-cataloger".

Friendly reminder ⬆  @willmurphyscode

Thanks for the ping @juan131!
I want to have a bit of a discussion about deduplication with the binary classifier:
❯ go run ./cmd/syft -q -o json bitnami/postgresql:17 | jq -c '.artifacts[] | select(.name | test("postgres")) | { name: .name, version: .version, location: .locations[0].path }'
{"name":"postgresql","version":"17.0","location":"/opt/bitnami/postgresql/bin/postgres"}
{"name":"postgresql","version":"17.0.0-8","location":"/opt/bitnami/postgresql/.spdx-postgresql.spdx"}

I wonder if it would be preferable or possible to collapse these. The bitnami entry has better version data, and the versions are "compatible" in the sense that bitnami just knows more about the fact that it's postgresql 17.0. I think ideally these would deduplicate, but today they don't because (I believe) the binary classifier sees a slightly less specific version number (it's essentially doing strings | grep looking for version numbers).
CC @wagoodman

I think Bitnami should be considered a distro provider like Redhat or Debian, here -- Grype will be using the distro's vulnerability feed and should be matching with the same versioning scheme because we think the information provided to the distro package is the most accurate.
We are deduplicating overlapping packages by owned files based on cataloger priority -- if the bitnami entries are cataloged with a specific type, I think the appropriate type just need to get added here and Syft should deduplicate the packages as long as there is an accurate overlap in the owned files from the SPDX entry.

I think Bitnami should be considered a distro provider like Redhat or Debian, here -- Grype will be using the distro's vulnerability feed and should be matching with the same versioning scheme because we think the information provided to the distro package is the most accurate.

Pragmatically I think this is the right short term answer in terms of grype's needs.
In terms of a more accurate SBOM, we could add a post-cataloging task that looks for package ownership overlap with packages found with the SBOM cataloger and keep more authoritative packages (bitnami over anything else for instance).

You're the experts here, I can adapt https://github.com/anchore/syft/pull/3341 based on your feedback

Hi @juan131! Thanks for your patience here.
Running #3341 right now on bitnami/postgresql looks like this:
go run ./cmd/syft -q  bitnami/postgresql | grep -e NAME -e postgres
NAME                    VERSION                  TYPE
postgresql              17.1                     binary
postgresql              17.1.0-0                 bitnami

I have a couple questions about this:

What is the version format for 17.1.0-0? (It looks like sem ver with -0 on the end)
Is there location data in the Bitnami SPDX (I didn't see any) or something else that would help us dedupe the two postgres packages that appear here?

In order to merge #3341, we'd like to find a way of collapsing these packages into one package. Right now, by default, if Syft finds for example an RPM that owns the postgres executable and a binary package at the postgres executable, it will, by default, collapse them into one package, deleting the binary detected in favor of the RPM and its richer metadata.
We'd like to do something similar with the bitnami SBOMs, but I don't see any reference to the path in the SDPX bitnami puts in the image, so we're not automatically inferring that the postgres executable path is owned by the bitnami package.
I see that /opt/bitnami/postgresql/bin/postgres is the path to the postgres binary. Is this path format predictable and stable? Maybe we could use it to deduplicate with the binary cataloger?

Hi @willmurphyscode
Regarding locations, it's true the SPDX file doesn't include information about the exact location where Bitnami packages (those with pURLs prefixed with pkg:bitnami) are. I remember that, when we implemented this on Trivy, we assumed that if a spdx file was found under /opt/bitnami/COMPONENT_NAME folder then the location was /opt/bitnami/COMPONENT_NAME for every Bitnami package included in the SBOM, see:

https://github.com/aquasecurity/trivy/blob/main/pkg/fanal/analyzer/sbom/sbom.go#L79-L85

What is the version format for 17.1.0-0? (It looks like semver with -0 on the end)

That's the revision, please find at [bitnami/go-version|https://github.com/bitnami/go-version] our versioning explained. That's why we use this library for version comparison.

Hi @juan131,
That solution sounds reasonable to me - we already have mechanism for de-duplicating binary packages in favor of OS packages when there's file overlap, and we should use them here.
For example if someone does yum install curl, then the package like pkg:rpm/curl will own the binary file /usr/bin/curl. Syft uses this relationship to remove the binary package curl in favor of the RPM curl, because it will have richer metadata. (This is configurable.) We'd like for #3341 to be changed so that this same configuration would cause the binary package pkg:generic/postgres to be deduplicated in favor of pkg:bitnami/postgresql.
This has a couple of steps:

The Bitnami package struct's Metadata field needs to implement the FileOwner interface, see https://github.com/anchore/syft/blob/main/syft/pkg/file_owner.go#L8 and  https://github.com/anchore/syft/blob/main/syft/pkg/rpm.go#L57
The actual cataloger needs to set files on the Metadata so that these can be access later when OwnedFiles is called, see https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/redhat/parse_rpm_db.go#L69-L82 as an example
https://github.com/anchore/syft/blob/main/internal/relationship/exclude_binaries_by_file_ownership_overlap.go#L45-L69 needs to be updated to identify relationships where the parent package is of type bitnami and the child package is of type binary.

Step 2 is where the logic you're describing lives, which I understand to mean:
In the cataloger, if Syft finds an spdx SBOM at /opt/bitnami/COMPONENT/.spdx-COMPONENT.spdx, then we should find in that SBOM a package with a purl like pkg:bitnami/COMPONENT, then we emit a package there named COMPONENT with a purl like pkg:bitnami/COMPONENT that owns all the files under /opt/bitnami/COMPONENT. If this is the logic you mean, it makes sense to us.
As a test case, when running Syft with this change on bitnami/postgresql:latest, Syft should emit a package with a PURL like pkg:bitnami/postgresql but not a binary package called postgres.


ParentDirectory related messages

There're lots of messages in the form:
ParentDirectory::dirnex:
   Directory Users/ppavel/Projects/wolfram-engine-app-server/paclet/Kernel/App
     Server does not exist.

ParentDirectory::nums:
   Argument ParentDirectory[Users/ppavel/Projects/wolfram-engine-app-server/pa
      clet/Kernel/AppServer] should be a positive machine-size integer, a
     nonempty string, or a File specification.

ParentDirectory::nums:
   Argument ParentDirectory[ParentDirectory[Users/ppavel/Projects/wolfram-engi
       ne-app-server/paclet/Kernel/AppServer]] should be a positive
     machine-size integer, a nonempty string, or a File specification.

ParentDirectory::nums:
   Argument ParentDirectory[ParentDirectory[ParentDirectory[Users/ppavel/Proje
        cts/wolfram-engine-app-server/paclet/Kernel/AppServer]]] should be a
     positive machine-size integer, a nonempty string, or a File
     specification.

General::stop: Further output of ParentDirectory::nums
     will be suppressed during this calculation.

It seems like absolute path is treated as relative. I don't know how to find the reason for this behaviour

Interesting, I've never try it on MacOS, so the directory lookup here might be a little buggy.
Could you share the path of the project that you opened (if not a privacy issue)? Since the log is complaining that Users/ppavel/Projects/wolfram-engine-app-server/paclet/Kernel/AppServer doesn't exist, I guess, when converting from the URI that VSCode recognizes to the real pathname, there was something wrong.

Could you share the path of the project that you opened (if not a privacy issue)? Since the log is complaining that Users/ppavel/Projects/wolfram-engine-app-server/paclet/Kernel/AppServer doesn't exist, I guess, when converting from the URI that VSCode recognizes to the real pathname, there was something wrong.

This is really the full path to the project. /Users is macOS analog to /home directory in other flavours of unix-like OSs.
So the path to the folder is indeed /Users/ppavel/Projects/wolfram-engine-app-server/paclet/Kernel/AppServer and the file in this folder is open in VSCode

I see, could you try the latest commit on develop branch (https://github.com/kenkangxgwe/lsp-wl/commit/f01f55ba6a136b46a9c313499c23909047b31edc) to see if the issue is resolved?

I see, could you try the latest commit on develop branch (f01f55b) to see if the issue is resolved?

I tried the latest commit (27f274b87128ab3cc902e0199c4fbd1ae53d9d1e) and indeed it seems like fixed the issue.
You're cool! :)


adding autoconfiguration for micrometer azuremonitor registry

Summary

This PR introduces Auto Configuration for enabling Azure Monitor Meter Registry for Micrometer (https://github.com/micrometer-metrics/micrometer/pull/806)
It introduces a new starter for metrics called - azure-spring-boot-metrics-starter (which will assist in enabling auto configuration for Azure Monitor Meter Registry to collect Pre Aggregated metrics to AzureMonitor).
Issue Type

New Feature

Starter Names
Adds a new starter
- auzre-spring-boot-metrics-starter
Additional Information
Currently the dependencies rely on snapshots published by Micrometer (https://repo.spring.io/libs-snapshot/io/micrometer/micrometer-registry-azure-monitor/)  as the official next release of Micrometer would be by late sept to early oct.
Few Concerns (Please do not merge PR yet):

I need the help from owners of the repository in proper dependency management. Seems like even after explicitly mentioning micrometer-core:1.1.0-SNAPSHOT when I run mvn clean package install and use the snapshots produced in test application I still get micrometer-core:1.0.5 included which doesn't work with the implementation as it has certain classes of Micrometer core API which were introduced after 1.0.5 version.
I  also tried explicitly excluding micrometer-core from any other packages which might have it as transitive dependency but still no luck.
I believe that the spring-boot-dependencies file has micrometer-core:1.0.5 included there which always somehow takes priority even after explicitly mentioning the 1.1.0-SNAPSHOT version to be included.

@Incarnation-p-lee  and @sophiaso  could you please help me out here?
@grlima @littleaj  just FYI please do start taking a look!

@jkschneider just FYI and if you would like to take a brief look at this porting for auto config in Boot 2.x

FYI build is also failing because of all the above mentioned questions.

@sophiaso the build also now passes. However somehow I still get Micrometer-core:1.0.5 when I do mvn clean package install and use the locally built azure-spring-boot-metrics-starter in one of the sample apps to test the scenario E2E. I am suspecting this is because of the spring-boot-dependencies which are imported and take the precedence. Looking at that pom file I can see it has 1.0.5 version of Micrometer core.

@dhaval24  I still get Micrometer-core:1.0.5 when I do mvn clean package install, could you share under which module you built get the 1.0.5 version? Just tried mvn dependency:tree under azure-spring-boot-metrics-starter(removed the explicit version), there is no problem with the version, see below:
[INFO] +- io.micrometer:micrometer-registry-azure-monitor:jar:1.1.0-SNAPSHOT:compile
[INFO] |  +- com.microsoft.azure:applicationinsights-core:jar:2.0.1:compile
...omitted
[INFO] +- io.micrometer:micrometer-core:jar:1.1.0-SNAPSHOT:compile

@sophiaso thanks for the tip. I removed the explicit version and I think now it works fine.

@sophiaso thanks for approving the PR. I have made all the required changes you asked for too! @grlima and @littleaj could you take a quick glance whenever you get some time?
Further, @sophiaso what is the expected time frame the code would be merged in master and do you plan to take this in the current release or the release after this one?


[kdeplasma] Add 5.27

Release Notes https://kde.org/announcements/plasma/5/5.27.0/
Marked as LTS on https://community.kde.org/Schedules/Plasma_5
5.28 not yet planned on https://phabricator.kde.org/calendar/query/all/

Looks like we created a PR at the same time : #2509. Feel free to close whichever you like best ;)


Can I config my liquibase.properties with envionment variables

Environment
Liquibase Version:
4.0.0
Liquibase Integration & Version: <Pick one: CLI, maven, gradle, spring boot, servlet, etc.>
docker
Liquibase Extension(s) & Version:
Database Vendor & Version:
Operating System Type & Version:
Description
This is not a bug,But I don't know where to ask.
I wrote down the specific problem here
https://github.com/liquibase/docker/issues/36
Please pay attention to this suggestion.Tks
┆Issue is synchronized with this Jira Bug by Unito

Hi @dazuimao1990 Thanks for asking the question.  There is a feature request to do that in  #1179 that has not been addressed yet.  Please add your vote to that one.


ManagedClient 3.0.14 file not found

Describe the bug
A clear and concise description of what the bug is.
Which project is your bug related to?

ManagedClient

To Reproduce
Steps to reproduce the behavior:
System.IO.FileNotFoundException: Could not load file or assembly 'MQTTnet.Extensions.ManagedClient, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b69712f52770c0a7'. The system cannot find the file specified

file version is <IP_ADDRESS>

I'm currently struggling with the same issue. It seems that only for Net5.0 the version of the generated dll is configured wrong (<IP_ADDRESS> instead of <IP_ADDRESS>). The dlls for other runtimes are fine.

@janzenisek me too

It turns out that the build script was not properly updated for .net 5.0. I changed this. Please test RC1 of 3.0.15. @JanEggers Please let me know if the settings are fine for the ASP.NET parts.

This should be done with 3.0.15-rc2.

Same issue here. With .NET Core 3.1


Milestone :Group capstone project set up your Kanban board

Link to GitHub projects
https://github.com/mrprotocoll/kanban-capstone-project/projects/1
Partners involved
Number of partners
2
Partners
@mrprotocoll - Student A
@Lancelot-SO  - Student B

Hello @Lancelot-SO  and @mrprotocoll      :wave: ,
I'm Nemwel your reviewer for this PR :man_beard:
Good job working on your project so far! :clap:
There are some issues that you still need to work on to go to the next project but you are almost there!

Highlights
Great job setting up the correct linters ✔
Professional documentation ✔
Correct workflow ✔
Meaningful commit messages ✔
♻ Required Changes ♻

[ ] Kindly ensure that for the tasks labeled as group tasks, you have assigned them to both of you and not to one person.

This is how your kanban board looks like:

Check the comments under the review.
[Optional] suggestions
Every comment with the [OPTIONAL] prefix is not crucial enough to stop the approval of this PR. However, I strongly recommend you to take them into account as they can make your code better.
Cheers and Happy coding! :partying_face: 👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me @Nemwel-Boniface in your question so I can receive the notification.
Please, do not open a new Pull Request for re-reviews. You should use the same Pull Request submitted for the first review, either valid or invalid unless it is requested otherwise.

As described in the Code reviews limits policy you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.

Thank you for the review
Below is a screenshot of the requested corrections

Hi @Lancelot-SO and @mrprotocoll  ,
Your project is complete!  There is nothing else to say other than... it's time to merge it :shipit:
Congratulations! 🎉

Highlights
Kanban board setup :heavy_check_mark:
Optional suggestions
Every comment with the [OPTIONAL] prefix won't stop the approval of this PR. However, I strongly recommend you to take them into account as they can make your code better. Some of them were simply missed by the previous reviewer and addressing them will really improve your application.
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.

As described in the Code reviews limits policy  you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.


1.9.1 train 171

#1530
#1574
#1592
#1597
#1598

test_octarine failed in https://teamcity.mesosphere.io/viewLog.html?buildId=669843. I restarted it and  updated DCOS-16020.

test_upgrade_vpc.marathon_app_tasks_survive_upgrade failed in https://teamcity.mesosphere.io/viewLog.html?buildId=669837. That's probably already tracked, via https://jira.mesosphere.com/browse/DCOS_OSS-1065.

Test failures:
test_networking.test_if_overlay_ok
 test_networking.test_ip_per_container
 test_service_discovery.test_if_service_discovery_works_docker_bridged_network
 test_service_discovery.test_if_service_discovery_works_docker_host_network

(https://teamcity.mesosphere.io/viewLog.html?buildId=669847)
Not funny. Created https://jira.mesosphere.com/browse/DCOS_OSS-1195 with blocker status, needs to be triaged.

The failure of test_if_ucr_app_runs_in_new_pid_namespace is tracked via https://jira.mesosphere.com/browse/DCOS_OSS-862. Restarting.

@mesosphere-mergebot bump-ee

bump-ee has been fixed to work across all branches. So, the corresponding enterprise PRs (if any) can be added to this mesosphere/dcos-enterprise#985

@mesosphere-mergebot bump-ee

Just FYI: the previous event was not triggered by Github API. I am not triggering again as the upstream bump was already done.


Removed support for JavaMarshaller as default marshaller

Changes in this PR:

Removed support for JavaMarshaller as default marshaller
Moved JavaMarshaller implementation to test sources for IT testing
New default marshaller supplier will throw an exception if the code falls to where the JavaMarshaller was returned before to inform developer of the change
Updated several tests to manually provide a marshaller supplier in order to run

This is a backward incompatible change.

This resolves the removal of JavaMarshaller as requested in https://github.com/oracle/helidon/issues/4367