maxUrls config not honored

I have tried the crawler and everything runs fine, except that the maxUrls parameter does not seem to get honored correctly. Admittedly, I set it to a rather low value of 10K. Is there something I am missing?

It should honor it. How did you set the maxUrlsPerSchemeAuthority parameter?


comfyui won't start due to :rshared

Not sure what the purpose of rshared is in the comfy ui config -- multiple instances?  I got the following error with it:
$ harbor up comfyui
WARN[0000] Found orphan containers ([harbor.plandex-server harbor.plandex-db harbor.plandex]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
[+] Running 2/3
 ⠧ Container harbor.comfyui  Starting                                                                                1.1s
 ✔ Container harbor.ollama   Running                                                                                0.0s
 ✔ Container harbor.webui    Started                                                                                0.9s
Error response from daemon: path /mnt/nvme1/home/lb/AI/harbor/comfyui/workspace is mounted on /mnt/nvme1 but it is not a shared mount

This patch fixes it:
diff --git a/compose.comfyui.yml b/compose.comfyui.yml
index 895508f..d97cd43 100644
--- a/compose.comfyui.yml
+++ b/compose.comfyui.yml
@@ -33,7 +33,7 @@ services:
       - ${HARBOR_COMFYUI_PORTAL_HOST_PORT}:${HARBOR_COMFYUI_PORTAL_HOST_PORT}
       - ${HARBOR_COMFYUI_SYNCTHING_HOST_PORT}:${HARBOR_COMFYUI_SYNCTHING_HOST_PORT}
     volumes:
-      - ./comfyui/workspace:/workspace:rshared
+      - ./comfyui/workspace:/workspace
       # - ./comfyui/storage:/storage
     networks:
       - harbor-network

But may break some other goal.

Hmm. This may be a red herring, warning, not fatal error.  Seems it won't ACTUALLY start regardless.  Confirming.

Thanks for reporting nonetheless, this was a leftover from the initial testing, I'm removing it in the upcoming release

Closing for now, feel free to follow-up if needed


Mint client will try to pay a paid invoice

% cargo run --release --bin mint-client-cli federations/testfed ln-pay <invoice>
{
  "ln_pay": {
    "contract_id": "c6bf42c2bd0afe5c9c51a2a179cb2494f2ef11afcad88974c14b916d478ad893"
  }
}

% cargo run --release --bin mint-client-cli federations/testfed info
{
  "info": {
    "total_amount": 929320,
    "total_num_notes": 25,
    "details": {
      "10": 2,
      "10000": 2,
      "100": 3,
      "1000": 9,
      "100000": 9
    }
  }
}

% cargo run --release --bin mint-client-cli federations/testfed ln-pay <invoice>
{
  "ln_pay": {
    "contract_id": "5d865102339f51aaba1a08a8109413589b3806463186b229398f07b330f47d27"
  }
}

% cargo run --release --bin mint-client-cli federations/testfed info
{
  "info": {
    "total_amount": 919220,
    "total_num_notes": 23,
    "details": {
      "10000": 1,
      "100000": 9,
      "100": 2,
      "10": 2,
      "1000": 9
    }
  }
}

From @elsirion: client could just keep track of invoices of every paid invoice or invoice which has a pending payment currently. This is a bit of a hack because the server doesn't enforce it. Would be better if server didn't let you try to pay a paid invoice.

I think client side filtering is reasonable since it doesn't put any additional index burdens on the federation and invoices should generally only be given to a single person. If the GW can't pay the invoice (because a third party already did) the normal refund logic should apply (although you messed up already and nothing is guaranteed since someone got the preimage already).

I won't be able to look at this until possibly next week so unassigning from myself for now in case someone else wants to work on it in the meanwhile.

I've assigned this back to myself now.
@justinmoon I was able to reproduce this, although I encountered a timeout error on the second ln-pay. I ran the following steps after using just tmuxinator:
$ fedimint-cli ln-invoice 100 "foo"
{
  "invoice": "<invoice>"
}

$ fedimint-cli info
{
  "federation_id": "b7e545d8330f15b616b17155b61f9b3f07190910a7fbc24197d1082ca4e53b118247ef191d195dba1859ee13f3126c0d",
  "network": "regtest",
  "meta": {
    "federation_name": "Hals_trusty_mint"
  },
  "total_amount": 10000000,
  "total_num_notes": 52,
  "details": {
    ...
  }
}

$ fedimint-cli ln-pay <invoice>
{
  "contract_id": "28f42e15f4419a1b110d2003ce68325cd90bef50f07a8884dac5ad439d024f31"
}

$ fedimint-cli info
{
  "federation_id": "b7e545d8330f15b616b17155b61f9b3f07190910a7fbc24197d1082ca4e53b118247ef191d195dba1859ee13f3126c0d",
  "network": "regtest",
  "meta": {
    "federation_name": "Hals_trusty_mint"
  },
  "total_amount": 9999899,
  "total_num_notes": 48,
  "details": {
    ...
  }
}

$ fedimint-cli ln-pay <invoice>
{
  "error": "CliError",
  "kind": "GeneralFederationError",
  "message": "gateway failed to execute contract",
  "raw_error": "Outgoing payment timeout"
}

$ fedimint-cli info
{
  "federation_id": "b7e545d8330f15b616b17155b61f9b3f07190910a7fbc24197d1082ca4e53b118247ef191d195dba1859ee13f3126c0d",
  "network": "regtest",
  "meta": {
    "federation_name": "Hals_trusty_mint"
  },
  "total_amount": 9999798,
  "total_num_notes": 44,
  "details": {
    ...
  }
}

For client-side filtering, will additional data need to be saved to the DB or will it be enough to use existing data? E.g. in the function create_outgoing_output an OutgoingPayment entry is inserted for the contract so perhaps this could be used for filtering if we can relate it back to the invoice.

... perhaps this could be used for filtering if we can relate [OutgoingPayment] back to the invoice.

One nuance here is that we want to be able to try an outgoing payment with gateway A, and if it fails retry with gateway B. So if we just used the OutgoingPaymentKey then we wouldn't know if that previous attempt succeeded or failed unless we update that key in the database after the payment succeeds or fails. Another problem here is that it doesn't seem like we can produce a contract_id from a given invoice, so I don't think we could look that key up currently.
So somehow we'll need a mapping from "invoice" -> "have we already paid?"
Ideally we could store a payment history and then look this information up there. But designing that API is kind of a hairy task. It's sort of a modularization question, too. Would we want to be able to produce some kind of payment history across all modules? Not sure how feasible that is ...
A simpler alternative would be to just have a key prefix where we store payment hashes of invoices we've paid. Then we can just check whether an invoice's payment hash is in there every time we try to pay an invoice, and refuse if it is.
Make sense?

OK thanks for explaining that. Yes I think the simpler alternative makes sense. I'll give it a try and will let you know if I have any further questions (I'll probably not have much time to work on this for another week or so).

Is this required for the alpha release?

In case you'd like this included in the alpha release, please note I might need some feedback on my draft PR before I can proceed further. Thanks.

I was just talking to @elsirion, and it sounds like this will be fixed in the new client automatically by using the payment hash as operation id: https://github.com/fedimint/fedimint/blob/77276886d8cbd3787d3dc941c9a1616e8a6c85d3/fedimint-client/src/lib.rs#L349-L351
Let's just close this as "won't fix" and focus on the new client for the alpha release.


[V1alpha2] Adding unit tests with mocks for baremetalcluster controller

Use interfaces for the managers
add mocks for the baremetalcluster manager
add unit tests using the mocks for the baremetal cluster controller
expand test coverage for baremetalcluster manager

/lgtm


As a CIF user, I want to delete attachments uploaded to a project, so that I can remove unneeded or duplicate attachments

Description:
There is currently no way to remove attachments once they are uploaded to a project. A delete button should be implemented to allow users to delete attachments.
Acceptance Criteria:
Given I am viewing the attachments page
When I want to remove an attachment
Then I see a delete button for each attachment
Given I have uploaded an attachment
When I click the delete button for that attachment
Then the attachment is removed from the project
Development Checklist:

[ ] Add a delete button to the AttachmentTableRow component
[ ] On click, it should update the corresponding attachment in the database to be 'archived'
[ ] The graphql query on the attachments page should filter out archived records
[ ] Meets the DOD

Definition of Ready (Note: If any of these points are not applicable, mark N/A)

[x] User story is included
[x] User role and type are identified
[x] Acceptance criteria are included
[ ] Wireframes are included (if required)
[ ] Design / Solution is accepted by Product Owner
[ ] Dependencies are identified (technical, business, regulatory/policy)
[ ] Story has been estimated (under 13 pts)

·Definition of Done (Note: If any of these points are not applicable, mark N/A)

[ ] Acceptance criteria are tested by the CI pipeline
[ ] UI meets accessibility requirements
[ ] Configuration changes are documented, documentation and designs are updated
[ ] Passes code peer-review
[ ] Passes QA of Acceptance Criteria with verification in Dev and Test
[ ] Ticket is ready to be merged to main branch
[ ] Can be demoed in Sprint Review
[ ] Bugs or future work cards are identified and created
[ ] Reviewed and approved by Product Owner

Hey team! Please add your planning poker estimate with Zenhub @BCerki @gurjmatharu @mikevespi @sam-warren @tmastrom

@mikevespi - awaiting your estimate on this one :)

Thank you @sam-warren! Tested in dev and I was able to upload and delete a file related to a project. Thank you for your work on this one :)


Can we implement memory as a Powdr machine?

I tried rewriting the write-once memory as a Powdr machine:
machine Main {

    degree 256;

    WriteOnceMemory memory;

    reg pc[@pc];
    reg X[<=];
    reg Y[<=];
    reg A;

    instr mload X -> Y = memory.access
    instr mstore X, Y -> = memory.access

    instr assert_eq X, Y { X = Y }

    function main {
        mstore(17, 42);
        A <== mload(17);
        assert_eq A, 42;

        return;
    }
}

machine WriteOnceMemory {

    operation access ADDR -> v;

    let ADDR = |i| i;
    let v;
}

These are all the errors:

This currently fails with: Machine  should have a latch column as it does not have a pc and has operations
Even if I add a latch with let latch = |i| 1;, it complains about mstore having the wrong signature. I could add more operations, but then I would need an operation ID?
Even if I remove mstore, witgen does not detect the machine as WriteOnceMemory because it has a latch.
If I fix that, it works!

I think these are symptoms of the abstraction not really working out for this use-case, so it could be a good case study for improving them!

Maybe done in https://github.com/powdr-labs/powdr/pull/1129 ?

No, I just tried and it looks like all of the issues above are still there.


Remove deprecated connector config fields

The first part of https://issues.redhat.com/browse/DBZ-5045

DB2 tests should work once debezium-connector-db2 PR #55 is merged.

@vjuranek LGTM. The grep -ir of string deprecated found few more instances in the code and docs. Could you please doublecheck it?

Thanks for the review @jpechane , you are right, found two more options in Mongo DB (removed), one obsolete option value and one more deprecated value in Oracle DB (not removed yet, not sure if it can be removed - discussing it with @Naros first).
Rest of the deprecated code are mostly methods - should they be removed as well as part of this issue?
Another part is deprecated JXM metrics - these are handled in PR #3571.
Fix for DB2 is here, for Cassandra here. I didn't find any other deprecated config options.

@vjuranek Nice cleanup!


Supertraits: Require a functionnality in another trait

Describe how one can require a trait to depend on another trait.
Rust equivalent: https://doc.rust-lang.org/book/ch19-03-advanced-traits.html#using-supertraits-to-require-one-traits-functionality-within-another-trait
This section can demonstrate this feature by using an example requiring the implementation of PrintTrait.
Example snippet associated:
use debug::PrintTrait;

trait C {
    fn foo();
}

impl B of C {
    fn foo() {
        'foo'.print();
    }
}

trait A<impl B: C> {
    fn bar();
}

impl CompositeImpl of A<B> {
    fn bar() {
        B::foo();
        'bar'.print();
    }
}

fn main() {
    A::bar();
}

Estimated time: 1hr

Note that there is currently a bug in Cairo (https://github.com/starkware-libs/cairo/issues/2995) that means this won't work with generic implementations.
I'm also fairly sure this should be writable as impl CompositeImpl of A { without specifying <B>, but it's crashing as well.

do we need intermediate impls or can we just do

trait A {
...
}

trait B<A> {
...
}

trait C<B> {
...
}

I would propose this example which is similar to the one in Rust doc:
use core::fmt::{Display, Formatter, Error};

#[derive(Copy, Drop)]
struct Point {
    x: u8,
    y: u8
}

impl PointDisplay of Display<Point> {
    fn fmt(self: @Point, ref f: Formatter) -> Result<(), Error> {
        let str: ByteArray = format!("({}, {})", *self.x, *self.y);
        f.buffer.append(@str);
        Result::Ok(())
    }
}

trait OutlinePrint<T, +Display<T>> {
    fn outline_print(self: @T);
}

impl PointOutlinePrint of OutlinePrint<Point> {
    fn outline_print(self: @Point) {
        println!("**********\n* {}\n**********", *self);
    }
}

fn main() {
    let p = Point { x: 1, y: 3};
    p.outline_print();
}

Where do you want to add this section ? There is a quite similar section in "Generic Data Types" which explains how to constraint generic types of functions to implement a specific behaviour but I guess it would be better to add this new section in "Traits in Cairo".
What do you think ?

this is using the trait bound syntax. In Cairo generic traits are always linked to a type so I don't think the concept can be applied correctly

this is using the trait bound syntax. In Cairo generic traits are always linked to a type so I don't think the concept can be applied correctly

Yes, I would say that as a trait implementation is always linked to a type, you can only constraint the type to  have implemented a trait, right ? As explained for functions in the chapter "Generic Data Types".
In the code snippet you gave as example, for me there is no link between CompositeImpl and B, and I think you can just write the same example as:
trait C {
    fn foo();
}

impl B of C {
    fn foo() {
        println!("foo");
    }
}

trait A {
    fn bar();
}

impl CompositeImpl of A {
    fn bar() {
        B::foo();
        println!("bar");
    }
}

So yes, I'm not sure we can contraint a trait to depend on another trait, but I have still a lot of things to learn in Cairo 😅

At the time I opened this issue I'm not sure trait bounds were already something 😄 maybe we should close this

At the time I opened this issue I'm not sure trait bounds were already something 😄 maybe we should close this

Yep, we can close it and create another task in the future if this kind of concept appear in a future Cairo version :)


adding tiny-use-media, tiny-use-detect-horizontal-overflow

Adding two tiny react dependent libraries

Handy ones. I think we'll need a special category for react hooks, they're often quite small.
I see the packages don't meet the star limit but have weekly downloads, maybe I'll adjust the rules to allow that.
Also, I'd really love to see unit tests.
I'm fine with adding to WIP, done


Updating pkg to latest for 0.7 release.

Proposed Changes

Prepping for 0.7 release. Update knative/pkg to latest from branch release-0.7.

Release Note
NONE

/cc @mattmoor

I ran update codegen and nothing changed. @mattmoor

IDK that we're going to release this (yet), so skipping this.


Recent changes in OGC API - Records conflict with this extension

See https://github.com/opengeospatial/ogcapi-records/pull/211#issuecomment-1500323399
The default value of dir differs (and also the allowed values).

@m-mohr any new updates on that ?

Was reverted, read https://github.com/opengeospatial/ogcapi-records/pull/211#issuecomment-1500323399 for details


CEF-AMA Setup

Required items, please complete
Change(s):

See guidance below

Reason for Change(s):

See guidance below

Version Updated:

Required only for Detections/Analytic Rule templates
See guidance below

Testing Completed:

See guidance below

Checked that the validations are passing and have addressed any issues that are present:

See guidance below

Guidance <- remove section before submitting

Before submitting this PR please ensure that you have read the following sections and filled out the changes, reason for change and testing complete sections:
Thank you for your contribution to the Microsoft Sentinel Github repo.

Details of the code changes in your submitted PR.  Providing descriptions for pull requests ensures there is context to changes being made and greatly enhances the code review process.  Providing associated Issues that this resolves also easily connects the reason.

Change(s):

Updated syntax for XYZ.yaml

Reason for Change(s):

New schema used for XYZ.yaml
Resolves ISSUE #1234

Version updated:

Yes
Detections/Analytic Rule templates are required to have the version updated

The code should have been tested in a Microsoft Sentinel environment that does not have any custom parsers, functions or tables, so that you validate no incorrect syntax and execution functions properly.  If your submission requires a custom parser or function, it must be submitted with the PR.

Testing Completed:

Yes/No/Need Help

Note: If updating a detection, you must update the version field.

Before the submission has been made, please look at running the KQL and Yaml Validation Checks locally.
https://github.com/Azure/Azure-Sentinel#run-kql-validation-locally

Checked that the validations are passing and have addressed any issues that are present:

Yes/No/Need Help

Note: Let us know if you have tried fixing the validation error and need help.

References:

Guidance for Detection checks
General contribution guidance
PR validation troubleshooting

Hello @Cyberlorians, Please try to resolve validation errors. Also if this is an function app its missing with function app and azure deploy file.
Please follow this folder structure and files required DataConnectors/AWS-SecurityHubFindings

Hello @Cyberlorians, Please try to resolve validation errors. Also if this is an function app its missing with function app and azure deploy file. Please follow this folder structure and files required DataConnectors/AWS-SecurityHubFindings

There is no function app. This builds upon the current CEF install that is published. We wrote an article on how to install CEF-AMA in Azure Public and Government. I am still hosting the gov.py script on my own GIT. I would like if MSFT would take this and make it published (own). I have no clue how to do any of this. If @Sreedhande wants to take ownership of this, great.
Articles/CEF-AMASetup.md at main · Cyberlorians/Articles (github.com)

Asked Sreedhar to take a look at this.

Hello @Cyberlorians, Code Ql is throwing error Too many parameters for special method __repr__, which has 2 parameters, but should have 1, in class
Please let me know if its ok, else please fix it.

Hello @Cyberlorians, can you please check once Codel QL error.

Hello @Cyberlorians, Please go through the Code QL error once

Hello @Cyberlorians, We know you might be busy onto something. Just wanted to know if you have some updates on the above mentioned comments.

No sorry I haven’t. I don’t understand what to fix because it’s actual code that needs to be there. Sreedhar was supposed to take this over and work on it, I think. Has he?
Get Outlook for iOShttps://aka.ms/o0ukef

From: v-prasadboke @.>
Sent: Wednesday, February 7, 2024 12:10:28 AM
To: Azure/Azure-Sentinel @.>
Cc: Cyberlorians @.>; Mention @.>
Subject: Re: [Azure/Azure-Sentinel] CEF-AMA Setup (PR #9764)
Hello @Cyberlorianshttps://github.com/Cyberlorians, We know you might be busy onto something. Just wanted to know if you have some updates on the above mentioned comments.
—
Reply to this email directly, view it on GitHubhttps://github.com/Azure/Azure-Sentinel/pull/9764#issuecomment-1931295699, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOZFFNFZCVOLPLGOKEINZJ3YSMEEJAVCNFSM6AAAAABB3NDFUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGI4TKNRZHE.
You are receiving this because you were mentioned.Message ID: @.***>

Hi @Cyberlorians,
Given that this pull request has remained in draft for a considerable time, please consider submitting a new request if you would like to continue your contributions. Thanks


Use only std feature for brotli

Closes https://github.com/Nemo157/async-compression/issues/83 by disabling ffi-api feature thanks to https://github.com/dropbox/rust-brotli/pull/48

Nice, I hadn't realised that was actually released.
bors r+

Codecov Report

Merging #139 (4577e83) into prīmum (40f2446) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           prīmum     #139   +/-   ##
=======================================
  Coverage   78.00%   78.00%
=======================================
  Files          92       92
  Lines        2950     2950
=======================================
  Hits         2301     2301
  Misses        649      649

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 40f2446...4577e83. Read the comment docs.


feat:(statistics): Statistics subclient and E2E tests

user secret is being checked in. Can you remove/edit that?

there is no secret being checked in.


Expose capability to create service credentials in root module - as "first class" citizen

Description
Creating service credential is currently exposed in one of the example (at https://github.com/terraform-ibm-modules/terraform-ibm-icd-postgresql/blob/main/examples/complete/main.tf#L125).
However, given that service credential is the primary way to created additional users/password in postgres, it would be desirable to expose this capability directly in the root module via an input variable.
In output, the service credentials json should be exposed. Furthermore, it would be nice to also expose as outputs a few additional variables that exposes some of the key fields from the service credentials to connect (username, password, cert, hostname)
New or affected modules

terraform-ibm-icd-postgresql

By submitting this issue, you agree to follow our Code of Conduct

@vburckhardt do we really want to expose password?

PR merged - hence closing


write hook for message validation in module's router

Right now we have instance for validating messages , for example here, but we're not actually using them.
Validation should happen after authentication but before transaction processing.

Or in the application level router, whatever makes more sense

Maybe we can get a generic implementation of checkTx at the same time, and somehow use that that common step in the application level router.

Do we want to match the same errors and error codes that the:
https://github.com/cosmos/cosmos-sdk/blob/master/types/errors.go#L31-L51

Or in the application level router, whatever makes more sense

You mean adding the validation here, correct?

Maybe we can get a generic implementation of checkTx at the same time, and somehow use that that common step in the application level router.

With the generic version would we prevent custom checkTx handlers?
Otherwise, making a checkTx function that simply performs validation and throws, then, using that in both the checkTx handler and in the routing could work.

Do we want to match the same errors and error codes that the:
not necessairily. We can use the sdk error namespace for now, you can find those errors here. You can make new summand for MessageValidation if you want.

The cosmos sdk has some notion of an antehandler, whose type is something like
type AnteHandler r msg = Member (Error AppError) r => RoutedTx msg -> Sem r ()

For example, this is where things like "msg doesn't has not enough fees" or whatever other checks can also come in.
We might eventually want to go there and make that part of the definition of the Module type, but if you don't see a clever way for now I think we can just ignore custom handlers and just use message validation.
The type of Response.CheckTx makes me think that it's possible to actually run the transaction and rollback the state (note that there is a place for events), so it could be that the only difference between the execution of deliverTx and checkTx is that at checkTx you rollback whatever you just did

The type of Response.CheckTx makes me think that it's possible to actually run the transaction and rollback the state (note that there is a place for events), so it could be that the only difference between the execution of deliverTx and checkTx is that at checkTx you rollback whatever you just did

This is pretty much what happens in Kadenamint (modulo logging).


feat(migrations): allow passing list of classes to migrationsList

closes #6099

I am currently not sure what your convention on testing is.
I would like to add a test for the new syntax but where should I add it and what is the naming conmvention on it?

some tests for this are here
https://github.com/mikro-orm/mikro-orm/blob/master/tests/features/migrations/Migrator.test.ts#L483
you could duplicate that file and use the new approach. same for mongo migrator, either move the changes a level above so they are covered, or add a test specific for mongo.

I added the docs directly to the Importing migrations statically section with a comment. If thats ok.