[BUG][DISTRIBUTED] client may hang if something wrong happens in the scheduler

Describe the bug
If some errors occur in the distributed, the client may hang, #8 and #17 are examples.
To Reproduce
As examples in #8 and #17.
Expected behavior
Either we deliver the error to the client, or we hide the real error and deliver an InternalServerError to the client.
Additional context
We should separate the system error with the user caused error. We should add some tests.

GraphActor.stop_graph will raise KeyError when operand actor hasn't been created.

Add the code below to mars.deploy.local.tests.test_cluster.Test.testMultipleOuputTensorExecute, the stop graph will fail and the client still hang.
with new_session(cluster.endpoint) as session2:
        U_result, s_result = session2.run(U + 1, s + 1)
        U_expected, s_expectd, _ = np.linalg.svd(raw, full_matrices=False)

        np.testing.assert_allclose(U_result, U_expected + 1)
        np.testing.assert_allclose(s_result, s_expectd + 1)


Can we a create a new storage group or a volume from a storage group snapshot.

Hi
I am currently working on the DELL EMC VMax Array and i am exploring how can we create a storage group from and existing snapshot of storage group or can we create a specific volume from a storage group snapshot.
The modify_storage_group_snapshot does provide modification, but to only link/ unlink, name, generation number updates.

Hi Gupta,
Referring you to a test that I think should help.  It creates a target storage group from source storage group using a snapvx.
https://github.com/dell/PyU4V/blob/master/PyU4V/tests/ci_tests/test_pyu4v_ci_replication.py#L123

Hi Helenwalsh,
Thanks, for the reference i'll try that and see if that resolves the purpose.


Feat: UnstructuredFileConverter meta field

Fix: https://github.com/deepset-ai/haystack-core-integrations/issues/241
This pull request add a new optional meta field to add custom metadata to Unstructured documents.
From:
class UnstructuredFileConverter:

    @component.output_types(documents=List[Document])
    def run(self, paths: Union[List[str], List[os.PathLike]]):

To:
class UnstructuredFileConverter:

    @component.output_types(documents=List[Document])
    def run(
        self,
        paths: Union[List[str], List[os.PathLike]],
        meta: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None,
    ):

The implementation is inspired from the handling of metadata in the PyPDFToDocument component from Haystack main repo.
Ping @anakin87

And maybe I should add more test case because all test case only have one file with one metadata dict. Maybe a test case with two files and a meta list of dict

@lambda-science tests are failing because of mypy. Something in the code should be fixed.
Thanks for your PR. I will take a look later or tomorrow...

@lambda-science tests are failing because of mypy. Something in the code should be fixed.
Thanks for your PR. I will take a look later or tomorrow...

No worries, I realized I could use my own fix before it is merged and released with a simple requirement.txt such as:
unstructured-fileconverter-haystack @ git+https://github.com/deepset-ai/haystack-core-integrations@3bc51a788356545dc692183591cf91bdf5391713#subdirectory=integrations/unstructured

That's very cool ! So it's not urgent at all 🌞

I have the feeling that something is off with metadata, because the auto-generated page_number is always the last page with one doc per elem, investigating...

New version: https://pypi.org/project/unstructured-fileconverter-haystack/0.3.0/


Error NO_KIND_SPECIFIED

i truy codeql database analyze <src-database-js> ql/javascript/ql/src/Security/ --format=csv -o test.csv, but i give error
Shutting down query evaluator.
Interpreting results.
A fatal error occurred: Could not process query metadata.
Error was: No query kind specified(NO_KIND_SPECIFIED).

I find a solution, i get check @id and @kind. I see all file .ql in Security folder and they exists.
Can anyone help me!

I think this may be due to the queries in ql/javascript/ql/src/Security/Summaries having somewhat exotic @kinds.  If you only run the queries in the ql/javascript/ql/src/Security/Summaries/CWE-* folders, that should work.
I agree, though, that the error message CodeQL gives is wrong; it shouldn't say "no kind specified" but perhaps something like "unknown kind specified" (and ideally identify which queries are affected). Would you agree, @hmakholm?

The error message definitely needs fixing -- at the very least it needs to reveal which query it thinks lacks a kind. Reopening.

Hmm, I can't move this issue to codeql-cli-binaries because it would be across orgs, so I've filed a fresh copy there and will close this one again.


Rendering issues with pipeline information in 'Jenkins' check

e.g.
https://github.com/jenkinsci/google-oauth-plugin/runs/1864403011
I think multiple failures breaks it, I've seen it in a couple of places

fyi @mrginglymus

Error: null doesn’t look great either!

Potato
potato in potato step, with arguments potato
potato

Potato
```
Hello
```

### `Potato`
potato in `potato` step, with arguments potato
```
potato
```

Potato
```
Hello
```

looks like we need to throw newlines at the problem


Generate Access Logs

Issue: #186
Borderpatrol generates access logs.

Current coverage is 97.04% (diff: 80.95%)

Merging #187 into master will decrease coverage by 0.34%

@@             master       #187   diff @@
==========================================
  Files            37         38     +1
  Lines           994       1014    +20
  Methods         979        997    +18
  Messages          0          0
  Branches         15         17     +2
==========================================
+ Hits            968        984    +16
- Misses           26         30     +4
  Partials          0          0

Powered by Codecov. Last update 60a47f9...b4b1e13

+2


Fix test/recipes/01-test_symbol_presence.t (multiple issues)

On some platforms, the compiler may add symbols that aren't ours and that we should ignore.
They are generally expected to start with a double underscore, and thereby easy to detect.
Furthermore, we find ourselves with a common symbol (type 'C' in the 'nm' output) on some platforms (x86, for example).  They are allowed to be defined more than once, so make this test recipe reflect that.
Fixes #22869
Fixes #22837

Note: we made a bunch of changes in the Arm-related code to try to avoid these duplicate common symbols (e.g. #20558, but also others). Those changes ultimately make it hard for us to fix current issues such as #22159, so this fix is welcome

Quite frankly, common symbols give me the creeps...  I had some trouble once with such a symbol, where some of the different definitions didn't quite match in size (and of course, bit placement).

Why not just use objdump instead and filter out hidden symbols? Or combine both.

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set.  Please review the updates and set the label manually.

The implementation is different in 3.1 and 3.0, so a backport to those will turn up in a separate PR


Trying to snap camera picture on device with keyboard shortcut

I am using scrcpy to remotely take pictures from an MFC app on Windows. Environment is:
Windows 7 SP1 running scrcpy 1.17
MFC app using SendInput() to push mouse and keyboard events to the Android device
Google Pixel 3  smartphone.
scrcpy 1.17
My Windows app calls ::ShellExecute() to start scrcpy. I use ::SendInput() to send mouse and keyboard events.
The Pixel 3 is running the built in Camera app.
I can successfully take photos every couple of sends by using ::SendInput() to send a fake left mouse click on the round "shutter" button on the Camera. I hate doing it this way because the coordinates of the mouse click have to be hardcoded, and when the screen orientation changes, the coordinates are wrong. I would like to use an Android keyboard shortcut instead. I've searched the internet far and wide for the required keystroke. All I've found is CR and "volume up", which is 0xaf. These do nothing. I'm not sure if these are the wrong shortcuts, or if scrcpy is not forwarding the keystrokes.
How do I use a keystroke to click the shutter button?
BTW, scrcpy is a great app!!

"volume up"

Alt+↑
https://github.com/Genymobile/scrcpy#shortcuts

Doesn't work. The keycodes for ALT up are 0x12 0x26, correct?

When I use a real keyboard, ALT up works. So the problem might be my implementation of ::SendInput() Maybe not setting up the INPUT structures correctly. Will have to dig in.... I've used my implementation of SendInput successfully on other apps, not sure what is wrong here.
To call:
vector	keys={VK_MENU,VK_UP};
Hardware.PressKeyCombo(keys);
void CHardware::PressKeyCombo(vector& keys)
{
vector inputs;
INPUT temp={0};
// keys down
temp.type = INPUT_KEYBOARD;
for ( int i=0 ; i<keys.size() ; i++ )
{
temp.ki.wVk = keys[i];
inputs.push_back(temp);
}
// keys up in reverse order
temp.ki.dwFlags = KEYEVENTF_KEYUP;
for ( int i=(int)keys.size()-1 ; i>=0 ; i-- )
{
temp.ki.wVk = keys[i];
inputs.push_back(temp);
}
::SendInput((UINT)inputs.size(),FrontPtr(inputs),(int)sizeof(INPUT));
}

Finally figured it out. Platform is Windows C++
void CScrcpyCameraApp::TakePhoto()
{
keybd_event(VK_MENU,0x38,0,0);
keybd_event(VK_UP,0x48,KEYEVENTF_EXTENDEDKEY,0);
keybd_event(VK_UP,0x48,KEYEVENTF_KEYUP|KEYEVENTF_EXTENDEDKEY,0);
keybd_event(VK_MENU,0x38,KEYEVENTF_KEYUP,0);
}


Add domain name entropy

Based on a char distribution generated from the Cisco Umbrella 1 million, calculate the relative entropy of a domain name

added in tag v2.1.0


Persistence on the browser....

Hey guys,
So I have had a little play around with Dashport and I feel like I am missing something fundamental, which is how do we persist logins between requests? I see that my Google auth strategy works, I come back to the redirect and the the serialisation/deserialisation has done it's thing but I don't seem to have anything on the browser side - that means the next request appear unauthorised... Am I missing something here? Do I have to implement cookies/headers whatever myself?
Cheers team!

Hey Royle!
Our session manager uses the browser's http object (by way of oak's ctx) to store your user's serialized id. It also stores that id locally. Authenticate, which should be getting called when trying to access a protected page, then checks that context object against the local id and bypasses the login process if there's a match.
Everything is functioning fine on my end so I'm drawing a blank on why it might be giving you issues. I'll do some more playing around, but if you could provide me with any more info about your setup that'd be greatly appreciated!
Since a couple people seem to be having issues, I'd be happy to hop on a zoom for 30 mins to troubleshoot to make sure we can resolve this issue and so I can better answer problems like yours in the future. Lmk your availability if interested.

Hi @sportelance I would be more than happy to have a chat. I will get back to you on times. I have done a little snooping and it seems that you are using the context.state from oak to add the session property which has the serialised ID - what I am not finding is anywhere that says that this is the right way to do it in oak - their documentation is a little poor. I set up a simple test - I added a simple middleware that logs the context.state calls next and then logs the context.state again. The theory being that the first call shows the state on the way in and then the second one shows the state on the way back to the browser. I then created a single GET route that will will attempt to set some dummy state context.state['hello'] = 'world' and then responds with a body that is just the stringified context.state.
// Logger
app.use(async (context, next) => {
  console.log("STATE IN", context.state);
  await next();
  console.log("STATE OUT", context.state);
});

const router = new Router();
router.get("/test", async (context: any) => {
  context.state["hello"] = "world";
  context.response.body = JSON.stringify(context.state);
});

My expectation is that if I hit this route twice I would first see an empty state on the way in, and a populated {hello: 'world'} on the way out, the second time I would see the previous {hello: 'world'} state on the way in, and the same on the way back, but I don't!
STATE IN {}
GET http://localhost:8000/test - 1ms
STATE OUT { hello: "world" }
STATE IN {}
GET http://localhost:8000/test - 1ms
STATE OUT { hello: "world" }

I actually just see the state coming in is always empty.
So then I started trying to work out what Oak is actually doing, I looked everywhere in the returned response from the first request, and I can't find a single thing that acts as storage on the browser - I would've expected a cookie because that's what most libraries do, but nothing:
HTTP/1.1 200 OK
content-length: 39
x-response-time: 1ms
content-type: text/plain; charset=utf-8

It's not much! So I guess I wonder how this works at all... it just looks like context.state is an ephemeral store for the lifetime of the request.... but not subsequent requests. I see this<EMAIL_ADDRESS>which might be the answer - Redis backed session stores are pretty norm so I wouldn't worry about adding dependencies - although it would be good if you could choose your own data-storage.

PS I am new to Deno, but have a lot of JS experience and run a lot of HTTP services with a lot of Auth! I used to use Passport a lot, but we mostly roll NextJS these days and absolutely love https://next-auth.js.org - I'd love to assist if you wanted to go somewhere like that with Dashport

That's really interesting! We store your serialized id at "ctx.state._dashport.session" and during testing I remember that trying to log ctx.state wouldn't show anything -- we needed to log ctx.state._dashport.


This condition will always return 'false' since the types '"string" | "number" | "bigint" | "boolean" | "symbol" | "undefined" | "object" | "function"' and '"array"' have no overlap?

TypeScript is complaining about this code:
function isThisJsonAnObjectOrArray(json: string) {
    let data = JSON.parse(json);
    return (typeof data === 'object' || typeof data === 'array');
}

With an error:
This condition will always return 'false' since the types '"string" | "number" | "bigint" | "boolean" | "symbol" | "undefined" | "object" | "function"' and '"array"' have no overlap. (2367)

But how would it always return false? I can see how it would be false if data is a string for example, but if it's an object or array why would it not return true?

You'll probably want to use Array.isArray(data) or if that doesn't exist in your runtime Object.prototype.toString.call(data) === "[object Array]"


Missing links in chainer official doc

Currently, Assertion and Testing page in Chainer official doc have missing links.
I made doc on my laptop, and it generated correct links, so it might be a doc server problem.
https://docs.chainer.org/en/latest/reference/check.html

Thanks for reporting.  The issue seems a duplicate of #5408.

Probably true.
#5408 doesn't notice that all links in this page are correctly generated when you build the doc pages on your laptop.
Anyway, I have no idea how to fix it and I really want to fix it.


USB ID 4000 on STM32H7R/S?

Howdy folks, hoping someone can point me in the right direction re: USB core IDs.
I spent the morning fiddling with the STM32H7 usb_serial example to get it to run on my STM32H7R/S board (this relatively new nucleo with a STM32H7S3L8H MCU), as there isn't a USB example for the R/S boards yet.
Changing some of the prescaler values in the initial config and pin #s got it to compile, but it would panic at runtime with this message:

not implemented: Unknown USB core id 4000

I added 0x0000_4000 to the core_id match statement in the relevant file here, and just gave it the same config as the 200x and 300x ids. That seems to run the example just fine without any panic.
Would it be appropriate to submit a PR adding my example and appending 0x_4000 to that match statement with a v2_v3 config? Or are there additional considerations for core ID config beyond "this basic example works" that need to be addressed before adding support for a 4000 usb ID?

yes, please send a PR!
treating v4 the same as v2/v3 should be OK. Checking the core version is mostly used to workaround weird quirks in v1, if you tried that and it works then it probably means there's no quirks we should care about in v4.

Turns out this was not working as I thought -- while it compiles, runs, and prints a lot of very beautiful INFOs, the usb FS-enabled pins don't actually exist on my board. Whoops. It compiles just fine if I switch it over to the HS pins (which I've confirmed are actually hooked up to the USB port this time, lol) but I'm getting a runtime panic that is completely stumping me:

ERROR panicked at 'peripheral 'USB_OTG_HS' is configured to use the 'USB' clock, which is not running. Either enable it in 'config.rcc' or change 'config.rcc.mux' to use another clock'

I don't see usb as a field in rcc or mux. Any idea how I can set up that clock? I see usb in the rcc clocks module but no idea how to get it into the config.
Also, I see there's only a new_fs function under Driver::, not a new_hs. I'm assuming that's fine, as a HS interface can run at FS speeds, unless I'm misunderstanding something? I don't need full HS data rates out of this thing, FS would be fine.
Here's the rest of my code for context, which I'm running from inside examples/stm32h7rs/src/bin:
`#![no_std]
#![no_main]
use defmt::{panic, *};
use embassy_executor::Spawner;
use embassy_futures::join::join;
use embassy_stm32::usb::{Driver, Instance};
use embassy_stm32::{bind_interrupts, peripherals, usb, Config};
use embassy_usb::class::cdc_acm::{CdcAcmClass, State};
use embassy_usb::driver::EndpointError;
use embassy_usb::Builder;
use {defmt_rtt as _, panic_probe as _};
bind_interrupts!(struct Irqs {
OTG_HS => usb::InterruptHandlerperipherals::USB_OTG_HS;
});
#[embassy_executor::main]
async fn main(_spawner: Spawner) {
info!("Hello World!");
let mut config = Config::default();

{
    use embassy_stm32::rcc::*;
    config.rcc.hsi = Some(HSIPrescaler::DIV1);
    config.rcc.csi = true;
    config.rcc.hsi48 = Some(Hsi48Config { sync_from_usb: true }); // needed for USB
    config.rcc.pll1 = Some(Pll {
        source: PllSource::HSI,
        prediv: PllPreDiv::DIV4,
        mul: PllMul::MUL50,
        divp: Some(PllDiv::DIV2),
        divq: None,
        divr: None,
    });
    config.rcc.sys = Sysclk::PLL1_P; // 400 Mhz
    config.rcc.ahb_pre = AHBPrescaler::DIV2; // 200 Mhz
    config.rcc.apb1_pre = APBPrescaler::DIV2; // 100 Mhz
    config.rcc.apb2_pre = APBPrescaler::DIV2; // 100 Mhz
    //config.rcc.apb3_pre = APBPrescaler::DIV2; // 100 Mhz
    config.rcc.apb4_pre = APBPrescaler::DIV2; // 100 Mhz
    config.rcc.apb5_pre = APBPrescaler::DIV2;
    config.rcc.voltage_scale = VoltageScale::HIGH;
}

let p = embassy_stm32::init(config);

// Create the driver, from the HAL.
let mut ep_out_buffer = [0u8; 256];
let mut config = embassy_stm32::usb::Config::default();

config.vbus_detection = false;

let driver = Driver::new_fs(p.USB_OTG_HS, Irqs, p.PM6, p.PM5, &mut ep_out_buffer, config);

// Create embassy-usb Config
let mut config = embassy_usb::Config::new(0xc0de, 0xcafe);
config.manufacturer = Some("Embassy");
config.product = Some("USB-serial example");
config.serial_number = Some("12345678");
// Required for windows compatibility.
// https://developer.nordicsemi.com/nRF_Connect_SDK/doc/1.9.1/kconfig/CONFIG_CDC_ACM_IAD.html#help
config.device_class = 0xEF;
config.device_sub_class = 0x02;
config.device_protocol = 0x01;
config.composite_with_iads = true;

info!("RILEYHERE");

// Create embassy-usb DeviceBuilder using the driver and config.
// It needs some buffers for building the descriptors.
let mut config_descriptor = [0; 256];
let mut bos_descriptor = [0; 256];
let mut control_buf = [0; 64];

let mut state = State::new();

let mut builder = Builder::new(
    driver,
    config,
    &mut config_descriptor,
    &mut bos_descriptor,
    &mut [], // no msos descriptors
    &mut control_buf,
);

// Create classes on the builder.
let mut class = CdcAcmClass::new(&mut builder, &mut state, 64);

// Build the builder.
let mut usb = builder.build();

// Run the USB device.
let usb_fut = usb.run();

info!("RILEYHERE2");

// Do stuff with the class!
let echo_fut = async {
    loop {
        info!("RILEYHERE3");
        class.wait_connection().await;
        info!("Connected");
        let _ = echo(&mut class).await;
        info!("Disconnected");
    }
};

// Run everything concurrently.
// If we had made everything `'static` above instead, we could do this using separate tasks instead.
join(usb_fut, echo_fut).await;

}
struct Disconnected {}
impl From for Disconnected {
fn from(val: EndpointError) -> Self {
match val {
EndpointError::BufferOverflow => panic!("Buffer overflow"),
EndpointError::Disabled => Disconnected {},
}
}
}
async fn echo<'d, T: Instance + 'd>(class: &mut CdcAcmClass<'d, Driver<'d, T>>) -> Result<(), Disconnected> {
let mut buf = [0; 64];
loop {
let n = class.read_packet(&mut buf).await?;
let data = &buf[..n];
info!("data: {:x}", data);
class.write_packet(data).await?;
}
}`

gonna try adding this bad boy to the config when i get back to the lab tomorrow
config.rcc.mux.usb_otg_fssel = mux::UsbOtgFssel::HSI48; // fix USB?
if it doesn't work i'm getting a consolation shawarma
if it works i'm getting a celebration shawarma

consolation shawarma it is, still broken. Any direction from anybody would be appreciated!

Hey, I'm trying to get USB device mode working on a Nucleo-H7S3 as well.
The problem might be the missing USB clock in the RCC configuration structure: https://github.com/embassy-rs/embassy/blob/1cf778904d597a5bc01a4b7862f965681636faf1/embassy-stm32/src/rcc/h.rs#L701
I've set this to 48MHz (usb: Some(Hertz(48_000_000))) and made sure the PHY PLL and USB clocks are running:
RCC.ahb1enr().modify(|w|
  w.set_usb_otg_hsen(true);
  w.set_usbphycen(true);
});
PWR.csr2().modify(|w| {
  w.set_usbregen(true);
  w.set_usb33den(true);
  w.set_usbhsregen(true);
});

Ultimately this should probably be integrated in the RCC and USB init functions.
I've also adapted your clock configuration like this to configure the HSE:
    {
        use embassy_stm32::rcc::*;
        config.rcc.hse = Some(Hse { freq: Hertz(24_000_000), mode: HseMode::Oscillator });
        config.rcc.hsi = Some(HSIPrescaler::DIV1);
        config.rcc.hsi48 = Some(Hsi48Config { sync_from_usb: true });
        config.rcc.pll1 = Some(Pll {
            source: PllSource::HSE,
            prediv: PllPreDiv::DIV12,
            mul: PllMul::MUL300,
            divp: Some(PllDiv::DIV1), //600 MHz
            divq: Some(PllDiv::DIV2), // 300 MHz
            divr: Some(PllDiv::DIV2), // 300 MHz
        });
        config.rcc.sys = Sysclk::PLL1_P; // 600 MHz
        config.rcc.ahb_pre = AHBPrescaler::DIV2; // 300 MHz
        config.rcc.apb1_pre = APBPrescaler::DIV2; // 150 MHz
        config.rcc.apb2_pre = APBPrescaler::DIV2; // 150 MHz
        config.rcc.apb4_pre = APBPrescaler::DIV2; // 150 MHz
        config.rcc.apb5_pre = APBPrescaler::DIV2; // 150 MHz
        config.rcc.voltage_scale = VoltageScale::HIGH;
        config.rcc.mux.usb_otg_fssel = mux::UsbOtgFssel::HSI48;
        config.rcc.pll2 = Some(Pll {
            source: PllSource::HSE,
            prediv: PllPreDiv::DIV12,
            mul: PllMul::MUL200,
            divp: Some(PllDiv::DIV2), // 200 MHz
            divq: Some(PllDiv::DIV2), // 200 MHz
            divr: Some(PllDiv::DIV2), // 200 MHz
        });
    }

By default, the USB PHY Clock Mux (which I didn't find represented anywhere in the embassy RCC configuration yet) should then use the HSE.
I've also set the phy_type in otg.rs to PhyType::InternalHighSpeed, although I'm not sure this changes anything.
Interestingly, for me the core ID seems to be 5000, so I've added 0x0000_5000 to the match statement in otg.rs as well.

Here I am stuck as well though, as there are no further interrupts being generated and the board still does not enumerate.

I've tried comparing the register state of the OTG_HS peripheral subsequent to calling the setup code (but before enumeration) both with Embassy and a working example from the Cube MCU package. I'm attaching the register dumps in case someone else also wants to take a look:
OTG_HS register state from working Cube example
OTG_HS register state from Embassy usb_serial example
So far I didn't get USB to work, but I've noticed that there seem to be some register definitions missing from stm32-metapac for the OTG_GCCFG register. There are currently two versions of that fieldset, but none of them accurately represent the peripheral on the H7RS series (see RM0477 sec. 62.14.18).
I'm not sure how much of an issue this is though. If this was an actual problem, the OTG_HS peripheral shouldn't work on the U5 series either, as that has the same register definitions (cp. RM0456 sec. 73.14.18).


ability to send messages via commandline

Is there an API to use the application in CLI/background mode and use commands to send messages to hangout?

I used the https://github.com/yakyak/hangupsjs  and its all ok now
Closing this ticket


how to merge overlapping svs

Hi there,
Reading the metaSV paper.
I am interested in how do you implement the "Merge overlapping calls per tool (intra-tool merging)"
I have read a post on bedtools google group https://groups.google.com/forum/#!topic/bedtools-discuss/JXZbJSwVxUo
where can I get the script for merging overlapping vcf files?
Thanks!
Ming

Hi Ming
Apologies for the late reply due to a combination of travels and other deadlines. The merging of overlapping calls from multiple files is implemented internally in MetaSV. You can have a look at the merge_intervals and merge_intervals_recursively functions in https://github.com/bioinform/metasv/blob/master/metasv/sv_interval.py to get a better idea.
Best
Marghoob.


[bug]: Database error when attempting to start image generation

Is there an existing issue for this problem?

[X] I have searched the existing issues

Operating system
Windows
GPU vendor
Nvidia (CUDA)
GPU model
RTX 3070
GPU VRAM
No response
Version number
v5.0.0
Browser
Chrome 129.0.6668.71
Python dependencies
{
"accelerate": "0.30.1",
"compel": "2.0.2",
"cuda": "12.1",
"diffusers": "0.27.2",
"numpy": "1.26.4",
"opencv": "<IP_ADDRESS>",
"onnx": "1.15.0",
"pillow": "10.4.0",
"python": "3.10.9",
"torch": "2.2.2+cu121",
"torchvision": "0.17.2+cu121",
"transformers": "4.41.1",
"xformers": "0.0.25.post1"
}
What happened
Attempting to start a standard text2img generation, but invocation generates an unknown error in the UI. The log output seems to indicate a problem with the database, the root cause being sqlite3.OperationalError: no such column: destination
Based on this I suspect some migration of the database has failed during an update or some form of corruption. I have attempted a reinstall but this has not worked. I'm reluctant to delete the database and start from scratch as I have a lot of previous generations I want to preserve.
Full log file:
invoke_db_error.log
What you expected to happen
A successful invocation of text2img generation.
How to reproduce the problem
No response
Additional context
I have previously been using a development version and have done repeated updates from the development branch. Perhaps this has done something to the database when updating to v5?
Discord username
No response

Rolling back to a previous database backup solved the problem. Relatively little data was lost. Seems like a database migration failed on a previous update.

Yes, because you used a development build, your database schema is out of sync. This doesn't mean a migration was unsuccessful.
This kind of issue is the reason we included a message about using a separate db when testing dev builds.
You have a two options:

Delete the old database and start a fresh one. You'll lose all your data.
Review the two (minor) changes in the last migration and edit your database to match. You can use the sqlite3 CLI to do this pretty easily - open the db file and paste each SQL query from the migration in. Probably will get an error about origin saying it already exists, but destination should add. Then you should be good to go.


[BrokenLinksH2] Fixed the link path

Fixed the link path

Docs Build status updates of commit bc5b138:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Odata-docs/webapi/capabilities-vocabulary-support.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


feat: enable user to reorder education and experience entries

when there are multiple education entries, as a user, I can reorder these, which will then be re-rendered in the CV preview, as some job positions are more important for the job that I'm currently applying for
when there are multiple experience entries, as a user, I can reorder these, which will then be re-rendered in the CV preview, as some job positions are more important for the job that I'm currently applying for

one way to fix this, design-wise, is to have in previous entries up and down arrow icons to sort (disabling the arrows at the two ends, and both arrows if it is the only entry). then, the edit and delete buttons are in a menu, clicked open by 3 vertical dots. inside the menu, we should use the words "Edit Entry" and "Delete Entry" (maybe in red?), alongside icons


Do comments add up?

This is just a simple question about how this action works: if the repo gets 100 PRs a day, does that mean that it'll comment 100 times all conflicting PRs?
Thanks

In current implementation, you are right.


Added preview package capabilities

Overview
Modernized the package with core dependency and azure pipelines
Changes

added preview core package dep
added azure pipelines yaml

/azp run

/azp run


Repository Level Code Completion format question

Hi, first of all, thanks for getting this model and make it open source.
I have a quick question on the format of Repository Level Code Completion, from the readme, it is # filename + code. I wonder # character is the format for Repository level or # make it as the comment for python.
Take an example, if the code language is cpp, should we use // instead to make it as a comment or still use # ?
thanks for answering my questions.

For cpp, we use# filename + code. @zch-cc

Why not use something consistent, like <filename> from starcoder?


Disable flaky tests that often fail in Scala PR validation

The tests were recently re-enabled, because they were improved. But they
are still not improved enough.
See:
https://scala-ci.typesafe.com/job/scala-2.11.x-integrate-ide/821/console

For future reference:
It seems that for some reason from time to time, randomly we get an event with deltas only for changed source files. Therefore, HCR isn't performed in such situations in tests and events are never fired. In a normal situation we have deltas both for source (we simply skip them) and class files.
I didn't manage to reproduce these problems on some computers (3 computers with Windows). On the other hand, they occur quite often on others (tested on Ubuntu - it would be rather strange, if OS would make any difference in this case though).


don't query program memory for add sketch line

Ostensibly this was to fix a bug where if create a second sketch, only the first segment would trigger code gen, the rest would fail.
But with this, we're actually hitting into an issue of us moving away from execute-everytime, to not. And basically, we need to stop querying programMemory, and get that information from elsewhere, namely the engine.
I've added a TODO for removing programMemory from SketchLineHelper/ModifyAstBase. Full write up here https://github.com/KittyCAD/modeling-app/issues/896

@Irev-Dev I've confirmed that codegen from sketches is working great now, thank you.
This might be outside the scope of the bug, but sketch code generated via point-and-click doesn't seem to persist between navigations:
https://github.com/KittyCAD/modeling-app/assets/23481541/7fae6558-521e-456f-a25c-09775ec32c2e


Profile vs Person

The new WebID definition is almost there.
The Profile Document I think is pretty much there.
A long standing question that I have is what is the difference between a Profile and a Person
When to use either?
Profile as in PersonalProfileDocument with primaryTopic for example

If a document is a profile document, why isnt the primary topic a Profile?  Maybe @danbri knows this.
Is there a @type for Profile as well as Person
Are there cases where Profile and Person could live in the same document

My working assumption is that Profile is not really used, and is a catch all term for Agent.  But I'd like to double check this assumption, before starting an implementation.

A person cannot be directly dereferenced over HTTP, the profile document can. This is HTTPRange-14.

Thanks.  So the primary topic of the profile document is indeed a person.  But its actually a PersonDocument, but that sounds a bit strange, so ProfileDocument is used.  So there is no actual "Profile" in the end, does that sound right?

Thanks for the feedback, closing.


Fixes for Live Migrate UI

Update: Working on the Evacuate pull request with @sseago revealed a number of problems that applied to Live Migrate that this pull request is intended to resolve.

Move Live Migrate button to the openstack-specific single vm toolbar
Gracefully flash an error if Live Migrate is selected from the
list view for an instance that doesn't support it.
Disable and show a title message on the Live Migrate button rather than skipping it if the action is unsupported for a given instance.
Show the human readable error message rather than the entire Fog exception when Migrate fails
Fix incorrect handling of parameters from the front end
Improve angular style of the live_migrate form partial.

@miq-bot add_label darga/yes

@h-kataria please review

@mansam got an error when i pressed Migrate selected Instance button for a Amazon Instance from a list view:
[----] I, [2016-05-09T17:27:25.946819 #1774:14fe880]  INFO -- :   Parameters: {"miq_grid_checks"=>"1r2130", "pressed"=>"instance_live_migrate"}
[----] F, [2016-05-09T17:27:26.032345 #1774:14fe880] FATAL -- : Error caught: [NoMethodError] undefined method is_available' for #<ManageIQ::Providers::Amazon::CloudManager::Vm:0x007fc3c3437ef0> /home/hkataria/.rvm/gems/ruby-2.2.3/bundler/gems/rails-d7e4083437a9/activemodel/lib/active_model/attribute_methods.rb:433:in method_missing'
/home/hkataria/dev/manageiq/app/controllers/vm_cloud_controller.rb:99:in live_migrate' /home/hkataria/dev/manageiq/app/controllers/application_controller/explorer.rb:88:in x_button'

@mansam another error when i press submit button on Migrat screen after checking a checkbox "Block migration"
=================================
[----] F, [2016-05-10T10:24:40.467855 #6733:7bad374] FATAL -- : Error caught: [RuntimeError] Invalid input
/home/hkataria/dev/manageiq/app/controllers/application_controller.rb:2305:in `find_by_id_filtered'
/home/hkataria/dev/manageiq/app/controllers/vm_cloud_controller.rb:152:in `live_migrate_vm'
/home/hkataria/.rvm/gems/ruby-2.2.3/bundler/gems/rails-d7e4083437a9/actionpack/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'

@h-kataria I have a set of fixes for a number of things with Live Migrate coming soon.

@h-kataria The committed updates should fix the problem you were experiencing as well as a few others I became aware of.

@mansam when i try to migrate an instance, i see following error in log & and i get redirected back to previous screen list/details screen with a flash message "Unable to live migrate Instance "BradTest2": No route to host - connect(2) for <IP_ADDRESS>:5000 (Errno::EHOSTUNREACH)":
[----] F, [2016-05-11T16:19:04.720126 #30717:1b25d30] FATAL -- : Error caught: [Errno::EHOSTUNREACH] No route to host - connect(2) for <IP_ADDRESS>:5000
/home/hkataria/.rvm/gems/ruby-2.2.3/gems/excon-0.49.0/lib/excon/socket.rb:132:in connect_nonblock' /home/hkataria/.rvm/gems/ruby-2.2.3/gems/excon-0.49.0/lib/excon/socket.rb:132:in rescue in block in connect'
/home/hkataria/.rvm/gems/ruby-2.2.3/gems/excon-0.49.0/lib/excon/socket.rb:111:in `block in connect'

@h-kataria Hmm. That sounds like something is wrong with your openstack networking. "no route to host - connect(2) for <IP_ADDRESS>:5000" -- it seems like it's trying to make a keystone API call to <IP_ADDRESS> and it can't reach it. Can manageiq reach both infra and cloud provider endpoints?

@sseago i don't have an environment set up to connect to host, but in case it can't reach the host, it should still not show errors in log, it should only show a flash message in UI.

@h-kataria My code is just catching the exception and parsing out the error. The logging isn't coming from the UI code.

Is it showing the error in log and the flash message now? I would have thought that this sort of  thing would go to the logs as well as the UI, but presumably only at an 'info' level (or maybe 'debug')?

@h-kataria Can you check whether the network call to live_migrate_form_fields is succeeding in the browser console?

@mansam getting 500 Internal Server Error on live_migrate_form_fields transaction

Okay, that's why you see it in the logs even though the migrate call catches it. Needs exception handling for getting the host list when there's a networking failure like that.

(that said, your migrate call is still going to fail due to whatever that networking issue is)

@h-kataria it should now handle that exception case more gracefully when loading the list of hosts.

:+1:

This fixes the UI issues I hit earlier. Works for me now.


Communicate better to the user

I think we can improve the communication to the user. (https://github.com/emilsvennesson/script.module.inputstreamhelper/pull/82#issuecomment-523003862)
And show fewer pop-ups (relates to #70).
If we do this correctly, an add-on does not have to provide its own pop-up.

This relates to https://github.com/emilsvennesson/script.module.inputstreamhelper/pull/82#issuecomment-523003862

I think this comes down to 3 major items:

We should aim at having only a single YES/NO dialog before installing Widevine. Ideally we would have just this:

Ask if the user wants to continue (give the ability to bail out)
Show a progress indicator (ideally the same one for download and extraction)
Show a notification on success and failure
On failure, show an OK-dialog with the exact error, or a hint where to report issues on GitHub

The dialog text will most likely be different depending on whether InputStream Helper was started from playing content, from another add-on or the InputStream Helper settings.

The license acceptance dialog could be integrated in this YES/NO dialog, although that means either downloading the license upfront, or showing a link to the license, or including the (copyrighted?) license in the add-on.

The default button is the Cancel button, which may give the impression that this is a dangerous or unwanted action. This may be very confusing to our users to the point that they will not continue with the installation. (I witnessed this a few times)

Including the license text in the addon seems like a bad idea. Google can change the license, and the new license should be displayed.
I suggest to download the license before displaying the dialog.

If we would include the license, obviously we would be monitoring for changes.
Also, today we use the license of the macOS ZIP for the ChromeOS image, so it is not like we are not already have tradeoffs here.

We can't include files from proprietary software without permission and distribute this. This is always copyright infringement.
We should do the following:

download Widevine zip to _temp_path
extract LICENSE.txt and load into memory
show license

License accepted > extract Widevine zip from _temp_path to ia_cdm_path
License declined > cleanup _temp_path

@mediaminister Don't we already do this exactly like that? Only for ARM we have a problem here.

Alright, let's postpone this item and translate what we have now. As I don't see how to improve this now.


Do not use an Action for the awaitable status

The SocketInput use an action as status.
Actions are expensive to compare and in interlock statements
This statement is 10 times slower with action
154:             if (awaitableState != _awaitableIsCompleted &&
155:                 awaitableState != _awaitableIsNotCompleted)
000007FCB1296789  mov         rdx,0E883067838h
000007FCB1296793  mov         rdx,qword ptr [rdx]
000007FCB1296796  mov         rcx,rsi
000007FCB1296799  call        000007FD05E74680
000007FCB129679E  test        eax,eax
000007FCB12967A0  je          000007FCB12967C6
000007FCB12967A2  mov         rdx,0E883067840h
000007FCB12967AC  mov         rdx,qword ptr [rdx]
000007FCB12967AF  mov         rcx,rsi
000007FCB12967B2  call        000007FD05E74680
000007FCB12967B7  test        eax,eax
000007FCB12967B9  je          000007FCB12967C6

Generated code with int
   154:             if (awaitableState2 != _awaitableIsCompleted2 &&
   155:                 awaitableState2 != _awaitableIsNotCompleted2)
000007FCB1296573  cmp         eax,1
000007FCB1296576  je          000007FCB1296583
000007FCB1296578  cmp         eax,2
000007FCB129657B  je          000007FCB1296583

Resolved by https://github.com/aspnet/KestrelHttpServer/pull/558?


Random rowid not working

I'm using the latest pre-built binary (0.19.0). I run this from the documentation:
CREATE TABLE shopping_list(item text, quantity int) RANDOM ROWID;

and get the following output (from sqld):

2023-08-29T15:33:04.877315Z ERROR tracing_panic: A panic occurred panic.payload="internal error: entered unreachable code" panic.location="/Users/runner/actions-runner/_work/sqld/sqld/target/release/build/sqlite3-parser-daa27c40267ed776/out/parse.rs:573:13"

I'm guessing it has something to do with the sqlite3-parser-crate not recognising RANDOM ROWID as valid syntax?

Similarly, altering columns does not parse, but it does not cause a panic.

/cc @psarna @MarinPostma

Yes, it's due to sqlite3-parser, which build from vanilla SQLite, not libSQL. We should consider one of the following:
a. keep contributing the support, e.g. under a libsql feature flag
b. add some kind of extension mechanism to sqlite3-parser to let us dynamically add our syntax extensions
c. fork (last resort)

@MarinPostma @LucioFranco fyi, we have more reports of that. We need to consider how to best approach parser extensions


FingerprintUSBHost is not compiling

Hi. I am compiling FingerprintUSBHost on Windows 11 to load to an Arduino UNO. I am getting compiler errors. I am using Arduino IDE 2.3.4 and the latest board 1.8.6
c:\Users\john\Documents\Arduino\libraries\FingerprintUSBHost\src/FingerprintUSBHost.h:38:55: error: expected class-name before '{' token
class FingerprintUSBHost_ : public PluggableUSBModule {
c:\Users\john\Documents\Arduino\libraries\FingerprintUSBHost\src/FingerprintUSBHost.h:52:23: error: 'USBSetup' has not been declared
int getDescriptor(USBSetup& setup);
I think that the PluggableUSB has changed as there is no constructor for PluggableUSBModule without arguments.
A number of other people have reported a problem with USBSetup.
My interest is to be able to distinguish between compiles on Windows 11 and Raspberry Pi OS (Debian) as some of my Arduino NANOs use a USB chip which is not supported on Windows 11 and is supported by R Pi OS.

Thanks
John

The 328 on an Uno doesn't speak enough USB. You need at least a leonardo.

Message ID: @.***>

Thank you. I have tried several different things, NANO, NANO EVERY, MEGA, DUE. None of them compile. Leonardo does, which I only have as a Raspberry Pi daughter board (DFROBOT DFR0327). Is there anything you can suggest so that I can identify what is uploading to a NANO?
Thanks John

Not really. The technique requires a 32U4 or other device with a similarly
capable USB stack.  And even then, it's not reliable.
On Thu, Dec 12, 2024 at 1:22 PM John Fletcher @.***>
wrote:

Thank you. I have tried several different things, NANO, NANO EVERY, MEGA,
DUE. None of them compile. Leonardo does, which I only have as a Raspberry
Pi daughter board (DFROBOT DFR0327). Is there anything you can suggest so
that I can identify what is uploading to a NANO?
Thanks John
—
Reply to this email directly, view it on GitHub
https://github.com/keyboardio/FingerprintUSBHost/issues/9#issuecomment-2540038916,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAALC2HAZUJQ2YIIIGTK5DD2FH5DBAVCNFSM6AAAAABTQT3PIKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBQGAZTQOJRGY
.
You are receiving this because you commented.Message ID:
@.***>

Thank you. I have acquired a Pro Mini and that does work and detects Windows when connected to Windows 11.


[DOCFIX] Update chinese doc for docs/cn/Configuring-Alluxio-with-secure-HDFS.md

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/11829/
Test PASSed.

Merged build finished. Test PASSed.

LGTM


Smaller changes to CIP-1694

These are changes that we wanted to make for a while but were too busy and didn't make it in time for the previous merge.

All governance actions are now enacted one epoch after they have been ratified. This is necessary to comply with requirements coming from moving the stake distribution to disk.
Remove dropping of governance actions in favour of delaying. The only effect this has is that if a NoConfidence GA is enacted and a new committee is elected later, any non-expired GAs can still be voted on and enacted (but the new committee has to approve them explicitly!).
Make the DRep registration not required. This improves liquidity of democracy.

Let me just clarify a few things. In the currently published design:

You can already delegate voting power to an unregistered DRep
You can already vote as an unregistered DRep
If you do those two things, the vote/stake is actively excluded from the results

All this change does is to not actively remove any votes. It simplifies the design (we need to do/check less things) and it means that DReps can't make the right to vote impossible to obtain by setting the deposit to an astronomical value.

VOTE NO!!! on D repTax


[Website] FAQ link leading to github main page

Hello,
I wanted to see the FAQs and noticed that the link on the frontpage of the website (Supposed to point to https://github.com/cppan/cppan/wiki/F.A.Q.) brings me back to the homepage of the Github project (https://github.com/cppan/cppan)

Thanks! Fixed.
Faq is very poor now, sorry. I'll be filling  it continuously. Feel free to ask any interested questions.
If you need more info:
Check other files in doc dir of a client.
Please, check demo example, some other examples in https://github.com/cppan
Maybe check this video https://www.youtube.com/watch?v=VE9TJJfwDEY and that channel.

I just wanted to know where does cppan download/generate stuff ?

$HOME/.cppan/storage
or for windows in %USER% or whatever is user home folder.

Added this info to faq, also the way how to change it.
https://github.com/cppan/cppan/blob/master/doc/faq.md

Exactly what I needed, thanks !


re_build_build_info: PR builds fail in CI

See for example:
https://github.com/rerun-io/rerun/actions/runs/5024217902/jobs/9009702716?pr=2161
Specific error:
  --- stderr
  thread 'main' panicked at 'Failed to run 'git ["symbolic-ref", "--short", "HEAD"]':

  fatal: ref HEAD is not a symbolic ref

  ', crates/re_build_build_info/src/lib.rs:103:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: The process '/usr/local/cargo/bin/cargo' failed with exit code 101

I believe is another problem stemming from similar conditions as what caused:

https://github.com/rerun-io/rerun/issues/2155

In this particular case, during certain pull-request builds, the build does not take place on the branch commit, but rather an ephemeral merge-commit derived from merging the PR into main.
However, regardless, re_build_build_info needs to be robust to all manner of weird git environments.
I was hoping merging main into the branch would prevent this from happening but it seems not to be the case.

Already broken in https://github.com/rerun-io/rerun/pull/2173

This was because I pushed a ill-thought-through assert to the PR in question. I'll do something better in #2173 instead


date_format type

Use case: Provide capability to cast date_format output
Feature description:
When using the date_format function, it currently returns a string, but if I'm interested in storing integer like values, it'd be beneficial to convert the value that gets stored into crate.
Stores as a string, so "0" for example instead of 0
startdateweekday AS date_format('%w', startdate)

Ideally, if it could support:
startdateweekday BYTE AS (date_format('%w', startdate) * 1)

This can already be done by adding an explicit cast to the generated column expression: startdateweekday AS cast(date_format('%w', startdate) as byte)


[test-only] add issue tag to failed tests

I've cleaned up a bit

deleted closed issue from expected failures file
added issue tag to tests

I analyzed the number of failed tests:

Passed
Failed
% of failed tests

1549
277
17,9%

If we talk about the new api, then

Passed
Failed
% of failed tests

860
122
14,2%

if we fix #5125 #5938 #5939 we remove 56 failed tests from expected failures list

E2E tests failed: https://drone.owncloud.com/owncloud/ocis/20983/68/1
:boom: To see the trace, please open the link in the console ...

npx playwright show-trace https://cache.owncloud.com/public/owncloud/ocis/20983/tracing/file-alice-2023-3-30-10-09-17.zip

e2e flaky, if you don't mind I bump weCommit here


DOM can create a new account

Given Dom sees the login page and has not created an account yet
When Dom clicks the button to "create new account"
Then he can input an email, display name, and password that will be added to the login database.

BE
--receive email and PW
--create salt
--make hash
--add it to the database
FE


Remove forward slash to make echos nicer

Hey ! A really stupid PR because I'm a fussy nerd

Thanks for the AUR package btw ;)

Haha sure thing, thank you for the contributions! :grin: I think this would break add-efi though, could you check? Maybe for simplicity we can add the slash on line 115, i.e. entry="/$EFI/$NAME.efi" ?

I does indeed, nice catch ! This new commit should fix the issue.

I would totally understand if you don't want to bother adding 2 commits just for the sake of removing a slash duplicate which is not even harmful :sweat_smile:
I didn't notice the fix you suggested (the entry="/$EFI/$NAME.efi"), it works too

Thanks for testing! And don't worry at all, I want aaaaaalllllllll the commits you have :grin: :grin: I have a small preference for my suggestion, because the magic in efibootmgr call is already unreadable enough :sweat_smile: If you could push that I'd appreciate, otherwise I'll just merge later and update, and cut a new release :stuck_out_tongue:

The reason why I didn't use your approach is because the next statement would evaluate to [ -f "/efi//EFI/arch" ], doubling the slash again. But this is not being echoed, so I guess you're right, readability should be prefered. Commit is on its way.
https://github.com/maximbaz/arch-secure-boot/blob/1fd8e9e933519af612f78aeb7f14a416f23912b6/arch-secure-boot#L115-L116

Done, I forced push to prevent polluting your git history, do not reproduce at ~ :stuck_out_tongue_winking_eye:

Your reasoning makes sense, I didn't spot that bit! Agree with you that we should nevertheless prefer the readability :+1:
Thanks a ton!


Fix types for **kwargs

**kwargs arguments should be typed according to their values (i.e. a single argument), not kwargs as a whole - see e.g. the mypy docs (last snippet).
This fixes typing issues such as this one:

Weirdly enough, the Pydantic mypy plugin seems to hide this issue in this repository. If I comment out plugins = pydantic.mypy in mypy.ini, this is flagged in the tests and such... That seems like an issue in that plugin perhaps, but I have no idea of Pydantic, so I won't look at that in detail I'm afraid.

Codecov Report

Merging #116 (61c0f07) into main (dd27c91) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #116   +/-   ##
=======================================
  Coverage   96.88%   96.88%
=======================================
  Files          42       42
  Lines        2278     2278
=======================================
  Hits         2207     2207
  Misses         71       71

Impacted Files
Coverage Δ

ical/event.py
97.00% <100.00%> (ø)

ical/freebusy.py
98.61% <100.00%> (ø)

ical/journal.py
100.00% <100.00%> (ø)

ical/timezone.py
96.45% <100.00%> (ø)

ical/todo.py
98.52% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


feat: Convert company id to vat number

:tada: This PR is included in version 1.4.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


in alpha, show the origin in completions instead of the brand name

What do you guys think? It helps me debug stuff - thought to add it permanently in alpha mode.


Dataloader refactoring, new javaparser class and navbar

Added parsing of java-classes with new service. Should work with multiple tasks now as well.
This pull request also includes my former updates on nav-bar, Dataloader.java refactoring and Task.java refactoring.

No tests for JavaClassParser

Added jUnit tests for JavaClassParser

Might want to change the name of the class, as this is code generation rather than parsing.


✨ Expose verbosity controls from kubestellar PCH

Summary
This PR generalizes the kubestellar PostCreateHook to take two controller verbosity parameters and the test/e2e/common/setup-kubestellar.sh script to pass them. This removes the need for a restriction on the those verbosity settings when using that PostCreateHook.
Related issue(s)
Fixes #2439

/cc @pdettori
/cc @francostellari
/cc @nirrozenbaum

/lgtm

/approve


Add 'retry' to opt_keys in 'async_task'

Hi again @Koed00
I'd like to add retry to opt_keys in async_task. We have implemented our own custom ORM broker that makes use of custom retry value like this:
def _timeout(retry=None):
    retry = retry or Conf.RETRY
    return timezone.now() - timedelta(seconds=retry)

class ORM(BaseORM):

    def enqueue(self, task):
        data = SignedPackage.loads(task)
        retry = data.get('retry')
        package = self.get_connection().create(
            key=self.list_key, payload=task, lock=_timeout(retry)
        )
        return package.pk

    def dequeue(self):
        # Query queued tasks using default _timeout
        tasks = self.get_connection().filter(key=self.list_key, lock__lt=_timeout())[
            0:Conf.BULK
        ]
        if tasks:
            task_list = []
            for task in tasks:
                # Check here if task has custom timeout. If lock
                # is greater than custom timeout time, skip
                retry = task.task().get('retry')
                if task.lock > _timeout(retry):
                    continue
                if (
                    self.get_connection()
                    .filter(id=task.id, lock=task.lock)
                    .update(lock=timezone.now())
                ):
                    task_list.append((task.pk, task.payload))
                # else don't process, as another cluster has been faster than us on that task
            return task_list
        # empty queue, spare the cpu
        sleep(Conf.POLL)

This isn't at the moment possible because retry is missing from opt_keys and thus not added in the task payload data.

I see this as a valuable feature. We have a few tasks that can take several minutes. I don't think it appropriate to raise the retry  for all tasks in our app (including some that take a few seconds) to several minutes. It is curious that the timeout can be configured per task, but not the retry.

@jasonbodily For now I have solved this by checking whether task has custom timeout and then adding +5 minutes to it:
def _timeout(retry=None):
    retry = retry or Conf.RETRY
    return timezone.now() - timedelta(seconds=retry)

class ORM(BaseORM):

    def _get_retry(self, data):
        # Retry 5 minutes after timeout
        if not data.get('timeout'):
            return None
        return timeout + 300

    def enqueue(self, task):
        data = SignedPackage.loads(task)
        retry = self._get_retry(data)
        package = self.get_connection().create(
            key=self.list_key, payload=task, lock=_timeout(retry)
        )
        return package.pk


[streamroles] alerts showing ID?

I'm not sure if this is a bug or intentional, but after an update (first in a long time), I noticed that the alert header went from "user#### updated" to "user#### (ID number) updated". Would it be possible to remove/toggle the ID number to how it was before? Thank you.

My mistake. I should have linked this issue to the general mod functions. However, I noticed a different issue with game titles not being displayed, and only the title (twice):

Tobotimus is now live on Twitch, playing Playing some T-Hunt:
Playing some T-Hunt
https://twitch.tv/Tobotimus

@Kovaelin that alert doesn't look like it's on the latest version. Can you update and check again?


Support Carthage

Thanks for this amazing project. Would love support for Carthage.
A framework was discussed in the past. Is a framework now available?
Other discussions.
https://github.com/krzysztofzablocki/Sourcery/issues/67 https://github.com/krzysztofzablocki/Sourcery/issues/388

There is no official Carthage support but I do not see why this would not work as all our framework schemes are shared. So Carthage should be able to build the Sourcery frameworks.
Maybe you could try to do a Carthage integration in your project and give us some feedback?

Thanks for response.
Carthage completed an update cycle successfully.
Cartfile
github "krzysztofzablocki/Sourcery" == 0.16.1

Output
$ carthage update Sourcery
*** Cloning Sourcery
*** Checking out Sourcery at "0.16.1"
*** xcodebuild output can be found in /var/folders/vl/630ywf4s62j1ppzyh271sdb80000gn/T/carthage-xcodebuild.sgrvg6.log
*** Downloading Sourcery.framework binary at "0.16.1"
*** Building scheme "SourceryJS" in Sourcery.xcworkspace
*** Building scheme "SourceryRuntime" in Sourcery.xcworkspace
*** Building scheme "SourceryFramework" in Sourcery.xcworkspace
*** Building scheme "SourcerySwift" in Sourcery.xcworkspace

Generated

Problem
I have attempted to run all of them as an Xcode run script build phase on my iOS project.
I'm not sure which binary to execute and I suspect these frameworks will only work for Mac targets anyway.
Cannot execute binary file.
./Carthage/Build/Mac/SourcerySwift.framework/Versions/A/SourcerySwift --config ./.sourcery.yml

I really like what @bradhilton did here https://github.com/krzysztofzablocki/Sourcery/issues/67#issuecomment-449759900 and it would be awesome to at least achieve that in the short term.

So if I understand your objective, you would like to run the Sourcery CLI in your project using Carthage?
Unfortunately for you, this is not something we will be supporting anytime soon, I don't even think that is even possible with Carthage.
Carthage is a tool for dependency management. It will generate frameworks for you to import, it will not generate binaries to run.
All you can do with Carthage is integrate our frameworks as dependencies but this will not give you the full power of Sourcery because we do not expose a SourceryCLI framework for you to build upon. I don't think that making such a framework is a rework that we wish to do. And more importantly this would not solve your use case. This would be more of a way to extend Sourcery, not to use it as it is in a project.
You have plenty of other options to use Sourcery in your project that we support.

Thanks for the response.

@BowdusBrown When I asked the Carthage developers about a situation like this, they said it's not really supported. However, there is one workaround they mentioned: Create a "fake framework" of your own that contains the CLI tool's binary. Nobody ever links to that framework, but you can Carthage will download the fake framework and you can reference the file in it. It's an ugly solution, but it would be an option.
But one of your comments sounds like you want to run the Sourcery CLI on iOS? You can't run CLI tools on iOS AFAIK. Sourcery will only be useful either if your app uses the framework to process files on iOS, or if you use the CLI tool on a Mac as part of the build process.

Hi @uli-objectbox thanks for the response.
At the moment, I am running Sourcery CLI as a build phase in my iOS project.
I was hoping to get the installation steps here down from 4 steps, to 3 steps by somehow getting Carthage to build sourcery (as a CLI tool) and then run it. I appreciate a custom script could do this, via homebrew. But I was kinda hoping it could be more local to my project somehow. Maybe a build phase would build sourcery in my project directory somewhere and cache it there for all future builds, so the CLI doesn't get installed in /usr/local/bin which is more global. I dunno, just thinking out loud.


[release-2.3] No-op rebuild

Attempt rebuild to pick up:

https://github.com/open-cluster-management/governance-policy-framework/pull/218

It looks like my changes for the Gatekeeper tests failed to create a new image.
/hold for 2.3 to open

2.3.5 GAed today.

/unhold


WIP: front-end logic to call geocoding & related functions

While investigating/refining tickets I created a few new endpoints to experiment with, just wanted to share them here for whoever will work on Add front-end logic to use AWS for geocoding + place lookup or Add front-end logic to use AWS for reverse geocoding
They will return both AWS and Google results in a JSON response, which I found useful for a quick view/comparison to debug what's returned for a given query. 🤷🏼♀
Just to be clear, it's super hacky.

Superseded by #1222


Accepting Payments

We should use PayPal for accepting donations/premium subscriptions. We should setup a webhook that communicates with Bookmark whenever somebody registers a site for premium.

We could also integrate with Stripe or BrainTree but we'd need to setup an organization bank account and I don't have time for that at the moment.


Issue 1301 simplifications

Description
Adds several automatic simplifications to the expression tree. See #1301 for details.
By informal testing, results in a ~10% speed-up for the models
The Simplify function may no longer be needed after this feature is added.
Fixes #1301
Type of change
Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

[ ] New feature (non-breaking change which adds functionality)
[x] Optimization (back-end change that speeds up the code)
[ ] Bug fix (non-breaking change which fixes an issue)

Key checklist:

[x] No style issues: $ flake8
[x] All tests pass: $ python run-tests.py --unit
[x] The documentation builds: $ cd docs and then $ make clean; make html

You can run all three at once, using $ python run-tests.py --quick.
Further checks:

[x] Code is commented, particularly in hard-to-understand areas
[x] Tests added that prove fix is effective or that feature works

@martinjrobins a lot of the missed coverage is now coming from the Simplify class not really needing to do anything now. Do you think we should get rid of that class, or keep it?

@martinjrobins a lot of the missed coverage is now coming from the Simplify class not really needing to do anything now. Do you think we should get rid of that class, or keep it?

just get rid of it if its not doing anything useful. It'll be in the git history if we ever feel the need to revive it


why  it shows ''loss_total_val=NaN''

I want to know why it shows up ''AssertionError: Model diverged with loss = NaN’' when I run the code.I have checked the code and try to revise it however it still shows ''loss_total_val=NaN''.Could you tell me how to handle it?plz.

I did not come across this problem. NaN is usually due to divided by 0, or out of range. Please check if images are correctly loaded.

thank you for yout reply ！
i used python3.6 before，but the requires 2.7，so it maybe the python version problem？and my operation system is win10 ，is your system ubuntu？
As you mentioned，i prepared the dataset as follow as your grpolist ，but i cant understand the loadimage code clearly。
And when i revise the loss，it could run several steps then still show the NaNerror，i am cofused about it。
In all，thank you very much！The deblurring result is the best i have occured，i will try it again。

Sorry to trouble you again。I try my best to find the error，but it still cant work。I want to know if the picture is successfully imported into it, but I don't know how to do it. Could you please help me see where there is a problem?Thank you very much!

We have tested again and do not produce NaN.
You can use sess.run() on input tensor to get its value to see images are loaded.

I'm gonna import the document again into the dataset or show NaN,
I have got the shape of image_in and image_gt.Forgive me for not knowing which input tensor to sess.run(). As I showed above, is my dataset imported correctly?
Please help me. I've been bothering it for days.

I found that in another case, when I ran with the CPU, the loss value could be displayed normally, but it was too slow. I think the input should be imported correctly.I really don't know how to solve the NaN problem of loss value when running with GPU.

@Akumio We also cannot figure out why NaN can happen. Really sorry about that.

@Akumio
Hi ,I want to know how to make a file named "datalist_gopro.txt", which is very important to me. Thank you very much for replying to me.

@Akumio We also cannot figure out why NaN can happen. Really sorry about that.

Could you plz tell me how to figure this problem out? I also get trapped in this issue. Thanks for your kind reply!


Byte responses support

Uint8List

I don't remember what is this ticket about. Please raise new issue if it's needed with more details.


[PlayState] Add support for sharing real play time to another extensions

Extension
PlayState
Describe the solution you'd like
Add support for sharing the real play time after using the pause feature to another extensions, like Game Activity.
Opening this to keep tracking here too.
Quote from @Lacro59 on https://github.com/Lacro59/playnite-gameactivity-plugin/issues/55 :

@darklinkpower
The ideal is for the plugin to share the different poses to remove them from the session.
But it will probably be necessary to wait until Playnite.SDK allows to share the data.

Depends on https://github.com/JosefNemec/Playnite/issues/2039


[ISSUE #3177]updateAclConfig in all brokers.

Make sure set the target branch to develop
What is the purpose of the change
#3177
Brief changelog
XX
Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Coverage decreased (-0.006%) to 54.028% when pulling 66ea71b79773caa2b5a6f7ca4f3ce2c23305bdf8 on zhangjidi2016:updateAcl into 36e6687d50b2de14d878dbc68e1b99fe254fed67 on apache:develop.


Allow variable number of repetitions for RA

The number of repetitions in the RA sampler was set to 3. With this PR it can be changed through CLI flags.
Related to #5051.
cc @datumbox

@sallysyw @datumbox any news about this PR? I can confirm that with this PR (ra_reps=4) and the latest ResNet-50/ImageNet recipe, there is an improvement in top1 accuracy to 80.802%

@tbennun Sounds good let's merge it. Do you have the log file and checkpoint for the ResNet50 run?

@tbennun Sounds good let's merge it. Do you have the log file and checkpoint for the ResNet50 run?

@datumbox
Checkpoints:

Last (epoch 600): https://polybox.ethz.ch/index.php/s/1y4nRKxNMYRjcRf
Best (epoch 566, 80.808% Top1): https://polybox.ethz.ch/index.php/s/rlMIvIhpP7nbnt6

The script didn't store a log file, so I only have the last few printouts (would be nice if it did by default :) ). However, it did store all the models from every epoch. I attached above the relevant two (last, best), and here is the log of running with --test-only on the last 50 models:  test.log
The command line I used for training is as follows (as opposed to the blog, I trained on four GPUs rather than eight, but with the same overall batch size):
torchrun --nproc_per_node=4 train.py --model resnet50 --batch-size 256 --lr 0.5 --lr-scheduler cosineannealinglr --lr-warmup-epochs 5 --lr-warmup-method linear --auto-augment ta_wide --epochs 600 --random-erase 0.1 --weight-decay 0.00002 --norm-weight-decay 0.0 --label-smoothing 0.1 --mixup-alpha 0.2 --cutmix-alpha 1.0 --train-crop-size 176 --model-ema --val-resize-size 232 --data-path /path/to/imagenet --cache-dataset --ra-sampler --ra-reps=4

I am now testing the mode where a full epoch is used on the same system (training will see 4x samples).

@tbennun Thanks for sharing the checkpoints and the info. This is great!
It's a bummer you don't got the log. We aim for full reproducibility and we normally store the logs and all checkpoints for every model we release. I'm going to fire a new set of training with --ra-sampler --ra-reps=4 on our side to see if we can reproduce and if we do, I'll be happy to update the checkpoint. We'll work out a way on the PR to credit you as the author (we still try to find out what's the best way to receive such contributions by the community so, bear with me).

@tbennun Happy new year! I was able to verify your findings and get a run which achieves:
Acc@1 80.858 Acc@5 95.434

Would you be interested sending a PR so that you can be credited for the contribution? Given that you don't have access to deploy the weights, I can do that for you. I propose the following:

Write a comment on the issue #3995, briefly describing the modification on top of the main recipe. Make is self sufficient, so that someone who reads it understands what you did to achieve the improvement. Feel free to include papers/references that inspired the change.
Send a new PR which modifies the accuracies of the V2 weights to the values I mention above. Moreover change the recipe link to point to your previous comment.
Make sure you have the "Allow edits from maintainers" checkbox on the PR checked, so that I can update the URL of the weights once I deploy them on S3.

@tbennun Happy new year! I was able to verify your findings and get a run which achieves:
Acc@1 80.858 Acc@5 95.434

Would you be interested sending a PR so that you can be credited for the contribution? Given that you don't have access to deploy the weights, I can do that for you. I propose the following:

Write a comment on the issue #3995, briefly describing the modification on top of the main recipe. Make is self sufficient, so that someone who reads it understands what you did to achieve the improvement. Feel free to include papers/references that inspired the change.
Send a new PR which modifies the accuracies of the V2 weights to the values I mention above. Moreover change the recipe link to point to your previous comment.
Make sure you have the "Allow edits from maintainers" checkbox on the PR checked, so that I can update the URL of the weights once I deploy them on S3.

@datumbox that's great! I'll take care of the comments and the PR. I'm currently running the other version I mentioned before (saving all logs this time :) ), is it ok if we wait with merging the new PR until training completes? This way we can upload a modified version if the accuracy improvement is substantial.

@tbennun Yes, no problem at all. If the accuracy diff is substantial, we can deploy yours. Though in that case, we would need to find a way to send us all the checkpoints and log files associated with the model. This is several hundreds of GB, so it might be a bit tricky but I'm sure we will work it out.

@tbennun Just checking to see if you have any update concerning the training. Thanks!

@tbennun Just checking to see if you have any update concerning the training. Thanks!

@datumbox yes. It's almost complete (epoch 552 right now) and max top1 accuracy is 81.05% (attained at epoch 513, top5 95.52%), but it's not getting any higher at this point. This is the exact same recipe but with the full BA. I kept the log and all checkpoints as well, but you can judge if this is something worth uploading. What do you think?

@tbennun Sounds good, let's review the full details once the training is completed. Please upload somewhere the best epoch checkpoint once the training is completed along with the log so that we can check. If we do end up using your run, it's going to be a challenge fetching all the logs and checkpoints. Bear with us as we try to work these details out; you are the first person who wants to contribute a half TB run, so that comes with its own challenges. 😄

@tbennun Thanks for the update.
I assume that by nodes you meant the number of available GPUs. Judging from the patch you provided, I understand that each GPU actually runs the full size of the dataset, meaning that for ra_reps=4 it takes 4x longer to train the model. This is why according to your log, the total number of iters per epoch is 5005 instead of 1252 which is the expected.
Unfortunately I don't think we would like to make such a modification to the RASampler, because we try to keep our implementations canonical. Moreover, I suspect that one could achieve a similar result by running the training for 4x more epochs. On the other hand, your original proposal of using the RASampler with 4 repetitions is a quite elegant as it keeps the training budget the same while still increasing the accuracy by 0.2 points. We can also use the checkpoint with accuracy 80.858 which will save us from having to exchange 1 TB worth of checkpoints.
If you send a PR with the changes described here, I can help you deploy the pre-trained model on PyTorch's S3 bucket and release it. Let me know what you think.

@datumbox agreed on all counts. If that's alright with you, I will create the PR over this weekend. Thanks again for your patience!

@datumbox Done:
Recipe and reference: https://github.com/pytorch/vision/issues/3995#issuecomment-1013906621
PR: #5201


feature/setup-continuous-deployment

Continuous Deployment

Connecting Travis CI to Heroku
Setting up continuous deployment

Looks good, let's merge!


(PUP-9991) Allow module uninstall in FIPS

The uninstall action detects if there are local modifications to a module
before deleting it. Since module checksum metadata is hardcoded to use MD5, the
entire uninstall action was disallowed in FIPS mode.
This commit relaxes the restriction so that uninstall is allowed provided
--ignore_changes or --force are specified, since they both skip the check
for local modifications.

I verified uninstalling works as expected using ignore_changes or force:
[root@pexi6qmamp2hflw puppet]# FACTER_fips_enabled=true bundle exec puppet module uninstall puppetlabs-stdlib --ignore_changes
Notice: Preparing to uninstall 'puppetlabs-stdlib' ...
Removed 'puppetlabs-stdlib' (v6.0.0) from /etc/puppetlabs/code/modules

[root@pexi6qmamp2hflw puppet]# FACTER_fips_enabled=true bundle exec puppet module uninstall puppetlabs-stdlib --force
Notice: Preparing to uninstall 'puppetlabs-stdlib' ...
Removed 'puppetlabs-stdlib' (v6.0.0) from /etc/puppetlabs/code/modules

While the default is to raise an error:
[root@pexi6qmamp2hflw puppet]# FACTER_fips_enabled=true bundle exec puppet module uninstall puppetlabs-stdlib
Notice: Preparing to uninstall 'puppetlabs-stdlib' ...
Error: Either the `--ignore_changes` or `--force` argument must be specified to uninstall modules when running in FIPS mode.
/root/puppet/lib/puppet/module_tool/applications/uninstaller.rb:92:in `validate_module'
/root/puppet/lib/puppet/module_tool/applications/uninstaller.rb:25:in `run'
/root/puppet/lib/puppet/module_tool/applications/application.rb:11:in `run'
/root/puppet/lib/puppet/face/module/uninstall.rb:70:in `block (3 levels) in <top (required)>'
...


az synapse workspace activate fails  for --ids option

zz --version
azure-cli 2.31.0
core 2.31.0
telemetry 1.0.6
Describe the bug
I am trying to activate Synapse workspace via AZ CLI command using --ids option. What exactly is needed for --ids to work? The command is unable to get the --name option. Its ignores if I provide the option.
--W/O --name option
az synapse workspace activate --ids /subscriptions/xxxxxxxxxxxxxxxxx/resourceGroups/rg-mddsha1prodeastus-synapse-ws/providers/Microsoft.Synapse/workspaces/synmddsha1prodeastus --key-identifier https://xxxxxxxxx.vault.azure.net/keys/key-cmk-mddsha1prodeastus
Argument --name cannot be derived from ID /subscriptions/a0b62ca5-4dbf-4ddb-8d97-9d36cea4875c/resourceGroups/rg-mddsha1prodeastus-synapse-ws/providers/Microsoft.Synapse/workspaces/synmddsha1prodeastus,. Please provide a complete resource ID containing all information of 'Resource Id' arguments.
---With --name option
az synapse workspace activate --ids /subscriptions/xxxxxxxxxxxxxxxxx/resourceGroups/rg-mddsha1prodeastus-synapse-ws/providers/Microsoft.Synapse/workspaces/synmddsha1prodeastus --key-identifier https://xxxxxxxxx.vault.azure.net/keys/key-cmk-mddsha1prodeastus --name key-cmk-xxxxxxxx
option '--name' will be ignored due to use of '--ids'.
To Reproduce
Create synapse workspace with double encryption/system managed identity and run above command
Additional context
Add any other context about the problem here.

Hi @shannawaz as this issue is more suitable for core azure cli team, we have raised an issue on your behalf. https://github.com/Azure/azure-cli/issues/20814

Hi @shannawaz , --ids need to send a keyvault resourceid, not workspace resourceid. And if we use --ids, don't need to use --workspace-name and --name.
Source :- https://github.com/Azure/azure-cli/issues/20814#issuecomment-1004552610

Thank you, This issue can be closed. Appreciate if the documentation be update for future customers.
Here is current doc
https://docs.microsoft.com/en-us/cli/azure/synapse/workspace?view=azure-cli-latest#az-synapse-workspace-activate
--ids
One or more resource IDs (space-delimited). It should be a complete resource ID containing all information of 'Resource Id' arguments. You should provide either --ids or other 'Resource Id' arguments.

Thanks @shannawaz, I am closing this issue now.


Current embedded mouse wheel plugin doesn't work with jQuery 1.7

Latest mouse wheel plugin is 3.0.6  jQuery 1.7 compatiblity added with 3.0.5  It looks like 3.0.2 is the one being embedded. It doesn't work at all form me on Firefox loading jQuery 1.7.1

Thanks, we are updating this for alpha 4 which should be released soon. Sorry for the delay.

The test build now includes the latest mousewheel plugin. If you have a chance, please try it & let me know how it works for you:
http://jquerygeo.com/test/jquery.geo-test.min.js
Thanks!

Hi strait, this should be fixed in the latest alpha release. Please test with:
http://code.jquerygeo.com/jquery.geo-1.0a4.min.js
If it works for you, please close this issue.


I can not send emails

Dear,
I can not send emails  :(
I get this response:
Error =>> 415
body =>> { "errors": [{ "message": "Content-Type Should be application / json." "field": null, "help": null}]}
headers =>> {server headers: 'nginx'
   date: 'Wed, June 15, 2016 2:56:08 GMT'
   'Content-type' 'application / json'
   'Content-length' '92',
   connection: 'close'
   'X-frame-options' 'DENY'}

sendgrid-nodejs Version: master (latest commit: [commit number])
Node.js Version: 4.1.2

Hello @eaparango,
Thanks for reaching out to us!
I'll need some further details to help you:

What version of the library are you using?
Can you provide the source code? (With credentials and emails redacted)

@eaparango, I just tested using the example in the README and I did not get your error. Please answer the questions above so we can further assist. Thanks!

Any luck @eaparango?

It works perfect thanks !!


Add ConversionException in FailedRecordProcessor.classifier to skip retrying

Affects Version(s): 2.5.4

Hi, I would like to request a convenience feature.
I am writing a kafka listener which need json deserialization.
I learned that spring-kafka support JsonDeserializer and JsonMessageConverter.
And, I did choose JsonMessageConverter because it's more flexible.
The problem is when using JsonMessageConverter, if deserialization fails, it keeps to retry because ConversionExceptionis not in exception classifications of FailedRecordProcessor
I solved this issue through errorHandler.addNotRetryableException(ConversionException.class)
But, It will be nice if ConversionException is in exception classifications by default
Please, Consider to add ConversionException in exception classifications
Thank you!

Thanks; yes; really the converter should throw a MessageConversionException (which is already in the default list) but that would be a breaking change now. So, yes, we'll add ConversionException to the default list.

If you don't mind, can I get this issue for a contribution?

Sure. That would be great.

Thanks for the contribution!
Cherry-picked to 2.5.x after resolving conflicts.


⚠ Bofrost.at has degraded performance

In b30c0f9, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1573 ms

Resolved: Bofrost.at performance has improved in 3c8a1cc after 13 minutes.


[MM-30391] E2E tests - Team Settings pt. 2

Summary
New E2E tests for the Manage Members modal, and a couple other team settings.
Was not able to create this test:
https://mattermost.atlassian.net/projects/MM?selectedItem=com.atlassian.plugins.atlassian-connect-plugin:com.kanoah.test-manager__main-project-page#!/testCase/MM-T2340
It is using the old Add Members to Team modal that doesn't exist anymore, and I believe the behaviour is covered here in the new Invite People modal:
https://github.com/mattermost/mattermost-webapp/blob/4b2d72ded4ec35873f0b2cf8702a7f4577b773aa/e2e/cypress/integration/team_settings/invite_members_spec.js#L178
Ticket Link
https://mattermost.atlassian.net/browse/MM-30391

/update-branch


Not running but no errors

When i start my ravencoin pool i get the big miningcore logo and the donation addresses + the "using configuration file config.json" but nothing more. I run relase 50 on Ubuntu server 20.04. I tried connecting to the pool incase its only a log problem. But i get errors about "can't convert to bool" so not sure What to do

Nvm had log on error


Store model in gatsby-node.js

Implement the store model dynamically into gatsby-node.js

In progress, Jose is close.

https://app.contentful.com/spaces/2n4rer1mpqy8/content_types/store/fields


Access Seafile with Nginx Proxy Manager via SSL

I have successfully installed Seafile using the Seafile Official Guide and got it running on port 8080. Now I want to use https on port 443 (10443). I get the error 502 Bad Gateway.
How can I solve this problem?

Please show me your NPM config. Please be aware that NPM is a reverse proxy. So you application (seafile) is only communicating on port 8080.
So
Plain HTTP no SSL
[Seafile Container] <---Port 8080---> [NPM Container] <---Port 80---> Client/Browser

Secure HTTPS / SSL
[Seafile Container] <---Port 8080---> [NPM Container] <--Port 443--> Client/Browser


Error message when trying to export a dataset from running ASReview on a server

Describe the bug
When running ASReview on a server, I receive an error when I am trying to do an export. This error only shows when I want to export as a dataset, but not as a project. The following error message is shown:
Failed to export the csv dataset. [Errno 2] No such file or directory: '/usr/local/lib/python3.8/site-packages/asreview/webapp/projects/demo/tmpm9zb3r19/export_dataset.csv' Please try again.
To Reproduce
Select dataset when exporting a file when ASReview is running on a server, and then export.
Screenshots

Version information

OS: Linux (not sure which distro)
ASReview version: 1.0

Could you try to upgrade to v1.0.3?

Thank you for the quick reply! I will relay the message to the party that hosts our server and I'll get back to you.

You can always extract the data from the .asreview file via the API

Is the path to the project folder adjusted? It looks like a very odd path to me. There shouldn't be data in that place.

Could you try to upgrade to v1.0.3?

I don't think this will solve the issue.

Firstly, apologies for the late follow-up.
Secondly, @J535D165 was correct, Our hosting party had changed the location where projects were saved, because that was not a practical place for their server. He said he changed that via an ASREVIEW_PATH environment variable as per the documentation. After removing this, exporting projects is working as normal once again.

I think the server admin set the environment variable to a relative path instead of an absolute path. Absolute paths are a better solution in my opinion. Therefore the projects were stored in the application folder (not a good idea because of upgrading!!).
Please ensure that the admin sets up a backup properly. Feel free to ask questions here or on the discussion platform. Feel free to reopen this issue if there is something left to discuss.


Validate certificate subject

The scion-pki tool does not have a way to validate that the distinguished name in the subject of a certificate matches a certain subject identifier.

Closed by #4179


Correct appconfig user agent string

User agent string from preview.3 library or before is showing up in logs as:
azsdk-net-config/<IP_ADDRESS> (.NET Core 4.6.28008.02; Linux 4.15.0-1059-azure #64-Ubuntu SMP Fri Sep 13 17:02:44 UTC 2019)
The user agent string should include the SemVer version of the release (i.e. 1.0.0-preview.3) as well as a more descriptive name like the library name, unless this is what the engineering team intends.

This was fixed in preview.5.


Authorino manifest config

Authorino manifests as config
Store authorino manifests in their own kustomization config file and include it in the default and install kustomizations. This allows an exact copy of the upstream version manifests to be stored giving us:

Easier comparisons (upstream vs local).
Ability to add kustomizations to those manifests if required.
Allows them to be included in different kustomizations
(default vs install)
Tidier make targets that don't differ so much from the default, which
coincidentally will alllow the "verify-bundle" make target to work
again on CI.

Add new make targets for dealing with the different manifest types (authorino, install and deploy) and trigger them all as part of the top level "manifests" target.
Moved all manifest targets under the development section of the Makefile.
Removed install-authorino target since install target includes authorino manifests.
Bundle generation and CI verification

Add missing role file to bundle.
Add back "verify-bundle" CI job.

@guicassolato @jjaferson There was a missing file in the bundle i added which led me to look at this again and try and get the CI job working (make verify-bundle) which would have caught it ( wouldn't work via GH Actions with the current changes to the bundle make target for some reason). This could probably have been fixed in other ways, but i had started looking into how best to manage the operand(kuadrant-controller) from the kuadrant-operator and had been doing something similar to what I've done here to just keep the installation manifests in their own kustomize base that can be included by others. This seems, to me at least, a nicer way of doing things than joining multiple sets of manifests via commands in the Makefile.
Anyway, let me know what you think, no rush on this one.

@mikenairn, IMO the manifests imported from https://raw.githubusercontent.com/Kuadrant/authorino/main/install/manifests.yaml should not change in the Authorino Operator repo via Makefile.
Authorino (service) owns the AuthConfig CRD and related RBAC, while Authorino Operator owns Authorino CRD and related RBAC. The reason why we decided to keep a copy of the Authorino manifests here was so one could install Authorino Operator and Authorino CRDs and RBAC and create a deployment of Authorino Operator, all in one single kubectl apply command. Other than that, those 2 sets of manifests could remain each one in its repo original repo only, where they are separately defined and maintained.
Say we fix the reconciler for the new names of the Authorino roles. The changes here proposed would still impact the dev workflow in the main Authorino repo for cases where the change there involves modifying the any of these resources. E.g., to add a new permission to the Authorino manager role, today you can change only in the Authorino repo; you don't even need to clone the Authorino Operator repo – just install/deploy the operator latest image and manifests, and in your branch for the Authorino repo, do the changes and re-apply on top.

@mikenairn, IMO the manifests imported from https://raw.githubusercontent.com/Kuadrant/authorino/main/install/manifests.yaml should not change in the Authorino Operator repo via Makefile.

Apart from the name prefix, which can be removed, they are unchanged. Only the means to which they are merged together for the OLM bundle and the deploy manifest file was updated.

Authorino (service) owns the AuthConfig CRD and related RBAC, while Authorino Operator owns Authorino CRD and related RBAC. The reason why we decided to keep a copy of the Authorino manifests here was so one could install Authorino Operator and Authorino CRDs and RBAC and create a deployment of Authorino Operator, all in one single kubectl apply command. Other than that, those 2 sets of manifests could remain each one in its respective original repo only, where they are separately defined and maintained.
Say we fix the reconciler for the new names of the Authorino roles that kustomize is now prefixing. The changes here proposed would still impact the dev workflow in the main Authorino repo for cases where the change there involves modifying any of these resources. E.g., to add a new permission to the Authorino manager role, today you can change only in the Authorino repo; you don't even need to clone the Authorino Operator repo – just install/deploy the operator latest image and manifests, and in your branch for the Authorino repo, do the changes and re-apply on top.

Read this a few times, not totally sure i understand, but i think the problem is that i have added the authorino manifests to the install manifests of the operator? That's the only difference between this set of manifests and what was here previously, and can be undone. In my mind i just couldn't see the point of installing the operator CRDs without the authorino CRDs, but it's preferable we can leave it that way.
I can see as it is now that changes made in authorino would potentially require an update here due to us pulling the install manifests from the main branch, but that is true for what's on main now also, or am i just misunderstanding your point entirely?

Apart from the name prefix, which can be removed, they are unchanged. Only the means to which they are merged together for the OLM bundle and the deploy manifest file was updated.

If there is zero difference between the resources from the Authorino repo merged here to the Authorino Operator ones and how they are there in the Authorino repo originally, not even the names of the resources differ, then the means to which they are merged together do not matter and I am fine with the change.

Read this a few times, not totally sure i understand, but i think the problem is that i have added the authorino manifests to the install manifests of the operator? That's the only functional difference between this set of manifests and what was here previously, and can be undone. In my mind i just couldn't see the point of installing the operator CRDs without the authorino CRDs, but if it's preferable we can leave it that way.

To me, having the Authorino manifests merged together into the Authorino Operator's ones under config/install feels a bit weird because those are resources that do not belong to the Authorino Operator repo, the types they refer to are not defined in this repo, and the operator does not technically require those resources to be applied to the cluster in order to be installed or deployed. That said, it is not a big deal though. In the end, you are right... To use Authorino, users do need to install both sets of manifests. It's more about the what it means (at least in my mind):

Install the operator: apply the Authorino CRD plus Authorino Operator RBAC
Deploy the operator: create an Authorino Operator Deployment
Install Authorino: apply the AuthConfig CRD plus Authorino role definitions (just cluster roles, no bindings!)
Deploy Authorino: create an Authorino CR, which causes a running instance of the operator to create the Authorino Deployment, Service, ServiceAccount, RoleBinding and ClusterRoleBindings

I can see as it is now that changes made in authorino would potentially require an update here due to us pulling the install manifests from the main branch, but that is true for what's on main now also, or am i just misunderstanding your point entirely?

I guess you got it right. Whenever the Authorino manifests change in the Authorino repo, we need to send a PR to the operator, to update the handy config/deploy/manifests.yaml file that we host here (for simple one-line installation and deployment of the Authorino Operator's and Authorino's CRDs)... Since #23 was merged, to update the bundle manifests as well. (The contrary is not true obviously – i.e. changes in the operator manifests do not require any PR to the main Authorino repo.)
An idea we discussed when deciding about this organisation for the manifests was of bringing Authorino's AuthConfig and RBAC definitions all to the operator code base, and remove them from Authorino's one. This repo would define all the types and generate all the manifests. Authorino (the service) would still need the types though, so it would to have to import the Go packages, but no k8s resource YAML file hosted there. In the end, we went with the current separation to respect each component's responsibilities and relative independence one from another, that kind of exist to some extent. We can revisit this decision.