Keep getting "Rate Limit Exceeded" for webmasters.searchanalytics.query

We keep getting "Rate Limit Exceeded" errors, mainly for webmasters.searchanalytics.query. I am using the most up to date version of the google-api-php-client . Our users connect via OAuth with the service. Our app status in console.developers.google.com shows "published" and from what I understand the app and scopes are all validated by Google. The Metrics tab in de console shows I am never over 0.08 queries/s when an error occurs. The errors by API methods usually shows a 100% error rate pretty randomly when errors occur. We don't receive the error messages for all accounts. This is the exact error message we get:
{ "error": { "errors": [ { "domain": "usageLimits", "reason": "rateLimitExceeded", "message": "Rate Limit Exceeded" } ], "code": 429, "message": "Rate Limit Exceeded" } }
Once we received this error message we stop the API calls. The developer console never shows we reached the limit, or what error exactly caused us to hit the API limit. I currently request 10 urls per 5 minutes via a cronjob with a 2 sec delay per request and I still get the API Limit errors. We're able to have 100.000.000 per day and 2.000 queries per 100 second per user. De data we request is the keywords per url with a limit of 200. When trying to replicate the error by starting our script manually don't get the error messages.
I've got no idea anymore how to fix this, so hopefully someone can assist me to help me move forward.

I am not using the batch requests. After some more testing I seem to be able to reduce the number of errors to only a few accounts. I now only check 300 urls per 2 hours at most. That works for most of the websites that I am querying the data from. But not for a handful of domains. They keep giving the error messages. This is the code I am using (and is causing the problems)
   ```
    $service = new \Google_Service_Webmasters($client);
    $q = new \Google_Service_Webmasters_SearchAnalyticsQueryRequest();

    $q->setStartDate(date( 'Y-m-d', strtotime( '1 month ago' ) ));
    $q->setEndDate(date( 'Y-m-d', strtotime( 'now' ) ));
    $q->setDimensions(array('query'));

    $q->setRowLimit("200");

    $filter = new \Google_Service_Webmasters_ApiDimensionFilter();
    $filter->setDimension("page");
    $filter->setOperator("equals");
    $filter->setExpression($url);
    $filters = new \Google_Service_Webmasters_ApiDimensionFilterGroup();
    $filters->setFilters(array($filter));

    $q->setDimensionFilterGroups(array($filters));

    $q->setSearchType('web');
    $arrResult= array();
    try {
        $u = $service->searchanalytics->query($url, $q);
        $i = 0;
        foreach($u->rows as $row)
        {
           //do stuff
        }
    }
    catch(\Google_Service_Exception $e )
    {
    }

Have you solved you problems? I have the same problem. I slept [2, 4] secs for every request. I checked my google cloud platform and the peak queries per user per 100 secs is 50, but my quota is 2k. It makes fairly confused.

No, unfortunately I haven't been able to solve the problem. I've got a different process running with almost the exact same code but now retrieving the data per day and strangely enough that process isn't giving any problems at all. It also seems some sites / access tokens cause more problems than others. But I still haven't found the root cause of the error messages. Just like you said the Google Cloud Platform isn't reporting any problems at all and the error messages vary.

Hey, friend,
I just solved the problem. I thing the error log might be wrong. I catch the error and sleep 30 secs. After that, we send the query again, it works!

So you only add the sleep for 30 seconds after you catch the error? I've added a sleep of 5 sec between each API call but that still triggers the error.

I think its ok to get the error. Actually I use exponential backoff retry here. My initial sleeping time is 30 secs and it always succeed after sleeping 30 secs.

I'm going to close this issue out. If you're still having trouble, please feel free to re-open!

We keep getting "Rate Limit Exceeded" errors, mainly for webmasters.searchanalytics.query. I am using the most up to date version of the google-api-php-client . Our users connect via OAuth with the service. Our app status in console.developers.google.com shows "published" and from what I understand the app and scopes are all validated by Google. The Metrics tab in de console shows I am never over 0.08 queries/s when an error occurs. The errors by API methods usually shows a 100% error rate pretty randomly when errors occur. We don't receive the error messages for all accounts. This is the exact error message we get:
{ "error": { "errors": [ { "domain": "usageLimits", "reason": "rateLimitExceeded", "message": "Rate Limit Exceeded" } ], "code": 429, "message": "Rate Limit Exceeded" } }
Once we received this error message we stop the API calls. The developer console never shows we reached the limit, or what error exactly caused us to hit the API limit. I currently request 10 urls per 5 minutes via a cronjob with a 2 sec delay per request and I still get the API Limit errors. We're able to have 100.000.000 per day and 2.000 queries per 100 second per user. De data we request is the keywords per url with a limit of 200. When trying to replicate the error by starting our script manually don't get the error messages.
I've got no idea anymore how to fix this, so hopefully someone can assist me to help me move forward.


Why llvm14 if we're using inkwell master?

https://github.com/Rick-Lang/rickroll-lang/issues/76#issuecomment-2265117996

This was already answered on Discord, lol
TLDR: SW simply has that version installed


samples: display: lvgl: add support for pwm lcd backlight

Add support for PWM LCD backlight, if defined in the board. This permit to switch ON the backlight in this sample.

Thanks for following up your display contribution with that PR. As mentioned in the PR the lcd pwm is now separated out. The thing is that we now need to add this to all samples. I was wondering if the addition of a chosen node, something like display-backlight-pwm would allow to add a method into lvgl module which takes care of unblanking and setting the pwm. This would reduce boilerplate at least somewhat.
Any oppinions on that idea @jfischer-no @fabiobaltieri @joelguittet ?

@faxe1008 yeah that's what I was suggesting in the original thread (https://github.com/zephyrproject-rtos/zephyr/pull/67311#discussion_r1475133631), this is proposing an alias, not too sure what is "most correct" but to me it makes sense to have something that mimics the display identifier (zephyr,display)

@fabiobaltieri I think @faxe1008 think the alias can then be automatically handled by LVGL subsystem, correct ? I don't know if this is something that is possible.
@faxe1008 you said:

The thing is that we now need to add this to all samples

I don't understand that, why do we need to modify all samples ? Which sample are you thinking about exactly ? Actually the modification proposed to turn on the backlight of the LCD if the board itself defined the right pwm entry.

think the alias can then be automatically handled by LVGL subsystem, correct ? I don't know if this is something that is possible.

It should be possible yes. We can add a function to for example lvgl_display.h, e.g. lv_enable_display. This function would unblank the screen and turn on the backlight pwm. You can call this function right before entering the event loop, after you are done with your initial widget construction.
The general setup of lvgl requires a chosen zephyr,display alias to work, so I don't see why this should be out of flavor.

I don't understand that, why do we need to modify all samples ? Which sample are you thinking about exactly ?

There are more samples located at samples/modules/lvgl that would not work "out of the box" with lcds that require a pwm backlight.

It should be possible yes. We can add a function to for example lvgl_display.h, e.g. lv_enable_display. This function would unblank the screen and turn on the backlight pwm.

Got it. What should be prototype for this function ? I see different scenarii:

no backlight gpio/pwm defined
backlight gpio ON/OFF with possible revert logic
backlight with PWM, and in this case probably the settings of the PWM (at least the ratio) should be defined somewhere, no reason it should be 100% every time, isn't it ?

Does LVGL have a backlight control API?
It's tempting to suggest using the LED subsystem otherwise, it already abstracts the gpio vs pwm case, you can already define a backlight at either, for example
        pwmleds {
                compatible = "pwm-leds";
                backlight: backlight {
                        pwms = <&pwm0 0 PWM_HZ(250) PWM_POLARITY_NORMAL>;
                };
        };

or
        leds {
                compatible = "gpio-leds";
                backlight: led_0 {
                        gpios = <&gpio0 8 GPIO_ACTIVE_HIGH>;
                };
        };

and use it with the same API, something like

#define BACKLIGHT_LED_NODE DT_NODELABEL(system)
#define BACKLIGHT_LED_IDX DT_NODE_CHILD_IDX(BACKLIGHT_LED_NODE)

static const struct device *leds = DEVICE_DT_GET(DT_PARENT(BACKLIGHT_LED_NODE));

led_set_brightness(leds, BACKLIGHT_LED_IDX, 100);

It'd be silly to reimplement the same thing just renamed to "backlight" instead of "led"

@fabiobaltieri @faxe1008 I may be missing something, but why cannot display_set_brightness() be used for this? The API is already there, doing it on the side like this is not nice and duplicates code in the apps.

@jfischer-no because that's specific to the display device, it's certainly the right thing to use if the display controller includes brightness control AND it's used to control brightness in the specific board, but in reality it can either be that, or a separate circuit using a GPIO or PWM channel.
Maybe we could have a common structure added to the display API where one could specificy a backlight gpio or pwm channel, and then display_set_brightness would use that if provided or fallback to the display specific one. That may be the cleanest approach, needs a bit of redesign of the api though, adding a common structure and making sure it's initialized on all the drivers. It'd be nice though, what do you think?

@jfischer-no because that's specific to the display device, it's certainly the right thing to use if the display controller includes brightness control AND it's used to control brightness in the specific board, but in reality it can either be that, or a separate circuit using a GPIO or PWM channel.

I think this is not a problem, most of the small TFT LCD displays with integrated controller need PWM or GPIO to control the backlight. OLED can also use PWM to control brightness. Not sure about external panels connected to a display controller on the SoC. What external display controller actually has built-in brightness control?

Maybe we could have a common structure added to the display API where one could specificy a backlight gpio or pwm channel, and then display_set_brightness would use that if provided or fallback to the display specific one. That may be the cleanest approach, needs a bit of redesign of the api though, adding a common structure and making sure it's initialized on all the drivers. It'd be nice though, what do you think?

That could be placed in display_common.[c,h], not sure API change is needed for that, but Display Interface API status is unstable since 1.14.

@fabiobaltieri @jfischer-no I was just reading again the integration of the driver I'm currently using at https://github.com/zephyrproject-rtos/zephyr/pull/67012
Seems this was already discussed and seems also the pwm backlight was proposed initially, but then it was removed.
And indeed, the set_brightness member of the driver api is used only by display_dummy.c... No other driver using this.
So, does it means the set_brightness should be implemented in a display_common.c/h and optionally overriden by the display driver itself in case the backlight is controlled by the controller itself as @jfischer-no indicates ? Me too I don't have references where the display controller controls the backlight directly, but maybe they are.

@joelguittet there's a few more implementations (display_max7219.c, display_nrf_led_matrix.c and display_otm8009a.c). Other displays have it well, if you look at the ili9341 manual for example (https://cdn-shop.adafruit.com/datasheets/ILI9341.pdf) there's a reference to a LEDPWM pin. I also haven't seen it used anywhere, but no doubt someone out there is controlling and LED backlight from it and may end up being supported.
About supporting this at subsystem level, this is what I had in mind (@jfischer-no please confirm if we are in line with it)

add a pair of properties for a backlight gpio or pwm in `display-controller.yaml
add a matching common data structure to hold a struct gpio_dt_spec or struct pwm_dt_spec in drivers/display.h
add a macro to initialize the data in such structure, also in drivers/display.h, add the structure to all current display config structures, change all current display drivers to init the structures, this is not an uncommon pattern, see struct gpio_driver_config, struct regulator_common_config, struct input_kbd_matrix_common_config and the corresponding initializers
at this point every display driver will have your common data structure in their config, you can modify display_set_brightness to get that structure, check if a gpio or pwm is set, if it is use that one to set the brightness, otherwise fallback to the controller specific api, if any

This would allow a display controller to implement brightness control, and a board to use either that, a pwm or a gpio, all through the display set brightness API.

@fabiobaltieri thanks to point the several drivers, seems I have checked to quickly!
And thanks fro the detailed overview you have in mind, looks almost clear to me and I will be happy to contribute on this part :-)
@jfischer-no is the above proposal from Fabio good for you as well ?

@danieldegrasse ^^

@danieldegrasse ^^

The idea here makes sense to me. Breaking out of tree drivers is a legitimate concern, perhaps we need a note in the migration guide to highlight this? Not sure what else we can do here to easily detect a driver that is not ported to use the new generic code.


Luis app insights sample changed to MVC model

Proposed Changes

Luis Bot App insights sample changed to match the other MVC samples

Testing

I'm not sure what we're doing with this.

Closing since a new PR was submited


BorderSizeValue should support 'none' to hide the border

BorderSizeValue should support none such that one can conditional remove the border based on responsive value.
export type BorderSizeValue =
  | 'thin'
  | 'thick'
  | 'thicker'
  | 'thickest';

<View borderEndWidth={{ base: 'none', M: 'thin' }}>...</View>

Perhaps we could modify this function so it handles cases where the value is undefined like so
function borderSizeValue(value: BorderSizeValue) {
  return value ? `var(--spectrum-alias-border-size-${value})` :  undefined;
}

Then you could omit the base (or whichever responsive break point) definition entirely:
<View borderEndWidth={{M: 'thin' }}>...</View>

Thoughts?

Seems reasonable, and that seems like a good solution.

Hi, can I pick this one ?
Also, can we give support for both options so that the users can choose to omit or add none for more readability as per their need. What do you think ?
function borderSizeValue(value: BorderSizeValue) {
  return value && value !== "none"
    ? `var(--spectrum-alias-border-size-${value})`
    : undefined;
}

@pr7prashant Go for it! And that seems like a good solution.

I noticed that styleProps.test.js has tests for only one function dimensionValue. May I add tests for other exported functions in a separate issue if you think it would be helpful ?

Tests are always good to add. Thanks!


[BUG] Incorrect transitive dependencies in 3.1.0-pre.11

Bug Report Quick Checklist

[x] I am on the latest version of Snowpack & all plugins.
[x] I use package manager yarn/npm (Fill in: npm, yarn, pnpm, etc).
[x] I run Snowpack on OS mac (Fill in: Windows, Mac, Linux, etc).
[x] I run Snowpack on Node.js v12+

Describe the bug
I've found a case where transitive dependencies don't seem to be honored and deduping of dependencies crosses major version boundaries in v3.1.0-pre.11. I think this is because old versions of thee uuid package exposed entrypoints like 'v4' and 'v5' and that's confusing the deduping.
In a project depending upon:

<EMAIL_ADDRESS><EMAIL_ADDRESS>which has a transitive dependency on<EMAIL_ADDRESS>
<EMAIL_ADDRESS>exposed uuid/v4 and uuid/v5 entrypoints that are not available in<EMAIL_ADDRESS>When running the snowpack dev server it gives me the error:
[14:02:58] [snowpack] +<EMAIL_ADDRESS>[14:02:58] [snowpack] └──<EMAIL_ADDRESS>[14:02:58] [snowpack] └──<EMAIL_ADDRESS>(dedupe)
[14:02:58] [snowpack] +<EMAIL_ADDRESS>[14:02:58] [snowpack] Package(s) failed to build: Package "uuid" exists but package.json "exports" does not include entry for "./v4".

This is confusing to me:

Why is<EMAIL_ADDRESS>marked as a dedupe when it is the only reference to that package name + version + entrypoint in the whole project?
Why am i getting an error message that talks about a version of uuid that contains BOTH an exports field and a ./v4 entry when no such version exists<EMAIL_ADDRESS>includes a package.jeson exports field, and only has a default entrypoint, while<EMAIL_ADDRESS>has no entrypoint but does contain a './v4' entry.

This is seems to be because uuidv4 tries to load<EMAIL_ADDRESS>which does not expose the /v4 and /v5 entrypoints, instead of loading<EMAIL_ADDRESS>which does contain an uuid/v4 entrypoint.
To Reproduce
Here's the reproduction.
This particular reproduction is slightly contrived, but where I ran into this with real-world code was with private packages that I wouldn't be able to share publicly.

Clone that repository
cd incorrect-transitive-dependency
npm install and `npm start
Receive error

Expected behavior
No error.
I would expect that when building uuidv4 it would use uuid/3.4.0 and that when viewing generated output for uuidv4 in the browser I would see an import referencing<EMAIL_ADDRESS>/v4 entrypoint.

@FredKSchott's original response to this was:

@BPScott yea, old versions of uuid (pre-export map) don't seem to work with Node's CJS named import scanner and fail in ESM<>CJS envs. You can work around this by adding namedExports, but this package is so popular I think I'll go ahead and add it to always treat uuid as namedExports.

To which I replied:

That reproduction never uses named exports from the old version of uuid though so I'm slightly surprised that adding namedExports config would fix it. This named import uses<EMAIL_ADDRESS>per package.json, which ships an esm version, and<EMAIL_ADDRESS>calls the default import of<EMAIL_ADDRESS>(which ships cjs only).
I would have thought that I'd be able do both of these at the same time:

In my app code that depends upon<EMAIL_ADDRESS>use a named import when referencing uuid (as v8.3.2 ships native esm with named exports)
In a dependency that depends upon<EMAIL_ADDRESS>use a default export when referencing uuid (as 3.4.0 ships cjs only)

And Fred responded with

That's correct that both of those (named in v8, default in v3) should be possible without config

An alternative rephrasing, originally made here:
Rephrasing in case it might help:
The console output is:
[14:02:58] [snowpack] +<EMAIL_ADDRESS>[14:02:58] [snowpack] └──<EMAIL_ADDRESS>[14:02:58] [snowpack] └──<EMAIL_ADDRESS>(dedupe)
[14:02:58] [snowpack] +<EMAIL_ADDRESS>[14:02:58] [snowpack] Package(s) failed to build: Package "uuid" exists but package.json "exports" does not include entry for "./v4".

From here two things jump out:

Why is<EMAIL_ADDRESS>marked as a dedupe? It is the only reference to that combo of package name, version and entrypoint (uuid, 3.4.0 and /v5 respectively) in whole dependency tree.
Package "uuid" exists but package.json "exports" does not include an entry for "./v4". is strange because the uuid/v4 path only exists in<EMAIL_ADDRESS>however 3.4.0's package.json does not contain an exports map - The exports map only exists in v8.3.2 but v8.3.2 doesn't contain any additional entrypoints. It's not clear from the error but one of two things seem to be happening

When processing<EMAIL_ADDRESS>it tries to look at the package map from v8.3.2. I would have thought it should look for the package map from v3.4.0 - where there isn't a package map so there shouldn't be any limiting of entrypoints.
When processing<EMAIL_ADDRESS>it tries to create /v4 and /v5 entrypoints, those entrypoints aren't present in<EMAIL_ADDRESS>, only in<EMAIL_ADDRESS>I would have thought it should only use entrypoints from v8.3.2 - which is just the default entrypoint, no subpaths.

Putting it another way:

<EMAIL_ADDRESS>contains no export map in its package.json. It exposes a default entrypoint uuid and two paths uuid/v4 and uuid/v5
<EMAIL_ADDRESS>contains an export map in its package.json. It exposes only the default entrypoint.

Why do I get an error that suggests snowpack is trying to parse an export map for a package that contains the /v4 entrypoint when neither<EMAIL_ADDRESS>or<EMAIL_ADDRESS>contain both an export map and a /v4 entrypoint?

Good catch, looks like we were tracking your imports internally by package name and not package name+version. Fixed in 27321b83


⚠ Bofrost.lu has degraded performance

In 1ef286e, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1460 ms

Resolved: Bofrost.lu performance has improved in b78cafb after 9 minutes.


coc.nvim skip semantictoken provider

Describe the bug
FYI:
coc.nvim
vscode-languageserver
lua-language-server
full should be true or nil if support semantictoken.

Sorry, look like I misunderstand the range and full. And coc.nvim doesn't support rang for now.


Add test suites fot DS certification

Fix #9
Add following test suites:

1_service_operation.robot
2_configuration_setting.robot
3_device_profile.robot
4_device.robot
5_data_transformation.robot
6_query_commands.robot
7_actuation_commands.robot

@ramya-ranganathan @vishwassp-hcl-intel
Could you please help review this PR?
You can also follow the README.md to run the test.

We have more discussion on the slack, see https://edgexfoundry.slack.com/archives/CE44Z88NL/p1573451717037400

Approved for merging! thanks for accommodating the change requests.


docs: update import module getting start

This hotfix only adjusts the README on how to import the project.

Why are you having to import like that?
For me, the code below works just fine.
import moipSdk from 'moip-sdk-node-ts';

moipSdk.connect({
	production: false,
	accessToken: 'secret',
});

Why are you having to import like that?
For me, the code below works just fine.
import moipSdk from 'moip-sdk-node-ts';

moipSdk.connect({
	production: false,
	accessToken: 'secret',
});

I'm not sure. When using the lib in my provider, I can't instantiate it in the exact way it appears in the documentation, only in the proposed format. I haven't tested it on other projects yet.

Why are you having to import like that?
For me, the code below works just fine.
import moipSdk from 'moip-sdk-node-ts';

moipSdk.connect({
	production: false,
	accessToken: 'secret',
});

I'm not sure. When using the lib in my provider, I can't instantiate it in the exact way it appears in the documentation, only in the proposed format. I haven't tested it on other projects yet.

I must say this behavior is a little awkward. I made a minimal example at CodeSandbox and it seems to work fine.
https://codesandbox.io/s/holy-snow-gkdemq?file=/src/index.ts

Why are you having to import like that?
For me, the code below works just fine.
import moipSdk from 'moip-sdk-node-ts';

moipSdk.connect({
	production: false,
	accessToken: 'secret',
});

I'm not sure. When using the lib in my provider, I can't instantiate it in the exact way it appears in the documentation, only in the proposed format. I haven't tested it on other projects yet.

I must say this behavior is a little awkward. I made a minimal example at CodeSandbox and it seems to work fine.
https://codesandbox.io/s/holy-snow-gkdemq?file=/src/index.ts

Strange. It could be some configuration of my project.

import moipSdk This expression is not callable.   Type 'typeof import("PROJECT_PATH/node_modules/moip-sdk-node-ts/dist/lib/index")' has no call signatures.ts(2349)
This is resolved when performing the instance with the default (according to PR) or when I change the import of index.ts to the following:

Why are you having to import like that?
For me, the code below works just fine.
import moipSdk from 'moip-sdk-node-ts';

moipSdk.connect({
	production: false,
	accessToken: 'secret',
});

I'm not sure. When using the lib in my provider, I can't instantiate it in the exact way it appears in the documentation, only in the proposed format. I haven't tested it on other projects yet.

I must say this behavior is a little awkward. I made a minimal example at CodeSandbox and it seems to work fine.
https://codesandbox.io/s/holy-snow-gkdemq?file=/src/index.ts

Strange. It could be some configuration of my project.

import moipSdk This expression is not callable. Type 'typeof import("PROJECT_PATH/node_modules/moip-sdk-node-ts/dist/lib/index")' has no call signatures.ts(2349)
This is resolved when performing the instance with the default (according to PR) or when I change the import of index.ts to the following:

Ah, right. The documentation is indeed wrong. You should do the following:
this.client = moipSdk.connect({ ... })

Why are you having to import like that?
For me, the code below works just fine.
import moipSdk from 'moip-sdk-node-ts';

moipSdk.connect({
	production: false,
	accessToken: 'secret',
});

I'm not sure. When using the lib in my provider, I can't instantiate it in the exact way it appears in the documentation, only in the proposed format. I haven't tested it on other projects yet.

I must say this behavior is a little awkward. I made a minimal example at CodeSandbox and it seems to work fine.
https://codesandbox.io/s/holy-snow-gkdemq?file=/src/index.ts

Strange. It could be some configuration of my project.

import moipSdk This expression is not callable. Type 'typeof import("PROJECT_PATH/node_modules/moip-sdk-node-ts/dist/lib/index")' has no call signatures.ts(2349)
This is resolved when performing the instance with the default (according to PR) or when I change the import of index.ts to the following:

Ah, right. The documentation is indeed wrong. You should do the following:
this.client = moipSdk.connect({ ... })

That solves it :smiley:


Added checks to ClientBehavior updateables to avoid updates being executed before CB has finished initialising

This PR is a fix to a bug/feature we encountered.
We have a manually triggered state transition where updateable client behaviours can be initialised and destroyed at any time (similar to an interrupt). However, we found that at unpredictable  times, segfaults occur in the CB's update loop. Debugging showed that the CB's class members are called before they have properly been initialised. Indeed, there don't seem to be any checks to see if the CBs' onEntry methods have finished executing and the class member initialised before their executeUpdate methods are called.
To address this, I've added an initialized flag to the ClientBehavior base class and a corresponding check in the signal detector class. Furthermore, I've modified the time updateables are initialised with to use ros::Time::now instead of 0. The idea is to further reduce the risk of executeUpdate being called in the first update loop, though its not clear ow it will affect working with the ROS sim and real time clocks.
Potential issues:

Use of reinterpret_cast in update loop to cast to CB base class to access initialized flag may be slow. Potentially better to cache this once as is done for the updatable elements?
Not quite clear how modifying initial time values will affect working with ROS sim and other clocks.

We implemented this fix in our own implementation (though slightly differently) and it seems to have fixed the issue we had. Hopefully someone else finds it helpful

Hello @yassiezar I am reading you PR and I agree this is something interesting to address. I think this is also related with the https://github.com/reelrbtx/SMACC/issues/51, isn't it?
The first I have to say is that SMACC should be prepared to receive any incoming event at any moment (even from any thread asynchronously) without generating any segmentation fault or problem on objects such as client behaviors.
At the current moment there is already a mechanism to avoid update calls during any transition. That mechanism is not only valid for ClientBehavior but also for other kind of objects such as components, clients or state reactors.
Essentially: "Any update is freeze during a transition"
Please have a look to this line of code:
https://github.com/reelrbtx/SMACC/blob/05f00f6504ff6c30ecf3865a27ec223ff3524225/smacc/src/smacc/signal_detector.cpp#L197
I think this discussion is rather related with the concepts. Should be the object be initialized in onEntry? should it be already initialized?
The current implementation considers the object is already initialized in onEntry. Any special initialization code is considered to be located in the constructor or in the onRuntimeConfigure method.
In the current implementation update or onEntry method could happen at any possible order.
Should we change this? Maybe, but we should discuss why. For me it was clear that the object should be already initialized in onEntry.

Hey @pabloinigoblasco. You're almost right - in trying to fix the issue described in #51, we came across this issue. In particular, when we tried adding mutexes, I noticed segfaults would occur on lock_guard() calls on the class member mutex. Googling seemed to indicate that the likely culprit was that the mutex simply didn't exist or wasn't constructed correctly. So I tried adding a flag to check for initialisation on the update() method (the flag is set at the end of the onEntry()` method) and the error was fixed (at least partially - running into different errors = progress, right?).
Looking at my code again, I see the flags were mostly used to prevent client members of a given CB class from being accessed before being assigned in the CB's onEntry() method. So it seems you're right, this is mostly a question of when to consider an object as initialised. I agree with you that one should consider the CB as constructed by the time its onEntry() is reached - it makes intuitive sense and seems to work well in almost all other cases. The way I see it is that the onEntry() methods' function is to define the CB's behaviour and it doesn't make intuitive sense to me that its allowed to update anything before this takes place. Particularly in this use-case where the members might not have been completely initialised before they are accessed. For example, member pointers to clients are certainly initialised by the time onUpdate() is called, but can be undefined if onEntry() has not finished running - a classic race condition which isn't immediately obvious to casual users.
The way I see it is that there are 2 options to address this issue:

Make it very clear (documentation, examples,etc.) that class members accessed in the update call need to be defined earlier inside the state's onRuntimeConfigure() method (it wasn't clear to me that this was even an option for CBs before now, but I'm keen to try it),
make provision for developers who access class members inside their onUpdates() before they were initialised in onEntry().

Option 1 would be the easiest to do and will not need any code modification, but will require very clear documentation, especially since this issue only seems to apply to updateable elements and does not present itself consistently. If this issue was an edge-case, I'd go for this option as a footnote somewhere in a tutorial or class doc or something. But in my opinion, this is a decently typical use-case for CBs and updatables and should be accommodated for in the base library itself.
I therefore think that option 2 is the way to go. How this should be done is an open question though and my PR isn't the only solution. I also don't think that modifying the current initialisation paradigm needs to necessarily change either. Enforcing correct behaviour at at compile-time would be great, but I'm not sure how it would be done. Runtime checks will likely be easiest to implement - e.g. add a try-catch around calls to updatable CBs (though here you run the risk of missing other errors), adding a flag to prevent updates from being called prematurely, etc.

Thanks for your work.
PR #61 was accepted as replacement of this PR.


Log timeouts in Identity functional tests

This might help us debug the identity functional test hangs.
FYI @dougbu @anurse. I'm only turning the logs on for this specific test assembly for now. I'm not sure if we can turn this on globally since in the documentation, it specifically says that there is no merging of different json files: https://xunit.net/docs/configuring-with-json. I'd imagine we'll need a custom target that runs before tests to create/append the configuration with global settings. Could explore this as a followup but I don't really have bandwidth for this right now.

Retargeting to release/3.0 and will send a tell mode email.

Actually I don't think we need this anymore, it looks like @javiercn already fixed the underlying issue. For the long term I'd like to see this option turned on for all of our tests to make it more diagnosable in the future. However, that will require a bit more work and I'll file a followup issue.


consistent command scheme

fixes https://github.com/xh3b4sd/anna/issues/26

Also fixes https://github.com/xh3b4sd/anna/issues/27

Also fixes https://github.com/xh3b4sd/anna/issues/25

Because I missed to squash commits in the last PR, I did it here.


[NTRN-163] implement dex stargate contract

task: https://hadronlabs.atlassian.net/browse/NTRN-163
this PR:

adds a contract that uses neutron-sdk's dex stargate query and msg helpers

https://github.com/neutron-org/neutron-tests/actions/runs/7111070219

https://github.com/neutron-org/neutron-tests/actions/runs/7111070219

https://github.com/neutron-org/neutron-tests/actions/runs/7206923966


Removed ratelimit_per_user field from create_news_channel

Summary
Removed ratelimit_per_user field from create_news_channel since news channels dont have a ratelimit field
Checklist

[x] I have run nox and all the pipelines have passed.
[x] I have made unittests according to the code I have added/modified/deleted.

Related issues

since news channels dont have a ratelimit field

they do though...
https://github.com/user-attachments/assets/c604432f-e71e-4b39-ada6-7b591bab3f7c
Besides the "janky" way of setting them using the normal client, they work as normal.

since news channels dont have a ratelimit field

they do though...
Besides the "janky" way of setting them using the normal client, they work as normal.

Well are u sure the ratelimit affects users if the channel is set as news channel?
If so, then it would be needed to add the ratelimit field to the GuildNewsChannel.

Well are u sure the ratelimit affects users if the channel is set as news channel?

yep

Well are u sure the ratelimit affects users if the channel is set as news channel?

yep

then my bad, do you know if the ratelimit field is sent in the news channel payload?
Ig it should be sent, that means i will add the ratelimit field to the NewsChannel class in the next hours


Absolute import "Cannot find module @/*"

I have cloned and installed necessary dependencies.
When I run yarn start, I got
[nodemon] starting `ts-node  --transpile-only ./src/app.ts`
Error: Cannot find module '@/config'
Require stack:
- /home/react/Desktop/friends-backend/src/loaders/logger.ts
- /home/react/Desktop/friends-backend/src/app.ts

I am not sure why this is happening. I restart or clean cache my vs code and yarn package. But it does not still work.

@nayyaung9  how did your resolve this? I am using the "@/config" as import path but still getting errors
@RobertoGallego  is any extra setup needed besides installing tsconfig-paths? I tried installing it and I get the same error.
// loaders/logger.ts
import config from "@/config";

@Nikola-Milovic  Using absolute import in local is work well. But when deploying on heroku. it show "module not found" :((
But I used with relative paths in heroku. @RobertoGallego  Is there any extra config for that?

@nayyaung9  can you reopen the issue? It seems we could use some help on this :rofl:

Hello, i didnt try to run with Heroku, I gave u 2 options for the issue, that works for me, i made some changes for the  tsconfig-paths, in the package.json i modified my api like this: "api": "nodemon --exec ts-node -r tsconfig-paths/register src/index.ts",
then change the app.js for index.js
cheers hope u can fix


/usr/include/features.h:185:3: warning: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-Wcpp]

after git clone and trying to compile it's showing
In file included from /usr/include/termios.h:25,
                 from kilo.c:40:
/usr/include/features.h:185:3: warning: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-Wcpp]
  185 | # warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
      |   ^~~~~~~
kilo.c: In function ‘editorRefreshScreen’:
kilo.c:953:19: warning: implicit declaration of function ‘time’ [-Wimplicit-function-declaration]
  953 |     if (msglen && time(NULL)-E.statusmsg_time < 5)
      |                   ^~~~

solved it, thanks

how did you solve it?


Migration to set the correct message limit for the Pro plan.

Description
Migration to set the correct message limit for the Pro plan.
We can't change the maxMessageLifetime on Poke yet, hence the need for this migration.

Risk

Deploy Plan

Was already reviewed as part of this PR, but the migration file ended up being removed thinking we could do the change manually in poke.
So PMRW.


Update csproj with current repo name etc?

I tried to fix all that in PR #3. (just merged) I'll do another pass to make sure I didn't miss anything

Oh, we should add a README too.


1.3 Agent stuck connecting to master

Seems that 1.3 agent can get stuck on connecting to master. I had "bouncy" connection between some nodes and master and after master node reboot, couple agents got stuck:
W, [2017-08-23T09:27:35.939126 #1]  WARN -- Kontena::WebsocketClient: keepalive ping 5.03s of 5.00s timeout
W, [2017-08-23T09:27:35.997351 #1]  WARN -- Kontena::WebsocketClient: connection closed with code 1000:
I, [2017-08-23T09:27:36.040636 #1]  INFO -- Kontena::WebsocketClient: connecting to master at wss://35.156.95.xxx
I, [2017-08-23T09:27:36.317672 #1]  INFO -- Kontena::WebsocketClient: connection established
E, [2017-08-23T09:30:35.916910 #1] ERROR -- Kontena::WebsocketClient: keepalive ping 5.00s timeout, closing connection
W, [2017-08-23T09:30:39.506854 #1]  WARN -- Kontena::WebsocketClient: keepalive ping 8.59s of 5.00s timeout
W, [2017-08-23T09:30:39.542442 #1]  WARN -- Kontena::WebsocketClient: connection closed with code 1000:
I, [2017-08-23T09:30:40.488345 #1]  INFO -- Kontena::WebsocketClient: connecting to master at wss://35.156.95.xxx
I, [2017-08-23T09:30:40.763119 #1]  INFO -- Kontena::WebsocketClient: connection established
E, [2017-08-23T09:31:05.914622 #1] ERROR -- Kontena::WebsocketClient: keepalive ping 5.00s timeout, closing connection
W, [2017-08-23T09:31:17.494906 #1]  WARN -- Kontena::WebsocketClient: connection closed with code 4030: ping timeout after 10.49s
I, [2017-08-23T09:31:17.769797 #1]  INFO -- Kontena::WebsocketClient: connecting to master at wss://35.156.95.xxx
I, [2017-08-23T09:31:25.103479 #1]  INFO -- Kontena::WebsocketClient: connection established
W, [2017-08-23T09:31:26.389103 #1]  WARN -- Kontena::WebsocketClient: connection closed with code 4020: agent clock offset 8.46s exceeds threshold
I, [2017-08-23T09:31:26.914499 #1]  INFO -- Kontena::WebsocketClient: connecting to master at wss://35.156.95.xxx
E, [2017-08-23T09:32:33.277268 #1] ERROR -- Kontena::WebsocketClient: connection error: Error during WebSocket handshake: Unexpected response code: 504
W, [2017-08-23T09:32:33.281411 #1]  WARN -- Kontena::WebsocketClient: connection closed with code 1002: Error during WebSocket handshake: Unexpected response code: 504
I, [2017-08-23T09:32:34.156822 #1]  INFO -- Kontena::WebsocketClient: connecting to master at wss://35.156.95.xxx
E, [2017-08-23T09:34:05.637931 #1] ERROR -- Kontena::WebsocketClient: connection error: Error during WebSocket handshake: Unexpected response code: 504
W, [2017-08-23T09:34:05.639734 #1]  WARN -- Kontena::WebsocketClient: connection closed with code 1002: Error during WebSocket handshake: Unexpected response code: 504
I, [2017-08-23T09:34:06.505256 #1]  INFO -- Kontena::WebsocketClient: connecting to master at wss://35.156.95.xxx
E, [2017-08-23T09:35:40.967060 #1] ERROR -- Kontena::WebsocketClient: connection error: Error during WebSocket handshake: Unexpected response code: 504
W, [2017-08-23T09:35:40.968863 #1]  WARN -- Kontena::WebsocketClient: connection closed with code 1002: Error during WebSocket handshake: Unexpected response code: 504
I, [2017-08-23T09:35:41.889969 #1]  INFO -- Kontena::WebsocketClient: connecting to master at wss://35.156.95.xxx

Effectively, there's no timeout while connecting in 1.3 Agent websocket client. As whole WS client is re-implemented in 1.4 this should be fixed.
Agent restart fixes this as expected.

Closing this issue, because I don't think we'll be fixing it for 1.3, and it doesn't make sense to have it hanging around as an open issue for 1.4.
I assume the 1.4 agent would reconnect with an timeout error in this situation:
I, [2017-09-11T12:06:06.255191 #1]  INFO -- Kontena::WebsocketClient: connecting to master at wss://<IP_ADDRESS>:9293
E, [2017-09-11T12:06:16.273411 #1] ERROR -- Kontena::WebsocketClient: websocket error: SSL connect read timeout after 10.0s

I, [2017-09-11T12:07:28.634321 #1]  INFO -- Kontena::WebsocketClient: connecting to master at wss://<IP_ADDRESS>:9293
E, [2017-09-11T12:07:38.677490 #1] ERROR -- Kontena::WebsocketClient: websocket error: read timeout after 9.999975523s while waiting 10.0s for open


Documentation for Tooltip is incorrect

On the Tooltip documentation page, the default example is:
<script>
  import { Tooltip } from "carbon-components-svelte";
</script>

<Tooltip tooltipBodyId="tooltip-body">
  <p id="tooltip-body">
    Resources are provisioned based on your account's organization.
  </p>
</Tooltip>

Though tooltipBodyId doesn't exist and must have been changed to tooltipId and overlooked the examples.

Nice catch; updating the examples in #1271.


[bug] Crash without error in dev console when object remove itself from the scene tree

Occurs in current head(6637a865187422b5b8303231dd43a38a5f5abd51) as well as 1.1
The worst crashes are those which don't give any output in developer console, and gdb dont help either since for some errors all he is telling is like "Program received signal SIGSEGV, Segmentation fault. 0x000000000044aeb8 in ?? ()"
Anyway I reproduced one of these errors and it turns out in it's core it's very easy to reproduce:

Just call get_parent().remove_child(self);

Some output in devconsole on why game crashes would be nice. Especially that in real projects it usually will not to be so obvious since it can crash in calls similar to this one ("root/global").generateNextLvl().
Also not sure if the project should crash in the first place, since I can imagine scenario when some objects might want to remove itself from the tree... Should I edit title/description to take this into account instead of no error in console?
Some more observations:
The crash ocours not in the moment of removing itself from the tree, but when exiting from function call. I mean if generateNextLvl() have 300 lines it don't matter when we remove caller from the tree, crash will always occur when exiting from this function (after line 300 in this case).
Sample simple project (really not sure if needed but just in case):
https://dl.dropboxusercontent.com/u/168251/godot/some/issues/SelfDeleteCrashWithoutError.7z

Probably godot calls to caller_node.get_tree() at some point after method execution termination and tries to use it without checking if it's null

Does this happen on a callback from an input event?

@reduz tested it again, it's less common than what I wrote in the first post.
Currently it looks like this happens only in case when button is sending a signal and receiver in a reaction for this signal is removing this button from the tree.
Sample from the first post is still valid.

Related to #2578 I guess?

@akien-mga yes currently it's exactly the same issue. There is no error in console, just nameless crash.

I have experienced somewhat related bug in my main project today, and it's not related to input event directly. Basically I have an body that I when entering an area (via signal call) is caching itself in another node and removing itself with get_parent().remove_child(self). Soon after that (not instantly, after couple seconds) I'm experiencing crash without any error in console (very similar to one from the sample). I'm unable to prepare proper sample project (really tried :(), but I have here is gdb backtrace (It's different from the one from the first post crash):
Program received signal SIGSEGV, Segmentation fault.
0x00000000012431c1 in Set<VisualServerRaster::Instance*, Comparator<VisualServerRaster::Instance*>, VisualServerRaster::OctreeAllocator>::_insert_rb(VisualServerRaster::Instance* const&) ()
(gdb) bt
#0  0x00000000012431c1 in Set<VisualServerRaster::Instance*, Comparator<VisualServerRaster::Instance*>, VisualServerRaster::OctreeAllocator>::_insert_rb(VisualServerRaster::Instance* const&) ()
#1  0x000000000122dd96 in VisualServerRaster::instance_pair(void*, unsigned int, VisualServerRaster::Instance*, int, unsigned int, VisualServerRaster::Instance*, int) ()
#2  0x0000000001240a8a in Octree<VisualServerRaster::Instance, true, DefaultAllocator>::_element_check_pairs(Octree<VisualServerRaster::Instance, true, DefaultAllocator>::Element*) ()
#3  0x0000000001257962 in Octree<VisualServerRaster::Instance, true, DefaultAllocator>::move(unsigned int, AABB const&) ()
#4  0x0000000001233561 in VisualServerRaster::_update_instance(VisualServerRaster::Instance*) ()
#5  0x000000000123371b in VisualServerRaster::_update_instances() ()
#6  0x0000000001236346 in VisualServerRaster::_draw_viewport_camera(VisualServerRaster::Viewport*, bool) ()
#7  0x00000000012379b1 in VisualServerRaster::_draw_viewport(VisualServerRaster::Viewport*, int, int, int, int) ()
#8  0x0000000001238b18 in VisualServerRaster::_draw_viewports() ()
#9  0x0000000001238de1 in VisualServerRaster::draw() ()
#10 0x0000000000417266 in Main::iteration() ()
#11 0x000000000041285d in OS_X11::run() ()
#12 0x000000000040ef80 in main ()

Can you provide a test caso wich always crashes?

@MarianoGnu for backtrace from the first post there is test project in the first post.
For the backtrace from my previous comment (which in the end don't seems to be related to this bug) I'm unable to recreate this in any way. Further more it seems that to fix it I needed to remove and add again problematic scene to the main level, so I'm totally confused on what it was.

Crash does no longer seem to happen. This bug was known since a while, but it required the recent rewrite of GUI code, moving the logic to viewport to be fixed. Thanks to that, this can be closed :D


Grid - extended options for Excel exports

The default exports to Excel from ag-Grids are very basic, lacking formatting, data types, a table structure with headers, and fine-grained control over what fields are / are not exported.
We should decide if we want to invest more in learning / wrapping up ag-Grid's own extensive Excel export functionality or if we instead want to build on what we have in hoist-sencha / hoist-core and use our server-side Excel generation via Apache POI.

@TomTirapani wondering if you would like to look at this one - a starting point might be to do a review of what ag-Grid offers and see if we think that could be a good / better alternative to our server-side stuff.
You know the server-side API and capabilities better than anyone, so if you think it would be quicker / more efficient to try extracting our store data first and POSTing it there, I would be fine with either.
Finally, we made some recent changes to GridModel and the column defs to make it easier to add and keep around extended attributes on those definition objects w/o them being fed to ag-Grid and triggering warnings.  So that should make it easier to standardize on an export formatter pattern or similar.


mafSurvival doesn't show confidence interval in KM plot

mafSurvival(maf=luad, genes=c("TP53", "TTN", "MUC16"), time="days_to_last_follow_up", Status="days_to_death", isTCGA=TRUE, showConfInt=TRUE)

Actually I found that the function mafSurvival.R does not have the code to process this argument.

Hello,
Sorry this argument is a reminiscent of older version, and has been deprecated since. I will remove this in next update. Or do you prefer to show CI bands?

Thanks for your reply. Does “deprecated” mean that CI was considered to be inappropriate / not recommended  in the KM plot?

No, its just previous versions used to show CI band, and now I removed the option. I will remove showConfInt argument in next update.
Would you prefer to include CI band?

I'm just learning to use this nice R package and testing available functions & arguments. It's not neccesary for me, but maybe someone else would need it.  : )

Ah got it. Thank you, will take care of it.


Missing crates folder when upload crate file

When I using the command download and upload for crates files:
freighter -c /mnt/rust crates download --init --upload --bucket rust-lang
I noticed the s3cmd display like this:
s3://rust-lang//gdcm-rs/gdcm-rs-0.5.0.crate
And the the s3cmd crate a folder name / in the bucket root path, and all files upload to the / folder.
I think the s3cmd command shuld like this:
s3://rust-lang/crates/gdcm-rs/gdcm-rs-0.5.0.crate

https://github.com/open-rust-initiative/freighter/commit/69ebda4c3de7d0a377013e3d35e2d78aa32a4c6c


how do i install it

hi
i've downloaded the zip file
extracted the file
now how do i install the plugin ?

cerebro isn't an elasticsearch plugin, it is a stand alone app. check https://github.com/lmenezes/cerebro#installation for how to run it :)

it means in the folder with the context of the zip package, there is a folder called bin, and an executable file named cerebro inside of it. you just have to execute it.

i turnes out i downloaded the master zip and it didnt have any bin folder
now it is working thank you!

the zip on extraction doesn't show bin folder

it also gives an error saying missing plugin.properties file as well


SP5: Breaking Change - WPF - Missing WpfPlotViewer

Issue: It's unfortunate but in an application I have we depend on ScottPlot.WpfPlotViewer() and that class is removed in SP5.
ScottPlot Version: 5.0.16
Code Sample:
// In console application
ScottPlot.Plot plt = new ScottPlot.Plot();
new ScottPlot.WpfPlotViewer(plt).ShowDialog();

Throws an exception: The type or namespace name 'WpfPlotViewer' does not exist in the namespace 'ScottPlot' (are you missing an assembly reference?)
In What's New it's not mentioned that this should happen.
(In that case I think the issue https://github.com/ScottPlot/ScottPlot/issues/308 is no longer valid)

Hi @chaojian-zhang, thanks for your suggestion!

we depend on ScottPlot.WpfPlotViewer() and that class is removed in SP5

I would like to add this feature to ScottPlot 5 or add documentation to clarify how users can achieve this functionality themselves. I'm really happy that you brought this to my attention. I'll look into this now and follow-up here later today!

In What's New in ScottPlot 5 it's not mentioned that this should happen.

There is a banner that indicates that ScottPlot 4 has many features, is still supported, and existing projects that use ScottPlot 4 don't have to upgrade. You are correct though that this missing feature isn't addressed specifically.

Hi @chaojian-zhang, I found some solutions that work using the existing version of ScottPlot:
Launch a new window from a Console application
public class Program
{
    [STAThread]
    static void Main()
    {
        ScottPlot.Plot plt = new();
        plt.Add.Signal(ScottPlot.Generate.Sin());
        plt.Add.Signal(ScottPlot.Generate.Cos());

        ScottPlot.WPF.WpfPlot wpfPlot = new();
        wpfPlot.Reset(plt);
        System.Windows.Window win = new() { Content = wpfPlot };
        win.ShowDialog();
    }
}

Launch a new window from the right-click menu
public partial class MainWindow : Window
{
    public MainWindow()
    {
        InitializeComponent();
        WpfPlot1.Plot.Add.Signal(Generate.Sin());
        WpfPlot1.Plot.Add.Signal(Generate.Cos());

        WpfPlot1.Menu.Add("Launch in New Window", (wpfPlot) => LaunchPlotViewer(wpfPlot.Plot));
    }

    static void LaunchPlotViewer(Plot plot)
    {
        WpfPlot wpfPlot = new();
        wpfPlot.Reset(plot);

        Window win = new() { Content = wpfPlot };
        win.ShowDialog();
    }
}

I'll update the library so this will be possible from console apps:
ScottPlot.WinForms.FormsPlotViewer.Launch(myPlot);
ScottPlot.WPF.FormsPlotViewer.Launch(myPlot);

information from this issue has been added to the website:
https://scottplot.net/faq/launch-console/

Outstanding efficiency!


How to use MVC6.Template

Hello Folks, Is there a How to use or read me for using MVC6.Template, there are 10 projects in the template and if you can give some direction on using this template it would be just great to put it to good use.

The only information available is in this article it's a bit outdated, but general ideas still apply.


Update template-common/config.yaml

Description
Update template-common/config.yaml to resolve issue #293

Additional context

What is the purpose of this pull request?

[ ] Bug fix
[ ] New feature
[x] Other

@puhuk thanks for the PR, can you please run the following commands to fix formatting issues:
# format
pnpm fmt
# lint
pnpm source_lint


The field with name 'SystemCreatedAt' and ID '2000000001' was found in the previous version, but is missing in the current extension

Describe the bug
I'm using the preview version 6.0.314405 of AL Language to verify that our apps will publish on BC17.
When using AppSourceCop to validate the app against the previous released version, I get error on rule AS0002 for every table.
Error message:
The field with name 'SystemCreatedAt' and ID '2000000001' was found in the previous version, but is missing in the current extension. This might break the upgrade of existing installations.
This is probably related to the upcoming feature documented here: https://docs.microsoft.com/en-us/dynamics365-release-plan/2020wave2/smb/dynamics365-business-central/data-audit-system-fields-are-added-every-table
To Reproduce
Steps and to reproduce the behavior:

Create an app for BC16, the app should contain at least one table.
Build the app file and copy the file to .alpackages folder
Configure AppSourceCop to compare to above app file
Increase version in app.json
Restart VSCode
Try to build your app file

Now you should receive above error.
Expected behavior
No errors should be thrown on the new system fields
Screenshots
N/A
5. Versions:

AL Language: 6.0.314405

Business Central: 17.0.15704.0

This issue should now be fixed in our developer preview for AL extensions with version higher than 6.0.323892.


Using an Azure MySQL DB is twice as slow as an AKS hosted MySQL DB

Good afternoon Will and Zach,
This is not an issue with the helm chart per say... but rather a question  regarding the recommendation to use a CSP PaaS DB vs a K8s POD based DB for production. I just tried migrating our site to an Azure MySQL Flexible instance and I am finding that the site is less responsive... about half as responsive. The Azure MySQL PaaS instance is using a Private Endpoint on the same vnet as the AKS cluster running the Drupal solution. The MySQL PaaS instance is also super beefy... like as big as the node pool running the whole Drupal solution ;-) and yet it still perform poorly (not that the POD based DB was fast either).
Have you experienced the same at Stats Canada? Do you have any idea why this would be the case?
Regards,
Bernard

What is even stranger is that using sysbench I can confirm that the Azure MySQL is way faster than the AKS POD MySQL instance... but yet serving pages is twice as slow when using Azure MySQL...
POD MySQL:
root@troubleshooting-shell:~# sysbench /usr/share/sysbench/oltp_read_write.lua --mysql-host=drupal-wxt-prod-mysql --mysql-port=3306 --mysql-user=wxt --mysql-password='ceX8JFS7rgzE839G' --mysql-db=wxt --db-driver=mysql --tables=3 --table-size=10000 --report-interval=10 --threads=128 --time=60 run 2>&1
sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 128
Report intermediate results every 10 second(s)
Initializing random number generator from current time

Initializing worker threads...

Threads started!

[ 10s ] thds: 128 tps: 36.17 qps: 870.10 (r/w/o: 630.92/154.36/84.82) lat (ms,95%): 5312.73 err/s: 0.00 reconn/s: 0.00
[ 20s ] thds: 128 tps: 43.40 qps: 879.95 (r/w/o: 619.73/173.61/86.60) lat (ms,95%): 4943.53 err/s: 0.00 reconn/s: 0.00
[ 30s ] thds: 128 tps: 44.80 qps: 872.40 (r/w/o: 605.60/176.70/90.10) lat (ms,95%): 4943.53 err/s: 0.00 reconn/s: 0.00
[ 40s ] thds: 128 tps: 44.70 qps: 896.60 (r/w/o: 630.40/177.00/89.20) lat (ms,95%): 5217.92 err/s: 0.10 reconn/s: 0.00
[ 50s ] thds: 128 tps: 42.60 qps: 860.10 (r/w/o: 602.20/172.80/85.10) lat (ms,95%): 5312.73 err/s: 0.00 reconn/s: 0.00
[ 60s ] thds: 128 tps: 43.40 qps: 863.50 (r/w/o: 601.70/175.00/86.80) lat (ms,95%): 5409.26 err/s: 0.00 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            37492
        write:                           10708
        other:                           5354
        total:                           53554
    transactions:                        2676   (43.09 per sec.)
    queries:                             53554  (862.41 per sec.)
    ignored errors:                      2      (0.03 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          62.0964s
    total number of events:              2676

Latency (ms):
         min:                                  202.49
         avg:                                 2926.82
         max:                                 9196.74
         95th percentile:                     5217.92
         sum:                              7832167.98

Threads fairness:
    events (avg/stddev):           20.9062/1.71
    execution time (avg/stddev):   61.1888/0.63

Azure MySQL:
root@troubleshooting-shell:~# sysbench /usr/share/sysbench/oltp_read_write.lua --mysql-host=myssc.mysql.database.azure.com --mysql-port=3306 --mysql-user=azureadmin --mysql-password='Canada123!' --mysql-db=wxt --db-driver=mysql --tables=3 --table-size=10000 --report-interval=10 --threads=128 --time=60 run 2>&1
sysbench 1.0.18 (using system LuaJIT 2.1.0-beta3)

Running the test with following options:
Number of threads: 128
Report intermediate results every 10 second(s)
Initializing random number generator from current time

Initializing worker threads...

Threads started!

[ 10s ] thds: 128 tps: 240.64 qps: 5108.67 (r/w/o: 3623.26/985.54/499.87) lat (ms,95%): 1708.63 err/s: 5.90 reconn/s: 0.00
[ 20s ] thds: 128 tps: 265.91 qps: 5424.82 (r/w/o: 3812.09/1074.22/538.51) lat (ms,95%): 1561.52 err/s: 6.60 reconn/s: 0.00
[ 30s ] thds: 128 tps: 235.31 qps: 4815.79 (r/w/o: 3391.73/947.14/476.92) lat (ms,95%): 1708.63 err/s: 6.30 reconn/s: 0.00
[ 40s ] thds: 128 tps: 201.70 qps: 4170.93 (r/w/o: 2940.32/818.91/411.70) lat (ms,95%): 2120.76 err/s: 8.30 reconn/s: 0.00
[ 50s ] thds: 128 tps: 273.79 qps: 5605.46 (r/w/o: 3943.40/1106.47/555.59) lat (ms,95%): 1479.41 err/s: 8.00 reconn/s: 0.00
[ 60s ] thds: 128 tps: 246.30 qps: 5032.82 (r/w/o: 3536.41/997.60/498.80) lat (ms,95%): 1648.20 err/s: 6.20 reconn/s: 0.00
SQL statistics:
    queries performed:
        read:                            212520
        write:                           59672
        other:                           29945
        total:                           302137
    transactions:                        14765  (241.76 per sec.)
    queries:                             302137 (4947.10 per sec.)
    ignored errors:                      415    (6.80 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          61.0717s
    total number of events:              14765

Latency (ms):
         min:                                   41.12
         avg:                                  525.00
         max:                                 5863.51
         95th percentile:                     1678.14
         sum:                              7751658.26

Threads fairness:
    events (avg/stddev):           115.3516/11.06
    execution time (avg/stddev):   60.5598/0.33

@bernardmaltais we actually solved this, and went from pretty horrible performance to it being roughly 3-5x faster.
You will need to use:
https://docs.microsoft.com/en-us/azure/mysql/howto-redirection
See how we did it:
https://github.com/drupalwxt/docker-scaffold/commit/103f2b6f6f9e1cc0db54bba6d2c2df889e27bde0

@bernardmaltais although i'm only using the regular Azure MySQL PaaS service with 512GB storage size for high IOPS. It works pretty well. I haven't tested the redirection module with the flexible service though.
https://github.com/canada-ca-terraform-modules/terraform-azurerm-mysql

@sylus Thank you for the pointer... I guess it is time to update our scaffold stuff ;-)
The flexible service is more "flexible" and actually less expansive than the basic offering... and can be made HA.

@bernardmaltais our problem at least when i last checked is the flexible service is missing some key security features such as key vault integration for CMK, threat_detection_policy, and managed identities.
We have started working on the terraform spec though and starting with the PostgreSQL one:
https://github.com/canada-ca-terraform-modules/terraform-azurerm-flex-postgresql

@bernardmaltais but yeah I'd be interested if the redirect module works with the flexible service so please let me know!

@sylus I know what I will work on this week-end.

I would check if the flexible service supports it in the regular Managed PaaS there is a configuration parameter. I wonder if it is there for the flex service?
On the server side, redirection also needs to be enabled. You can enable redirection by updating the redirect_enabled server parameter using the Azure portal or Azure CLI. You can also use "show global variables like 'redirect_enabled'" to check the current redirection enable status on server side.

I am not attached to the flexible version as I just started testing today ;-) I will rebuild with the basic version and make it working 1st. Then I can try switching to flexible and see if this also work... baby steps. Man, this is turning in much more work than I expected...

@sylus Just saw that redirection does not support Private Link... arg! Not so pleased about that. Azure is a bit of a bag of hurt when it comes to AKS and PaaS DB access!

Yeah though not AKS related since its really their database implementations that are the core problem. Azure Web Apps suffers from the same problems though now they have an option for the redirect issue as well.
https://stackoverflow.com/questions/63970567/php-on-azure-app-service-slow-performance-when-connected-to-azure-database-for-m
Interesting while PostgreSQL and MySQL suffer from this problem strangely MSSQL doesn't as they do something underneath the scenes for some reason.
Just a reminder don't forget to set your storage size to 512GB to get decent IOPS which is what we do in conjunction with the redirect setting. Then just do a reserved instance for a year or 3 if u can and will save 60%

@sylus I have been able to deploy and enable mysqlnd_azure... but I am not seeing any performance difference with my previous Flexible Private Endpoint DB... Maybe the responsiveness issue is not related to an Azure Gateway latency afterall... But it was worth trying:
bash-5.0$ php test.php
mysqlnd_azure.enableRedirect: 1
b1e662624122.tr2764.canadacentral1-a.worker.database.windows.net via TCP/IP
mysqli_result Object
(
    [current_field] => 0
    [field_count] => 1
    [lengths] =>
    [num_rows] => 294
    [type] => 0
)

The strange thing is that running sysbench show faster throughput and smaller latency when using the Azure MySQL DB vs POD... but real world site performance it at the opposite I have no idea what is going on.

So you did the following?
a) Enabled the mysqlnd redirection on php side
b) Enabled the redirection server side in the mysql portal in the database configuration parameters
c) You are using tls 1.2 and the baltimore cert
d) sqlproxy is disabled
The performance difference was night and day for me, a fresh drupal install went from about 20-25 mins down to about 4-6 mins. If you do a fresh install how long does it take? Also while it is doing the install and you go to the database and select metrics and then IOPS do you see it peaking above say 60% and not always below 10%?

I also did some minor tweaks MySQL wise but I don't think any of these settings were particularly important:
https://github.com/canada-ca-terraform-modules/terraform-azurerm-mysql/blob/master/main.tf#L85...L143
https://github.com/canada-ca-terraform-modules/terraform-azurerm-mysql/blob/master/variables.tf#L233...L271

Doing a DB restore (fairly small DB that take 10 seconds on my dev machine to restore) take about 5 minutes on POD. I did notice that the Azure MySQL was giving my equivalent page loads as the POD based DB... so maybe the direct connect made a slight difference in leveling things out... but DB performance is still bad compared to a local dev laptop.
I am not expecting performance to be equivalent... but I was hoping to at least reduce the small restore to take only 1 minutes instead of 5 ;-(

Yeah i'd expect a hit of around 20% in my experience then local and the MySQL db will def not be faster then an in cluster db as well due to networking and other considerations so that will never match unfortunately. I'd take a look at the mysql configuration I did above and set similar values which should help you a bit.
If you want near performance as your local sadly it will be flex server since technically gives you your own VM and SSD but the gateway problem will be your limiter.
I am using Gen5_v4 at 512GB storage and MySQL v8.0

OK, I will try making the suggested changes and then try upgrading the DB to match yours... maybe this will make a difference.

Yeah so:
Gen5_v4
512 GB for 1532 IOPS
mysql_version 8.0
innodb_buffer_pool_size<PHONE_NUMBER>0
max_allowed_packet 536870912
table_definition_cache 5000
table_open_cache 5000

I do use varnish... but not redis... maybe the fact that I am not using redis is putting too much pressure on the DB for caching... and is not helping...

Yeah but redis is mainly for logged in users and if you have lots of content / views and make user of lots of entity references etc.
Agreed 5 mins for a small db seems a bit long, I don't fully remember for the recalls site but it is about 15 gb uncompressed and I think takes about 15 mins to import, but I am doing a restore to dev tomorrow so can get a more accurate time.
Basically I just log into the container and do a:
drush sql-cli < db.sql

@bernardmaltais just one last tip just in case, copy your db.sql.gz to the /tmp folder in container and ensure not running the restore when its in the azure file mount or the restore will take forever.
cd /tmp
drush sql-drop
zcat db.sql.gz | drush sql-cli

@sylus Interesting point. I do restore from the backup file located in the Azure File volume... I will try to use a custom restore script to copy the file 1st to temp and see if this make a difference.
So far my testing has shown that running MySQL in a POD is still the fastest... Here are the time to do a restore on 3 different MySQL:
POD: 4m49s
Azure MySQL direct connect: 5m23s
Azure MySQL flexible with private endpoint: 7m1s
Bernard

Hey @bernardmaltais so I was wrong about flex it doesn’t need redirect so given that and it’s properly colocated it should absolutely not be slower then the single server.
As I to eventually want to move to flex server for the advantages and it being pure Linux I’d love to know what is going on as well.
Planning to take a look myself but please to let me know if found anything. I can even engage with azure support.

Oh did u check out thread pools for flex?
https://techcommunity.microsoft.com/t5/azure-database-for-mysql-blog/achieve-up-to-a-50-performance-boost-in-azure-database-for-mysql/ba-p/2909691

@sylus I will have a look at the blog post on flexible server. Not sure why it is the slowest for me... possibly a configuration issue... but you can most certainly feel the difference when accessing the site as pages load twice as slow. I reached out to MS about that and I have not gotten an answer. Not a support ticket but reaching to their AKS guy...

@sylus I have deployed redis from the chart. How can I tell if it is active? I just used the defaults from the chart values.yaml file... one of the line was commented out so I left it not knowing what it would do. Any recommendations? You site is super fast and I am trying to get close to that.
Also, have you ever written a tuning guide for your site? Asking for a friend (literally) ;-)

Hi @bernardmaltais for redis yeah all you should need:
Here is what mine looks like:
## Redis
## Ref: https://github.com/bitnami/charts/tree/master/bitnami/redis
redis:
  enabled: true
  password: ${var.redis_password_rsams}
  image:
    registry: docker.io
    pullSecrets:
      - acr-registry
  clientInterface: PhpRedis

You should then see a redis master and 2 replicas come up. The helm chart should enable redis for you but in case doesn't just run a drush en redis. Then you should see redis enabled and connection in the status report. With redis enabled the following gets enabled in settings.php:

https://github.com/drupalwxt/helm-drupal/blob/master/drupal/conf/settings.d9.php#L915...L988

Thanks. I was able to deploy and according to JMeter is reduced logged in user page load by about 20%. I saw that APCu is also recommended to be deployed on the drupal instance... but from what I could read it look like it is better fir for single node systems and that Redis essentially does the same thing. Have you done any testing between APCu and redis?

As i understand it there is no need for APC (or any of the similar sort of bytecode caching extensions like XCache) as of PHP 5.5 and later. The PHP developers directly integrated what they call OPCache which we are using.

Any word on Flex Server @bernardmaltais also did you enable proxysql im the helm chart? As they say should do that to.

@sylus I have not. I have been testing with the redirection. Once the log4j wave has passed I should have more time to look into that again. But for now the speed of the redirection is acceptable ;-)

Just wanted to chime back that I made the switch to Flex Server using the Business Critical tier and performance seems to be pretty good!


Support Prometheus and Grafana by Default

These are all the files we should need for Prometheus and Grafana. Unfortunately, I can't get the connection between the two to work.
From my notes when I first tried deploying these, I see

Cannot read property "result" of undefined
Prometheus as a data source wasn't configured properly. Needed "https://" before the url and "proxy" as the access method

I can create a dashboard, but everything linked to Prometheus says "Bad Gateway"
Fix by setting Grafana's access to Prometheus to Direct, answer from here.

These two solutions seem at odds with each other, and also neither of them appear to work for me. Enabling https might help (since one of the solutions wants that as the first part of your url), but I'm not sure. Just putting notes down to help solve the problem.

Oh, this also has some commits in place to enable making a standalone user in the aws-creds directory. Should I make that its own PR?

@yuvipanda , I set this up to work as a module under the current directory layout. There was a minor hiccup in deployment (I think related to which aws profile k8s sees in system:masters) that I'm going to try working on. Aside from that, running it as a module worked! I could load up grafana and confirmed the connection to prometheus as a data source was working.
A couple To-Dos:

[ ] Enable changing of grafana admin's username and password. These are present in grafana-values.yaml.
[ ] Enable secrets, mostly to mask the grafana admin's username and password.

After fiddling a bit with deployment options for Prometheus and Grafana, I think it makes the most sense to deploy them with the JupyterHub itself. This falls under the realm of hubploy or hubploy-template, so I will close the PR here.


Argo CLI error - FATA - configmaps "workflow-controller-configmap" not found

Pre-requisites

[X] I have double-checked my configuration
[X] I can confirm the issues exists when I tested with :latest
[ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?
At the time that I start Argo Workflows server via $ argo server I get the error configmaps "workflow-controller-configmap" not found.
All other commands (e.g. argo list, argo auth token) is not working as well.
We have full access to our pods via kubectl
Related issues:

https://github.com/argoproj/argo-workflows/issues/2285
https://github.com/argoproj/argo-workflows/issues/2158

Version
v3.4.1
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
`$ kubectl version -o yaml`

clientVersion:
  buildDate: "2022-08-23T17:36:43Z"
  compiler: gc
  gitCommit: a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2
  gitTreeState: clean
  gitVersion: v1.25.0
  goVersion: go1.19
  major: "1"
  minor: "25"
  platform: darwin/amd64
kustomizeVersion: v4.5.7
serverVersion:
  buildDate: "2022-08-16T09:24:51Z"
  compiler: gc
  gitCommit: e55564cf3a1384026a54920174977659c8c56a50
  gitTreeState: clean
  gitVersion: v1.22.12-gke.2300
  goVersion: go1.16.15b7
  major: "1"
  minor: "22"
  platform: linux/amd64

$ argo version
argo: v3.4.1
  BuildDate: 2022-10-01T15:51:47Z
  GitCommit: 0546fef0b096d84c9e3362d2b241614e743ebe97
  GitTreeState: clean
  GitTag: v3.4.1
  GoVersion: go1.18.6
  Compiler: gc
  Platform: darwin/amd64

$ argo server
INFO[2022-10-20T11:40:44.217Z] not enabling pprof debug endpoints
INFO[2022-10-20T11:40:44.219Z]                                               authModes="[client]" baseHRef=/ managedNamespace= namespace=default secure=true ssoNamespace=default
INFO[2022-10-20T11:40:44.219Z] Generating Self Signed TLS Certificates for Secure Mode
INFO[2022-10-20T11:40:44.220Z] SSO disabled
FATA[2022-10-20T11:40:44.361Z] configmaps "workflow-controller-configmap" not found

Logs from the workflow controller
INFO[2022-10-20T11:40:44.217Z] not enabling pprof debug endpoints
INFO[2022-10-20T11:40:44.219Z]                                               authModes="[client]" baseHRef=/ managedNamespace= namespace=default secure=true ssoNamespace=default
INFO[2022-10-20T11:40:44.219Z] Generating Self Signed TLS Certificates for Secure Mode
INFO[2022-10-20T11:40:44.220Z] SSO disabled
FATA[2022-10-20T11:40:44.361Z] configmaps "workflow-controller-configmap" not found

Logs from in your workflow's wait container
kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

Check if this returns anything kubectl get configmap workflow-controller-configmap.

@terrytangyuan Same message:
Error from server (NotFound): configmaps "workflow-controller-configmap" not found

I had the same problem. Try with:
argo server --auth-mode server -n argo

Thanks @ognjen-it, same error:
FATA[2022-10-26T10:13:24.008Z] configmaps "workflow-controller-configmap" not found

@fclesio
It's practically impossible...
Please, try the next commands and share the full logs that you got.
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: workflow-controller-configmap
  namespace: argo
data: {}
EOF

And then:
argo server --auth-mode server -n argo

@ognjen-it Worked. Thanks a lot.
The issue was that the configmap was missing. Thanks!

My post at https://github.com/argoproj/argo-workflows/discussions/7649
For some reason, the server wants to use the default kubernetes namespace, try stating the server with the argo namespace specified:
$ argo server --auth-mode server -n argo


Update the design with more efficient CPU

Is your feature request related to a problem? Please describe.**

Got Pinebook Pro recently and i hate that:

I don't have KiCAD files for it to easily add functionality that i need e.g. fingerprint, hotswappable batteries, etc..
The RK3399 has spectre and meltdown vulnerabilities and is allegedly using proprietary blobs

Describe the solution you'd like

My end goal is better, equal or tolerably worse alternative to Pinebook Pro that doesn't have security issues and proprietary blobs.
Can we do better with TERES-1? It's currently using the A64 which is too slow for me
Describe alternatives you've considered

The IMX8QM-TUKHLA seems to be a good alternative to RK3399 with allegedly no blobs, but it's still not released and it seems that it's development stalled likely due to blobs..
The next best option seems to be the iMX8QuadPlus which compared to RK3399 has one less Cortex-A72, but according to https://olimex.wordpress.com/2022/01/25/imx8mplus-som-is-alive-and-boots there is an issue with supply
Additional context

I have experience with hardware and software development and am willing to help if you can do the brainstorm on what CPU should be used.

IMX8QM processor alone is EUR 200 this is automotive grade processor and you are not correct, there is co-processor running non free firmware inside it, which takes care of all initializations of memory, etc. NXP provide SDK so you can write your own firmware, but do not have public the sources of the firmware which are used to ship the processors, which may need quite of work.
For IMX8MQP NXP is working to mainline it, so IMO is more suitable for FOSS OSHW product.

@TsvetanUsunov In hypothetical scenario where we have the documentation needed to know how to make a board for it:
What about SnapDragon Mobile 845?
The PostmarketOS wiki says that it's mainlined[https://wiki.postmarketos.org/index.php?title=Qualcomm_Snapdragon_845_(SDM845)&mobileaction=toggle_view_desktop]

Found RK3588S for which the datasheet is widely available and seems like a good option considering budget and resource efficiency.
In terms of mainlining, the work seems to be ongoing in 6.1

Relevant: https://github.com/starfive-tech/VisionFive2/issues/39

Considered won't fix


bugfix: strip status information for proper package id

When git status retrieved info, in some cases, it could include the status of a file, e.g. "added" or "modified". Stripping that off ensures that just the filename is reported and can be matched to the proper Go package for use in the unit-test-patch target (and friends).
Issue #, if available:
Description of changes:
Testing (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve


Add note to Organizational Settings section regarding inbound MFA claim

Add a note to exclude external users from MFA registration policy (if exists) in case the cross-tenant access is configured to accept inbound MFA claim.

@custorod : Thanks for your contribution! The author(s) have been notified to review your proposed change.

IMPORTANT: When this content is ready to merge, the reviewer must add #sign-off in a comment or the approval may get overlooked.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

@Jak-MS : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Obsolete pycc script/code

Since pycc.main() has been deprecated for almost 7 years (https://github.com/numba/numba/commit/7342cb9aa7aded02209605299dbbdbe377794078), it seems that the pycc script (shown below) should be removed or at least be archived in a way that does not deploy it to new installations. For that matter, is there a reason to keep pycc.main() around, or should removal/archival of the function be considered?
https://github.com/numba/numba/blob/61ec1fd0f69aeadece218dccf4c39ebc5c7dfbc4/numba/pycc/pycc#L1-L3
https://github.com/numba/numba/blob/61ec1fd0f69aeadece218dccf4c39ebc5c7dfbc4/numba/pycc/__init__.py#L26-L100

@firai thank you for this suggestion. As discussed in the developer meeting last night (https://github.com/numba/numba/wiki/Minutes_2022_08_09) this can indeed be removed.

Removed in #8335.


Installation Help

Good morning,
I was able to get through all of the installation process until the very end. I have my pacman set up and updated. Then the readme tells me to pacman -Syy
and then
curl [https://raw.githubusercontent.com/FT-Labs/phyOS-Aug-22/master/packages.x86_64](https://raw.githubusercontent.com/FT-Labs/phyOS-Aug-22/master/packages.x86_64) | sed -e '/\#/d' | tr -s "\n" | xargs sudo pacman -S --noconfirm
The former works just fine. But the latter returns zsh: unknown file attribute: h error: no targets specified (use -h for help
I literally copied and pasted that second curl code into my terminal, so I assume it's correct. But there are several commands I am not familiar with, so I'm posting this here for some guidance.
Thank you.

In the curl command delete the markup only the link should be there

curl https://raw.githubusercontent.com/FT-Labs/phyOS-Aug-22/master/packages.x86_64 | sed -e '/#/d' | tr -s "\n" | xargs sudo pacman -S --noconfirm
Could you try this please?

Same issue:
$ curl https://raw.githubusercontent.com/FT-Labs/phyOS-Aug-22/master/packages.x86_64 | sed -e '/#/d' | tr -s "\n" | xargs sudo pacman -S --noconfirm   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                  Dload  Upload   Total   Spent    Left  Speed 100  2923  100  2923    0     0  16930      0 --:--:-- --:--:-- --:--:-- 16994 [sudo] password for desktop:  warning: arandr-0.1.11-1 is up to date -- reinstalling warning: base-3-1 is up to date -- reinstalling warning: btrfs-progs-6.1.3-1 is up to date -- reinstalling warning: curl-7.87.0-3 is up to date -- reinstalling warning: cryptsetup-2.6.0-1 is up to date -- reinstalling warning: dbus-1.14.4-1 is up to date -- reinstalling warning: diffutils-3.8-1 is up to date -- reinstalling warning: dmraid-1.0.0.rc16.3-14 is up to date -- reinstalling warning: dosfstools-4.2-3 is up to date -- reinstalling warning: e2fsprogs-1.46.6-1 is up to date -- reinstalling warning: efibootmgr-18-1 is up to date -- reinstalling warning: fakeroot-1.30.1-1 is up to date -- reinstalling warning: ffmpeg-2:5.1.2-1 is up to date -- reinstalling warning: fzf-0.37.0-1 is up to date -- reinstalling warning: git-2.39.1-1 is up to date -- reinstalling warning: gpm-1.20.7.r38.ge82d1a6-4 is up to date -- reinstalling warning: gptfdisk-1.0.9-2 is up to date -- reinstalling warning: harfbuzz-6.0.0-1 is up to date -- reinstalling warning: lib32-gtk3-1:3.24.36-1 is up to date -- reinstalling warning: lib32-libpulse-16.1-1 is up to date -- reinstalling warning: lib32-libxcomposite-0.4.6-1 is up to date -- reinstalling warning: lib32-libxinerama-1.1.5-1 is up to date -- reinstalling warning: lib32-mpg123-1.31.2-1 is up to date -- reinstalling warning: lib32-nvidia-utils-525.85.05-1 is up to date -- reinstalling warning: lib32-v4l-utils-1.22.1-1 is up to date -- reinstalling warning: libxss-1.2.3-4 is up to date -- reinstalling warning: linux-6.1.9.arch1-2 is up to date -- reinstalling warning: linux-firmware-20230117.7e4f0ed-1 is up to date -- reinstalling warning: lvm2-2.03.18-4 is up to date -- reinstalling warning: lsb-release-2.0.r48.3cf5103-1 is up to date -- reinstalling warning: make-4.4-1 is up to date -- reinstalling warning: man-db-2.11.2-1 is up to date -- reinstalling warning: maim-5.7.4-5 is up to date -- reinstalling warning: mdadm-4.2-2 is up to date -- reinstalling warning: mediainfo-22.12-1 is up to date -- reinstalling warning: mpc-0.34-2 is up to date -- reinstalling warning: mpd-0.23.12-1 is up to date -- reinstalling warning: mpv-1:0.35.1-1 is up to date -- reinstalling warning: mkinitcpio-34-2 is up to date -- reinstalling warning: neovim-0.8.3-1 is up to date -- reinstalling warning: ntfs-3g-2022.10.3-1 is up to date -- reinstalling warning: openssh-9.2p1-1 is up to date -- reinstalling warning: parted-3.5-1 is up to date -- reinstalling warning: polkit-122-1 is up to date -- reinstalling warning: python-virtualenv-20.17.1-1 is up to date -- reinstalling warning: rsync-3.2.7-3 is up to date -- reinstalling warning: sg3_utils-1.47-1 is up to date -- reinstalling warning: sudo-1.9.12.p2-1 is up to date -- reinstalling warning: systemd-252.5-1 is up to date -- reinstalling warning: usbmuxd-1.1.1-2 is up to date -- reinstalling warning: usbutils-015-2 is up to date -- reinstalling warning: vlc-3.0.18-6 is up to date -- reinstalling warning: wget-1.21.3-1 is up to date -- reinstalling warning: wpa_supplicant-2:2.10-8 is up to date -- reinstalling warning: xfsprogs-6.1.1-2 is up to date -- reinstalling warning: zip-3.0-10 is up to date -- reinstalling warning: unzip-6.0-19 is up to date -- reinstalling warning: zsh-5.9-2 is up to date -- reinstalling warning: downgrading package zsh-fast-syntax-highlighting-git (r908.7c390ee-1 => r905.721be2a-1) error: target not found: gotop warning: avahi-0.8+22+gfd482a7-3 is up to date -- reinstalling warning: mobile-broadband-provider-info-20221107-2 is up to date -- reinstalling warning: networkmanager-1.40.12-1 is up to date -- reinstalling warning: xcape-1.2-4 is up to date -- reinstalling warning: xclip-0.13-3 is up to date -- reinstalling warning: xdotool-3.20211022.1-1 is up to date -- reinstalling warning: xorg-xinit-1.4.2-1 is up to date -- reinstalling warning: xorg-xmessage-1.0.6-1 is up to date -- reinstalling warning: xwallpaper-0.7.4-1 is up to date -- reinstalling warning: alsa-plugins-1:<IP_ADDRESS>-1 is up to date -- reinstalling warning: alsa-lib-1.2.8-1 is up to date -- reinstalling warning: pamixer-1.6-2 is up to date -- reinstalling warning: pipewire-1:0.3.65-6 is up to date -- reinstalling warning: pulsemixer-1.5.1-3 is up to date -- reinstalling error: target not found: plymouth-git warning: pass-1.7.4-3 is up to date -- reinstalling warning: xf86-input-libinput-1.2.1-1 is up to date -- reinstalling warning: xf86-video-vesa-2.6.0-1 is up to date -- reinstalling warning: mesa-22.3.4-1 is up to date -- reinstalling warning: bluez-5.66-1 is up to date -- reinstalling warning: bluez-libs-5.66-1 is up to date -- reinstalling warning: grub-2:2.06.r456.g65bc45963-1 is up to date -- reinstalling warning: nvidia-utils-525.85.05-1 is up to date -- reinstalling warning: nvidia-settings-525.85.05-1 is up to date -- reinstalling warning: mesa-utils-8.5.0-2 is up to date -- reinstalling warning: phyOS-keyring-20240603-4 is up to date -- reinstalling warning: cointop-1.6.10-1 is up to date -- reinstalling error: target not found: i3lock-color error: target not found: betterlockscreen warning: libnotify-0.8.1-1 is up to date -- reinstalling warning: libxft-2.3.7-1 is up to date -- reinstalling warning: rofi-1.7.5-1 is up to date -- reinstalling warning: ttf-linux-libertine-5.3.0-8 is up to date -- reinstalling warning: unclutter-1:1.6-2 is up to date -- reinstalling

it states everything is installed

Yes; but your explanation points out that after a reboot I'll be running on this window manager. This is not the case; I'm still on my old window manager.
I have a config file that opens dwm on login (after login, I then type in startx on tty), so maybe I need to call a different wm? Or maybe the config files didn't download? Or maybe I failed to follow directions elsewhere?

Are you using a display manager? There should be a pdwm option there. Also you can get the dotfiles by:
phyup dots --force


Error when processing data in event ... from MISP Invalid value for Note 'object_refs': referen ce to custom object type: x-misp-attribute

I Received 0 indicators in MISP but I am getting the following errors when processing MISP events:
2023-08-12 10:05:28,311 - misp2sentinel - INFO - Received MISP events page 1 with 100 events
2023-08-12 10:05:28,459 - misp2sentinel - ERROR - Error when processing data in event 494 from MISP Invalid value for Note 'object_refs': referen
ce to custom object type: x-misp-attribute
2023-08-12 10:05:28,466 - misp2sentinel - ERROR - Error when processing data in event 495 from MISP Invalid value for Note 'object_refs': referen
ce to custom object type: x-misp-attribute
2023-08-12 10:05:28,484 - misp2sentinel - ERROR - Error when processing data in event 496 from MISP Invalid value for Note 'object_refs': referen
ce to custom object type: x-misp-attribute
2023-08-12 10:05:28,603 - misp2sentinel - ERROR - Ignoring non STIX pattern type yara
2023-08-12 10:05:28,603 - misp2sentinel - ERROR - Unable to process indicator
2023-08-12 10:05:28,610 - misp2sentinel - ERROR - Ignoring non STIX pattern type yara
2023-08-12 10:05:28,610 - misp2sentinel - ERROR - Unable to process indicator
2023-08-12 10:05:28,610 - misp2sentinel - ERROR - Ignoring non STIX pattern type yara
2023-08-12 10:05:28,610 - misp2sentinel - ERROR - Unable to process indicator
2023-08-12 10:05:28,686 - misp2sentinel - ERROR - Error when processing data in event 503 from MISP Invalid value for Note 'object_refs': referen
ce to custom object type: x-misp-attribute
misp filters in config.py:
misp_event_filters = {
"published": 1,
#"tags": [ "sentinel-threattype="Malware""],
"enforceWarninglist": True,
"includeEventTags": True,
"publish_timestamp": "7d",
}

Can you check with the latest commit?
The Ignoring non STIX pattern type yara is normal. Microsoft Sentinel does not accept yara indicators for uploading via Upload Indicators API, these attributes are ignored, this causes the ERROR - Unable to process indicator.
The Invalid value for Note 'object_refs': reference to custom object type: x-misp-attribute is caused by the conversion by MISP to STIX (this is done with the misp-stix package). I don't know why, but some EventReports seem to be causing problems for this conversion, and it's more a bug that needs to be raised in misp-stix. I added an option that removes the EventReports from the event, before conversion takes place. Currently Microsoft Sentinel does not support STIX Notes (to which reports get translated) so you don't lose any functionality anyway. The change is in https://github.com/cudeso/misp2sentinel/blob/main/RequestObject.py#L172  and the value "misp_remove_eventreports" is set to True by default when it's missing from the config file (https://github.com/cudeso/misp2sentinel/blob/main/script.py#L181)

I will try shortly thanks!
On Sunday, August 13, 2023, Koen Van Impe @.***> wrote:

Can you check with the latest commit?
The Ignoring non STIX pattern type yara is normal. Microsoft Sentinel
does not accept yara indicators for uploading via Upload Indicators API,
these attributes are ignored, this causes the ERROR - Unable to process
indicator.
The Invalid value for Note 'object_refs': reference to custom object
type: x-misp-attribute is caused by the conversion by MISP to STIX (this
is done with the misp-stix package). I don't know why, but some
EventReports seem to be causing problems for this conversion, and it's more
a bug that needs to be raised in misp-stix. I added an option that removes
the EventReports from the event, before conversion takes place. Currently
Microsoft Sentinel does not support STIX Notes (to which reports get
translated) so you don't lose any functionality anyway. The change is in
https://github.com/cudeso/misp2sentinel/blob/main/RequestObject.py#L172
and the value "misp_remove_eventreports" is set to True by default when
it's missing from the config file (https://github.com/cudeso/
misp2sentinel/blob/main/script.py#L181)
—
Reply to this email directly, view it on GitHub
https://github.com/cudeso/misp2sentinel/issues/54#issuecomment-1676325639,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA6TXBFD6MA2YR44AQFSF3XVCZ6VANCNFSM6AAAAAA3OLOPUQ
.
You are receiving this because you authored the thread.Message ID:
@.***>

Thanks works great!

What would be the best way to print the contents of the event with the yara
rule that is rejected? Thanks
On Monday, August 14, 2023, Koen Van Impe @.***> wrote:

Closed #54 https://github.com/cudeso/misp2sentinel/issues/54 as
completed.
—
Reply to this email directly, view it on GitHub
https://github.com/cudeso/misp2sentinel/issues/54#event-10094140786, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA6TXBROMUZ4O6B3CDRKUTXVMI75ANCNFSM6AAAAAA3OLOPUQ
.
You are receiving this because you authored the thread.Message ID:
@.***>

It's not printing the content of the event; but this change logs the event ID. This should already point you in the right direction.
https://github.com/cudeso/misp2sentinel/commit/6f625de61ace076b4d8fafcfdd49dd74fcfc8d6a