ACS70331 equivalent/alternative

The ACS70331 is not available anywhere. Is there an alternative/equivalent to use

Some people reported that they successfully used the TMCS1101A4BQDT as a
drop in replacement, though don't have personal experience with it.
https://forum.duet3d.com/topic/23461/anybody-wants-a-stepper-motor-analyzer/180?_=1644177022197
On Sun, Feb 6, 2022 at 11:20 AM springsteenrob @.***>
wrote:

The ACS70331 is not available anywhere. Is there an alternative/equivalent
to use
—
Reply to this email directly, view it on GitHub
https://github.com/zapta/legacy_stepper_motor_analyzer/issues/12, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQVMQL4LOCAPJLBKBRSJB3UZ3CXHANCNFSM5NVWVGBQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Thank you for your quick response.
I have ordered the drop in replacement and I will let you know the outcome
Best Regards
Sent from my iPhone

On 6 Feb 2022, at 20:56, Zapta @.***> wrote:

Some people reported that they successfully used the TMCS1101A4BQDT as a
drop in replacement, though don't have personal experience with it.
https://forum.duet3d.com/topic/23461/anybody-wants-a-stepper-motor-analyzer/180?_=1644177022197
On Sun, Feb 6, 2022 at 11:20 AM springsteenrob @.***>
wrote:

The ACS70331 is not available anywhere. Is there an alternative/equivalent
to use
—
Reply to this email directly, view it on GitHub
https://github.com/zapta/legacy_stepper_motor_analyzer/issues/12, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQVMQL4LOCAPJLBKBRSJB3UZ3CXHANCNFSM5NVWVGBQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you authored the thread.

Great!  These are difficult times for sourcing parts.
Looking forward to seeing how it worked for you.
On Sun, Feb 6, 2022 at 12:34 PM springsteenrob @.***>
wrote:

Thank you for your quick response.
I have ordered the drop in replacement and I will let you know the outcome
Best Regards
Sent from my iPhone

On 6 Feb 2022, at 20:56, Zapta @.***> wrote:

Some people reported that they successfully used the TMCS1101A4BQDT as a
drop in replacement, though don't have personal experience with it.

https://forum.duet3d.com/topic/23461/anybody-wants-a-stepper-motor-analyzer/180?_=1644177022197

On Sun, Feb 6, 2022 at 11:20 AM springsteenrob @.***>
wrote:

The ACS70331 is not available anywhere. Is there an
alternative/equivalent
to use
—
Reply to this email directly, view it on GitHub
https://github.com/zapta/legacy_stepper_motor_analyzer/issues/12, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAQVMQL4LOCAPJLBKBRSJB3UZ3CXHANCNFSM5NVWVGBQ

.
Triage notifications on the go with GitHub Mobile for iOS
<
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android
<
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub
.
You are receiving this because you are subscribed to this
thread.Message
ID: @.***>

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you authored the thread.

—
Reply to this email directly, view it on GitHub
https://github.com/zapta/legacy_stepper_motor_analyzer/issues/12#issuecomment-1030909048,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQVMQLRFS7RDPJYUZAXMLTUZ3LPDANCNFSM5NVWVGBQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

Hello,
I promised you to come back on the outcome of the alternative part : TMCS1101A4BQDT
I have one analyzer with the ACS70331EOLCTR AND one with the  TMCS1101A4BQDT
The one with the  TMCS1101A4BQDT does not work very well for me:
It counts steps in the Home menu  when motor are not moving and for example the current by step/sec read out is not stable (current varies and with the ACS70331 not)
Also the graphs are less crispy and contain noisy lines. The lines with the  ACS70331 are sharper.
So for me it is not an good drop in replacement.
Best Regards,
Rob
Van: Zapta @.>
Verzonden: Monday, February 07, 2022 4:01 AM
Aan: zapta/legacy_stepper_motor_analyzer @.>
CC: springsteenrob @.>; Author @.>
Onderwerp: Re: [zapta/legacy_stepper_motor_analyzer] ACS70331 equivalent/alternative (Issue #12)
Great! These are difficult times for sourcing parts.
Looking forward to seeing how it worked for you.
On Sun, Feb 6, 2022 at 12:34 PM springsteenrob @.***<EMAIL_ADDRESS>>
wrote:

Thank you for your quick response.
I have ordered the drop in replacement and I will let you know the outcome
Best Regards
Sent from my iPhone

On 6 Feb 2022, at 20:56, Zapta @.***<EMAIL_ADDRESS>> wrote:

Some people reported that they successfully used the TMCS1101A4BQDT as a
drop in replacement, though don't have personal experience with it.

https://forum.duet3d.com/topic/23461/anybody-wants-a-stepper-motor-analyzer/180?_=1644177022197

On Sun, Feb 6, 2022 at 11:20 AM springsteenrob @.***<EMAIL_ADDRESS>>
wrote:

The ACS70331 is not available anywhere. Is there an
alternative/equivalent
to use
—
Reply to this email directly, view it on GitHub
https://github.com/zapta/legacy_stepper_motor_analyzer/issues/12, or
unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAQVMQL4LOCAPJLBKBRSJB3UZ3CXHANCNFSM5NVWVGBQ

.
Triage notifications on the go with GitHub Mobile for iOS
<
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 &mt=8&pt=524675

or Android
<
https://play.google.com/store/apps/details?id=com.github.android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub &referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub
.
You are receiving this because you are subscribed to this
thread.Message
ID: @.***<EMAIL_ADDRESS>>

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you authored the thread.

—
Reply to this email directly, view it on GitHub
https://github.com/zapta/legacy_stepper_motor_analyzer/issues/12#issuecomment-1030909048,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQVMQLRFS7RDPJYUZAXMLTUZ3LPDANCNFSM5NVWVGBQ
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 &mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub &referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you commented.Message ID:
@.***<EMAIL_ADDRESS>>

—
Reply to this email directly, view it on GitHub https://github.com/zapta/legacy_stepper_motor_analyzer/issues/12#issuecomment-1031023139 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AXUMWEPEC2ALIFJ6N5GEYH3UZ4YW7ANCNFSM5NVWVGBQ .
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675  or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .
You are receiving this because you authored the thread.  https://github.com/notifications/beacon/AXUMWEKDJITFHOVKZ7R6U63UZ4YW7A5CNFSM5NVWVGB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHV2CUIY.gif Message ID: @.*** @.***> >


AS with a few rules

Finding correlation among: AS with a few rules, their customer cone size, (and what, @cunha?).

I guess the first action point is to build a file with as-number number-of-rules customer-cone-size. With this you can compute the correlation between number of rules and customer cone sizes, and then I can look at a few ASes to try to characterize the behaviors of the ASes with less than 10 rules.

We already have a similar one from #120 (originally from #60): as_neighbors_vs_rules4.csv, but with #customer instead of customer cone size.

LACNIC does not contain any rules.
[2024-01-22T09:11:38Z DEBUG route_verification_irr] Read `../data/irrs/priority/lacnic.db`: 1847 aut_nums, 1349 as_sets, 0 route_sets, 0 peering_sets, 0 filter_sets, 1522 as_routes (12759 routes); 0 imports, 0 exports. Lexing: . Parsing: .

Latin America style! Such great IRRs. =D NIC.br also doesn't support any.
On Mon, Jan 22, 2024 at 8:29 PM Steven Hé (Sīchàng) <
@.***> wrote:

LACNIC does not contain any rules.
[2024-01-22T09:11:38Z DEBUG route_verification_irr] Read ../data/irrs/priority/lacnic.db: 1847 aut_nums, 1349 as_sets, 0 route_sets, 0 peering_sets, 0 filter_sets, 1522 as_routes (12759 routes); 0 imports, 0 exports. Lexing: . Parsing: .
—
Reply to this email directly, view it on GitHub
https://github.com/SichangHe/internet_route_verification/issues/122#issuecomment-1903904320,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACPO5ZY2GJ32KIGPPNVVX3YPZLQLAVCNFSM6AAAAABBUVW5OCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBTHEYDIMZSGA
.
You are receiving this because you were mentioned.Message ID:
@.***>


Enhance template with logging and bash checks

The current templates do not provide a central logging nor do they check mandatory parameter. Scripts must be enhanced to include basic checks and a central logging.
similar to the following code
# Define a bunch of bash option see
# https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html
set -o nounset          # stop script after 1st cmd failed
set -o errexit          # exit when 1st unset variable found
set -o pipefail         # pipefail exit after 1st piped commands failed

readonly LOGFILE="${LOG_BASE}/$(basename ${SCRIPT_NAME} .sh).log"
touch ${LOGFILE} 2>/dev/null
exec 1> >(tee -a "$LOGFILE")
exec 2>&1               # Redirect standard error to standard out
echo "INFO: Start to build MODS white papers using ${SCRIPT_NAME}"

add basic check for valid variables in v3.0.0


Env fields (broad and local)

Current term details
Term name - broad-scale environment field
Term ID - MIXS:0000012
Definition - In this field, report which major environmental system your sample or specimen came from. The systems identified should have a coarse spatial grain, to provide the general environmental context of where the sampling was done (e.g. were you in the desert or a rainforest?). We recommend using subclasses of ENVO’s biome class: http://purl.obolibrary.org/obo/ENVO_00000428. Format (one term): termLabel [termID], Format (multiple terms): termLabel [termID]|termLabel [termID]|termLabel [termID]. Example: Annotating a water sample from the photic zone in middle of the Atlantic Ocean, consider: oceanic epipelagic zone biome [ENVO:01000033]. Example: Annotating a sample from the Amazon rainforest consider: tropical moist broadleaf forest biome [ENVO:01000228]. If needed, request new terms on the ENVO tracker, identified here: http://www.obofoundry.org/ontology/envo.html

Suggested update(s)
Definition -

The definition needs to be updated to include the narrow synonym "biome" as well as a link to the ENVO documentation about how to use the field https://github.com/EnvironmentOntology/envo/wiki/ENVO-annotations-for-MIxS-v5
Current term details
Term name - local scale environment field
Term ID - MIXS:0000013
Definition - In this field, report the entity or entities which are in your sample or specimen’s local vicinity and which you believe have significant causal influences on your sample or specimen. Please use terms that are present in ENVO and which are of smaller spatial grain than your entry for env_broad_scale. Format (one term): termLabel [termID]; Format (multiple terms): termLabel [termID]|termLabel [termID]|termLabel [termID]. Example: Annotating a pooled sample taken from various vegetation layers in a forest consider: canopy [ENVO:00000047]|herb and fern layer [ENVO:01000337]|litter layer [ENVO:01000338]|understory [01000335]|shrub layer [ENVO:01000336]. If needed, request new terms on the ENVO tracker, identified here: http://www.obofoundry.org/ontology/envo.html

Suggested update(s)
Definition -

The definition needs to be updated to include the link to the ENVO documentation about how to use the field https://github.com/EnvironmentOntology/envo/wiki/ENVO-annotations-for-MIxS-v5, and to include mention of the possibility to use anotomical ontologies for environments relating to a host

The definition needs to be updated to include the narrow synonym "biome"

The key issue here is that "biome" is not a direct synonym of the broad scale environment. It is either a narrow or related synonym - if we can't express that, we should remove biome.

include mention of the possibility to use anotomical ontologies for environments relating to a host

Or other OBO ontologies that interoperate with ENVO. This prevents us from needing to unnecessarily inflate ENVO. We cover this in the wiki page noted below (this section)

as well as a link to the ENVO documentation about how to use the field https://github.com/EnvironmentOntology/envo/wiki/ENVO-annotations-for-MIxS-v5

We can create a new wiki page for MIxS v6.

@only1chunts

We can create a new wiki page for MIxS v6. This would greatly shorten the definition field.

We've created a generic "Using ENVO with MIxS" wiki page here, which we'll update whenever MIxS does. Please use this in all the relevant ENVO fields.

Could 'biome' be added back to EnvO ?
On Mon, Feb 8, 2021 at 11:47 AM Pier Luigi Buttigieg <
@.***> wrote:

The definition needs to be updated to include the narrow synonym "biome"
The key issue here is that "biome" is not a direct synonym of the broad
scale environment. It is either a narrow or related synonym - if we can't
express that, we should remove biome.
include mention of the possibility to use anotomical ontologies for
environments relating to a host
Or other OBO ontologies that interoperate with ENVO. This prevents us from
needing to unnecessarily inflate ENVO. We cover this in the wiki page noted
below (this section
https://github.com/EnvironmentOntology/envo/wiki/ENVO-annotations-for-MIxS-v5#notes-on-the-use-of-other-ontologies-in-mixs-environment-fields
)
as well as a link to the ENVO documentation about how to use the field
https://github.com/EnvironmentOntology/envo/wiki/ENVO-annotations-for-MIxS-v5
We can create a new wiki page for MIxS v6.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/GenomicsStandardsConsortium/mixs/issues/106#issuecomment-775282222,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABBB4DMPMU5VSKL37FEZ7NTS6AISHANCNFSM4XJKEQAQ
.

--
Lynn M. Schriml, Ph.D.
Associate Professor
Institute for Genome Sciences
University of Maryland School of Medicine
Department of Epidemiology and Public Health
670 W. Baltimore St., HSFIII, Room 3061
Baltimore, MD 21201
P:<PHONE_NUMBER> |  F:<PHONE_NUMBER>
@.***


Decompile to C

As an additional layer on top of HLIL we could translate directly to C.

For those that prefer a more C like syntax for decompilation, a new option has been added to surround scopes in HLIL with { and }. This does not affect the syntax of the individual HLIL instructions in any way. Added in build 2651.
A full C syntax renderer will be added at a later time.

I definitely like the added option regarding the brackets and would like to see a few a few additional similar options be added in time as well.  I don't understand the benefit of building a C layer on top of the HLIL though unless it's just for visualization purposes, could someone expand on the benefits it would offer?

There are some I would consider minor benefits:

Ease of recompilation - people frequently want to recompile the code they see, right now thats difficult.
Ease of understanding  - Even though C syntax is arguably worse to read its something many people are familiar with. Some one might not understand that uint32_t var_8 = rax_1.d in HLIL is the same as uint32_t var_8 = (uint32_t)rax_1;

Why would we consider this? There are a few reasons.

Marketing - Some people complain that Binary Ninja doesn't have a "real" decompiler. (I'd argue we have something better)
Its easy - I'm pretty sure if Rusty started this task it would take him about a week to get done.

Good point but I since it's a renderer then couldn't that be let handled by the community?
Another fair point however what if I wanted to have a kotlin/native renderer instead of a C renderer. Would you guys consider writing an exposed rendering api so that people who are more familiar with particular languages could write a rendering object that is more familiar to them?
I assumed marketing had a big part to do with it, being able to show more than just C style renderings of the HLIL would be beneficial for those with more experience reading, say golang or some such.
I'm sure he'd write a nice community API and it would get rid of a lot of complaints in the future as well from people requesting other languages be used to represent the HLIL.

Yes it could be left to the community, but I'd rather have it built into the product. Though for marketing purposes it doesn't have the same impact as first party support.
The rendering API is already all exposed. Its honestly really easy to add another view. Take a look at the TokenizedTextView
Yes we intend to be able to render HLIL as multiple different High level languages some of those could take significant effort though.
There is already a plugin which could be used as a starting point, you just have to iterate HLIL rather than MLIL linear_mlil.py

Understood, I hope you guys continue to invest time in the current renderer since it's finally grown on me some. I was more thinking you would write it and provide first party support but open source it.
Oh wow, I'm so used to seeing the term linear disassembly in the UI that I didn't piece it together.
I personally don't have any request, I was more just trying to back up the reasonings I think a public API would be good. I actually think developing even more in house would take away the time that would otherwise be spent polishing other aspects. I'm sure your team discussed it and decided it was worth it though.
I'll take a look shortly, thanks!

I will say in general we do love hearing feedback like this. Sometimes it's hard to know what will or won't be used and while I think we'll probably still pursue this based on it being one of the more common "papercuts" for some people, hearing about features you /don't/ want (in exchange for others with time for development being a finite resource!) is very helpful, thanks. :-)

Instructions that still need support:

[ ] HLIL_DEREF_FIELD
[ ] HLIL_EXTERN_PTR
[ ] HLIL_ADC
[ ] HLIL_SBB
[ ] HLIL_ROL
[ ] HLIL_RLC
[ ] HLIL_ROR
[ ] HLIL_RRC
[ ] HLIL_NEG
[ ] HLIL_NOT
[ ] HLIL_LOW_PART
[ ] HLIL_TEST_BIT
[ ] HLIL_BOOL_TO_INT
[ ] HLIL_ADD_OVERFLOW
[ ] HLIL_SYSCALL
[ ] HLIL_BP
[ ] HLIL_TRAP
[ ] HLIL_UNIMPL_MEM
[ ] HLIL_FSQRT
[ ] HLIL_FNEG
[ ] HLIL_FABS
[ ] HLIL_INT_TO_FLOAT
[ ] HLIL_ROUND_TO_INT
[ ] HLIL_FLOOR
[ ] HLIL_CEIL
[ ] HLIL_FTRUNC
[ ] HLIL_FCMP_O
[ ] HLIL_FCMP_UO

Added in 2.5.3174


add shorter flag for quiet and debug

cli: add short versions of quiet and debug flag #811

@michiboo Nice work ! Thanks a lot.


Mobile disable ruins the layout

Hello, I'm using  disable: 'mobile' option to disable aos on mobile devices for my website, so it disables everything, but all elements that were fading in/sliding in etc now are completely invisible in the layout. They stay at their "pre-animation" positions as if they were animated, instead of being present where they have to be with aos off. Is there any way to disable mobile aos use without ruining the layout for all elements?
Thank you
Quick update: Problem solved by initiating AFTER the layout, not before it. Thanks.

By "after the layout", do you mean with script tag at the end of the body? I'm not getting the disable to work at all, either with 'mobile' or window.innerWidth < 1000... It's not however, disappearing as the OP was troubling with, but simply keeps animating regardless of vp width.


update point #8 on README for Windows users

update the README for Windows users

Hey @DevipriyaSarkar , I don't see a need for this if it is the same for Windows command prompt as in UNIX.

On Windows command prompt, we need to mention the .py part, that is, web2py.py instead of web2py (as in UNIX).
Otherwise it shows
python: can't open file 'web2py': [Errno 2] No such file or directory
The .py part is what I meant to point out.

Oh I see! That's a typo change it to web2py.py, in UNIX also it is supposed to be web2py.py. So no need for a separate Windows part now.

@raj454raj updated :)


Cannot read property 'charCodeAt' of undefined

Issue
when importing a package const redis = require("redis"); in the console, an error appears: TypeError: Cannot read property 'charCodeAt' of undefined
Environment

Node.js Version: 14.5.0
Redis Version: 6.0.5

Has anyone encountered such a problem? there is a solution?

I've encountered the same problem. Can somebody help me?
I am using VueJS and trying to import package by const redis = require("redis");

I've encountered the same problem. Can somebody help me?
I am using VueJS and trying to import package by const redis = require("redis");
Edit: Oh, it cannot be used in the browser... i get it now.

this package only works on the server.
it will not work on the client, so it cannot be built through cli


Detail trust levels

Info on what the different types of trust are, how they can be changed, how they are represented, etc.

Internally as an int between 0-inf
Higher is better (more trust)
Levels
Admin 9999
Verified 999
User 99
Registered 9
Unregistered 1
(banned 0) ?
Only admins may change trust level

Explicit, automatic trust changes:

Unregistered ⇒ Registered
User registers on website and can log in.
Registered ⇒ User
Registered users must prove they own their email used during registration by clicking on the verify link inside a registration email sent during signup

Admin can set trust level of any other user (including other admins).

 #23 Detail trust levels


fix match_100_set_linkbw_2G policy as per expected result in README on cisco

Pull Request Functional Test Report for #3331 / 4338df4c76a38918c94eb835ee1103ff26a2b32e
Virtual Devices

Device
Test
Test Documentation
Job
Raw Log

Arista cEOS

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community
3e0005ba
Log

Cisco 8000E

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community
3fac600e
Log

Cisco XRd

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community
ad76627e
Log

Juniper ncPTX

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community
ccf7f67f
Log

Nokia SR Linux

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community
94eabe97
Log

Openconfig Lemming

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community
1a1702a1
Log

Hardware Devices

Device
Test
Test Documentation
Raw Log

Arista 7808

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community

Cisco 8808

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community

Juniper PTX10008

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community

Nokia 7250 IXR-10e

RT-7.5: BGP Policy - Match and Set Link Bandwidth Community

Help

Pull Request Test Coverage Report for Build<PHONE_NUMBER>7
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 55.5%

Totals

Change from base Build<PHONE_NUMBER>4:
0.0%

Covered Lines:
1983

Relevant Lines:
3573

💛  - Coveralls


Content-type of json responses is not correct

Requests which specify ?format=j1 are returned as JSON in the body but with a content-type header of text/plain (here).
This should be application/json. Many services which consume APIs will not attempt to parse responses unless the content-type indicates that the payload is JSON.

It works now:
$ curl -I wttr.in/London?format=v1
HTTP/1.1 200 OK
Server: nginx/1.10.3
Date: Wed, 14 Oct 2020 05:21:34 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 2
Connection: keep-alive
Access-Control-Allow-Origin: *

$ curl -I wttr.in/London?format=j1
HTTP/1.1 200 OK
Server: nginx/1.10.3
Date: Wed, 14 Oct 2020 05:21:38 GMT
Content-Type: application/json
Content-Length: 45452
Connection: keep-alive
Access-Control-Allow-Origin: *

Thank you for reporting!


Updated and improved README.md

Corrected grammatical and spelling mistakes and also improved language.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Closing PR as this is not formatted, and has adds just as many errors as it fixes.


POST /containers/(id)/stats broken in 1.7?

Has the spec for this endpoint been changed in docker 1.7? Using the latest version of https://github.com/fsouza/go-dockerclient against https://master.dockerproject.com/ results in Unrecognized input header being returned when calling client.Stats(). Works correctly in 1.6. I can dig deeper if this isn't already a known issue.

Hi!
If you are reporting a new issue, PLEASE make sure that does not have any duplicates already open.
We would like to take this time to remind you of the information we need to debug the problem you are seeing. This is an automated response so if this ticket is not about a bug, do not fret.
If you fail to provide this information within 7 days, we will close this because we cannot debug your issue. We can reopen whenever the information is provided.
Thank you.
Please see:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#reporting-other-issues
Description of problem:

`docker version`:

`docker info`:

`uname -a`:

Environment details (AWS, VirtualBox, physical, etc.):

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual Results:

Expected Results:

Additional info:

#ENEEDMOREINFO

I'm not aware of anything changed in this, apart from adding an optional parameter to disable streaming.
Could you add the information requested above?
/cc @cpuguy83 think you made the last changes to the stats?

Nothing other than a new param to disable streaming.

That's what I thought, thanks
@ejholmes if you could add the requested information and a way to reproduce this, that would be very welcome!

@thaJeztah cool! I'll try to get a repro case for this. Thanks!

@thaJeztah @cpuguy83 updated the description with info and steps to reproduce. Seems to be 100% reproducible with docker 1.7-dev; I get one set of stats sent to me, then it just stops streaming.

Ah! Browsing the source for the container stats endpoint on master and noticed the new stream param. Works as expected once that param is provided, so I'll try to get go-dockerclient to add this by default, since it shouldn't have any effect against docker 1.6.
Sorry for the trouble!

@ejholmes thanks for researching this! Actually, I am going to re-open; the stream parameter should be optional, not a required parameter, see the description;

stream – 1/True/true or 0/False/false, pull stats once then disconnect. Default true

@cpuguy83 This looks like a bug/regression to me, I haven't tested yet, but this doesn't sound right

Yeah, I bet that happened with engine removal.

@cpuguy83 will you look at it? Or @ejholmes are you interested in creating a PR to fix this (and a regression test)?

#10766 introduced the stream value check right?

@runcom yup

@ejholmes we've discussed this briefly on IRC and think it's important to have this fixed ASAP.
@cpuguy83 or @runcom will pick this up so that it'll be fixed soon.
Thanks for reporting!! And if you feel like contributing for other issues, you're more than welcome!

Should probably have "boolValue" accept a default value.

@cpuguy83 yes I did a branch but without a default value for now because I talked with @tiborvass on IRC and I'll come up with another PR changing how the stats endpoing behave (like having it no stream by default and adding stream=1 will stream or having nofollow query param that do the opposite because stats is the only endpoint to have a query param that is default'ed to true)

@runcom It does make sense, but I'm hesitant on changing that so short before code/feature freeze. There might be quite some projects relying on the current behavior.


Prevent /boot from filling up by disabling unattended upgrades

Simply disables the security branch of unattended upgrades (the only one enabled by default).
Closes #24

Looks good, I will run a build with it just to be sure, though.

cool, I did test it here as well.

backport is in #214

Verified that re-running the play does not alter the file if the line is already commented, so I'm +1.

Verified, +1. Merging.


incorrect double negation in template

https://github.com/oas-tools/oas-cli/blob/7d9e6b201a58ad52f4529f1a2e90b219522e2839/templates/server/index.js.hbs#L61
the line should display the output if disable !== true in stead of false

That's true, I'm adding this issue to the project for the next patch, thanks :smile:

Fixed at #14


Nydus support for bypassing guest cache with DAX

Nydus support for bypassing guest cache with DAX
tldr: We would like to get nydus with DAX working to bypass the guest page cache and therefore deduplicate the memory that is being used by a machine, overall allowing us to spawn more workloads and use memory more efficiently.
I have been debugging an issue over the last week which has been plaguing our nodes for a few months. Workloads run with Kata report memory usage within the VM of the running workload as much less than compared to what the host machine reports for the workloads from outside of the VM.
From my debugging and investigation I think the main cause of the issue comes from the page cache of the guest operating system. The guest OS’s page cache differs from the host's because the guest’s can only be reclaimed by workloads running within the context of the guest VM, whereas the host’s can be reclaimed at any point when needed. This looks to be because the host machine is unable to differentiate what a guest VM is assigning the memory to, it has no understanding of how the guest OS is allocating the memory pages it requests from the host and thus is unable to free them back to the host when in a low memory condition.
I can prove this by running a write heavy workload in Kata and watching how the memory is allocated in both the guest and the host, monitoring what these allocations correspond to. In the particular case below I was running dd if=/dev/zero of=/file.dat bs=512 count=12582912 to write a 6GB file to the root filesystem of the container, backed by nydus (virtiofs). The container was also running inside a Cloud Hypervisor VM provisioned by Kata containers.

The above graph shows the current shared memory usage of host machine as reported in /proc/meminfo, this relates to the memory allocated to cloud hypervisor for the VM’s RAM. Over the course of the write, the memory can be seen slowly increasing, which corresponds to the growth of page cache used within the guest:

This graph also illustrates the above memory usage of the guest VM in terms of process RSS gathered from /proc/<pid>/status

The two graphs above show how the host’s page cache is affected by the memory now in use by the VM. In the first graph we can see that the total page cache of the host machine grows in accordance to the workload within the VM writing to a file shared by virtiofs. When compared to the previous graphs detailing the guest’s memory usage, the host cache stops growing at around the 18:33 mark. The reason for this is illustrated in the bottom graph, the total memory usage of the node is essentially “full”, after 18:33 the host kernel starts to evict some of the host page cache to allow for the growing allocations required by the guest VM, in accordance to the guest’s growing page cache.
From the above setup we can see that the filesystem is being cached in two places, first in the host page cache and then again in the guest’s page cache. Due to the differences in how these allocations are perceived by the host, the guest’s page cache is essentially wasting memory because it can no longer be reclaimed by the host when needed.
This soon starts to add up when running 100s of Kata based pods, which is often amplified when running read/write intensive workloads like databases. This wastes memory that could otherwise be used for starting new workloads, and in some cases may also lead to a node becoming unresponsive due to a low memory condition. We often see quite a large difference in the memory usage of the host node when compared to the sum of the memory usage of the workloads within the VMs.
In order to combat this problem I was looking at the various caching options available to virtiofs/nydus. First we tried the CachePolicy=Never option by overriding settings passed to the passthrough FS setup in the nydus code. This doesnt have any affect on the guest cache, which we discovered after running some tests and reading more into what the flag controls. Since then I have been having a look at DAX, which is supposedly the solution to the problem of double caching.
According to this reply in an issue on the Kata Container’s github (https://github.com/kata-containers/kata-containers/pull/4648#issuecomment-1183861849) nydus should support dax. From my testing this doesnt seem to be the case, first off running Cloud Hypervisor on version 22 (later versions removed the dax option), the VM fails to boot and crashes nydus. I could see from the trace logs from nydus that a setup mapping request is received, but would be replied to with EINVAL causing the mapping request to fail. I managed to fix this by adjusting the protocol features set by nydus when initiating the vhost_user connection to the VMM. I added
VhostUserProtocolFeatures::SLAVE_SEND_FD

to https://github.com/dragonflyoss/image-service/blob/master/src/bin/nydusd/virtiofs.rs#L159
This matches the required features for Cloud Hypervisor to start the vhost_user manager instance to deal with setup mapping requests.
(https://github.com/cloud-hypervisor/cloud-hypervisor/blob/v22.0/virtio-devices/src/vhost_user/fs.rs#L400 where the protocol features are checked, controlling the creation of the master request handler)
(https://github.com/cloud-hypervisor/cloud-hypervisor/blob/v22.0/virtio-devices/src/vhost_user/fs.rs#L516 where the setup mapping handler is created later on)
(https://github.com/cloud-hypervisor/cloud-hypervisor/blob/v22.0/virtio-devices/src/vhost_user/fs.rs#L74 the vhost master handler for the mapping/unmapping requests)
Even though this change allows the workload to start running, and the virtiofs filesystem is mounted with the dax option. Most workloads I tested with this enabled, ended up failing at some point during its run time, for some reason or another. I have not had a chance yet to debug what might be causing the failures.
We would appreciate some thoughts on the DAX situation and whether or not nydus can support it and if there are any upsides or downsides to running virtiofs in DAX mode.
So far we have been impressed with the performance boost nydus has enabled and if we can get DAX support working as expected I think we will be very happy overall.

@Champ-Goblem Thanks for raising it! There are two host access modes provided by nydusd to Kata: rafs and passthroughfs. rafs is a container image acceleration file system while passthroughfs is an equivalance to virtiofsd. May I ask which part you want to enable DAX with? The passthroughfs should be easy to enable DAX support, while rafs needs more changes but it's do-able as well.

Actually, we (Alibaba Cloud) have a new pending academic paper to evaluate virtiofs dax vs Nydus blob passthough(blobfs) + erofs dax.  The performance number shown that the workload with 50000 small files, raw virtiofs dax (30 sec) is much much slower than Nydus blob passthough + erofs dax (6s). Also Nydus blob pass though + erofs dax is already in production at Alibaba Cloud and it has great performance and memory saving.  However, it still needs some step to upstream it to the mainline kernel.  Another way to achieve this is to use on-demand Virtio-pmem + erofs dax, it has almost native performance as well.

Thanks for the above answers!
@bergwolf Looks like we are currently using PassthroughFs as seen when checking the daemon info of a running nydusd instance:
"backend_collection":{"/containers":{"backend_type":"PassthroughFs","mountpoint":"/containers","mounted_time":"2022-10-23 00:22:09.405991151 +01:00","config":null}}}

@hsiangkao these options sound interesting, I was wondering if the Nydus blob pass though + erofs dax has a limit on the container image type? Are the performance enhancements this would bring limited to rafs formatted images, or will it also work with the standard OCI spec images as well without conversion?
To add to the above options, we would ideally like to have a solution that covers mounting persistent volumes, as these mounts can contribute to cache usage for stateful workloads. Plus any performance improvements we could bring to persistent volumes would also be beneficial.

@hsiangkao these options sound interesting, I was wondering if the Nydus blob pass though + erofs dax has a limit on the container image type? Are the performance enhancements this would bring limited to rafs formatted images, or will it also work with the standard OCI spec images as well without conversion?

Well, OCI images (targz) also need to be extracted to some localfs (like ext4), likewise, we could also make a light conversion to erofs type for guest or runC to use, that is what we're doing.

@Champ-Goblem If image conversion is a non-starter, the passthrough mode is the way to go. For DAX support with the passthrough mode, we have it working with the builtin virtiofs with dragonball. So in theroy it is just about enabling the same feature with nydusd where the vhost-user protocol is the only addition. I would suggest that we test it with QEMU first to make sure that there aren't virtio spec intepretion ceavats and then switch the vmm to cloud-hypervisor.
As for the downside of running DAX, as @hsiangkao mentioned, small files may have a performance penaty due to the extra setupmapping calls. It is possible to mitigate it a bit by enabling the per-file DAX feature so that only large files are shared via DAX.

@hsiangkao converting OCI spec images to work with EROFS sounds interesting. Regarding the comment that is what we're doing in your previous answer, do you mean nydus already has this capability?
@bergwolf I think passthroughFS would be a good starting point for now as this would also support persistent volumes as well without having to alter lots in Kata and would also work with regular OCI images.
By the way I tested Kata 3.1 this morning with dragonball + DAX, which showed the guest page cache bypass was working as expected. Based on this, what is required to get DAX in nydus for passthroughFS implemented?
As a side note which is probably out of scope for this issue (happy to make a ticket in the relevant place if applicable):
We have a set of filesystem performance tests that we often use to test different settings and versions of VMMs and Kata, which is comprised of 3 fio commands and 3 different container image builds. When we ran the benchmarks in dragonball this morning, we noticed that build performance/stability seems to be degraded when compared to CLH/Qemu, 2/3 of the builds failed due to socket timeouts which I dont think was related to the backend server, the only build that succeeded took over 18 minutes vs 1.5 mins for CLH or Qemu.

@hsiangkao converting OCI spec images to work with EROFS sounds interesting. Regarding the comment that is what we're doing in your previous answer, do you mean nydus already has this capability?

I haven't followed Nydus userspace code anymore for months, hopefully @zyfjeff @liubogithub @jiangliu could give more hints about this.

@Champ-Goblem

By the way I tested Kata 3.1 this morning with dragonball + DAX, which showed the guest page cache bypass was working as expected. Based on this, what is required to get DAX in nydus for passthroughFS implemented?

The only addition is the vhost-user-fs protocol. I would suggest upgrading the vhost crate in nydus first to include the vhost-user-backend there (instead of the old stand-alone one) to have the latest working code and see what we can get there.

We have a set of filesystem performance tests that we often use to test different settings and versions of VMMs and Kata, which is comprised of 3 fio commands and 3 different container image builds. When we ran the benchmarks in dragonball this morning, we noticed that build performance/stability seems to be degraded when compared to CLH/Qemu, 2/3 of the builds failed due to socket timeouts which I dont think was related to the backend server, the only build that succeeded took over 18 minutes vs 1.5 mins for CLH or Qemu.

Any chance you can share the fio commands and/or the container images? It makes sense to create an issue in kata-containers repo to track it down there. We certainly don't expect to see such performance/stability issues with dragonball.

Hi @Champ-Goblem please help to share some simple instructions to recreate the test so that we could follow up with the issue

I’ve been doing some more digging into DAX and nydus over the last day and I have a couple of questions regarding a potential problem I found that I hope you might be able to share some insight on.
So currently the status with DAX + Qemu is that the vhost crate doesnt have support for the newer VhostUserFSSlaveMsg schema, so I decided to continue with looking at CLH and DAX as CLH should be using the same crate version and thus the same schema.
When running nydus with DAX in CLH it is very close to working, currently it boots and allows the virtiofs filesystem to be mounted with DAX. Simple read and writes to the filesystem work as expected but it fails when doing more complicated tasks such as git clones.
I managed to pinpoint the problem to a virtio message that nydus receives:
[2022-10-26 16:11:38.236757 +01:00] INFO [/home/administrator/Work/fuse-backend-rs/src/transport/virtiofs/mod.rs:86] fuse: from_descriptor_chain len 36 addr<PHONE_NUMBER>96 flags 1
[2022-10-26 16:11:38.236805 +01:00] INFO [/home/administrator/Work/fuse-backend-rs/src/transport/virtiofs/mod.rs:88] fuse: from_descriptor_chain last addr<PHONE_NUMBER> num regions 1
[2022-10-26 12:52:17.023344 +01:00] ERROR [error/src/error.rs:21] Error:
	InvalidDescriptorChain(FindMemoryRegion)
	at src/bin/nydusd/daemon.rs:140
	note: enable `RUST_BACKTRACE=1` env to display a backtrace

from_descriptor_chain len 36 addr<PHONE_NUMBER>96 flags 1 this line in the above output details the current descriptor in the virtio descriptor chain being parsed. The address listed in the above message translates to hex 0x7f9c800000 which incidentally corresponds to the address of the region allocated for the dax cache:
"_virtio-pci-_fs1": {
      "id": "_virtio-pci-_fs1",
      "resources": [
        {
          "MmioAddressRange": {
            "base":<PHONE_NUMBER>20,
            "size": 524288
          }
        },
        {
          "MmioAddressRange": {
            "base":<PHONE_NUMBER>96, <--- cache
            "size":<PHONE_NUMBER>
          }
        }
      ],
      "parent": null,
      "children": [
        "_fs1"
      ],
      "pci_bdf": "0000:00:04.0"
    }

It points more specifically at the only file that has been mapped into this region so far for reading/writing.
Is this an expected entry in the descriptor table? I thought that the descriptor table should only point to entries in the virtio ring and that the address referred to by the descriptor should point to some fuse/virtiofs message rather than a base address of a newly mmaped file?
My other question is, which part of the stack controls writing into the ring created by virtiofs, is it the virtiofs kernel module in the guest or the actual VMM that writes into the virtio queues?
For reference the error message is thrown here in the nydus code https://github.com/dragonflyoss/image-service/blob/v2.1.0-rc.3.1/src/bin/nydusd/virtiofs.rs#L78
Any help would be appreciated in getting to the bottom of this.

@Champ-Goblem By looking at the  https://github.com/dragonflyoss/image-service/blob/v2.1.0-rc.3.1/src/bin/nydusd/virtiofs.rs#L78, I think the workload such as git clone is using mmap and dio, which causes the above panic.
fd1 = open("/mnt/virtiofs/foo");
fd2 = open("/mnt/virtiofs/bar");

addr = mmap(fd1...);
ret = write(fd2, addr...);

With dax enabled, the above addr refers to some address of virtiofs dax range, not a guest memory address.
For now nydus, as a vhost-user-fs backend, only shares the guest memory, and it doesn't know dax memory yet.
But it should be not difficult to fix as clh has gained that VHOST_USER_SLAVE_FS_IO capability.
I think dragonball and its inline-virtiofs(a virtio device mode instead of a vhost-user device mode) has fixed the above issue.

@liubogithub Thank you for the detailed message!
Last week I spent some time POCing VHOST_USER_SLAVE_FS_IO in the fuse-backend-rs/vhost crates and managed to get this somewhat working.
I would be interested to know if you had any ideas on how to debug the following problem. Since implementing the changes, I found that Cloud Hypervisor when mapping a file into the DAX region would sometimes pass and sometimes fail. When this fails it causes the KVM_RUN ioctl to fail with -EFAULT.
The log for the failed KVM_RUN:
cloud-hypervisor: 13.739285612s: <_fs1> INFO:virtio-devices/src/vhost_user/fs.rs:82 -- fs_slave_map
cloud-hypervisor: 13.739303236s: <_fs1> INFO:virtio-devices/src/vhost_user/fs.rs:65 -- fs: is_req_valid
cloud-hypervisor: 13.739310276s: <_fs1> INFO:virtio-devices/src/vhost_user/fs.rs:99 -- fs_slave_map: addr<PHONE_NUMBER>08640 flags MAP_R len 2097152 fd_offset 0 cache_offset 0 mmap_cache_addr<PHONE_NUMBER>08640
cloud-hypervisor: 13.739333094s: <vcpu0> ERROR:vmm/src/cpu.rs:979 -- VCPU generated error: VcpuRun(Failed to run vcpu: VCPU error regs Ok(kvm_regs { rax:<PHONE_NUMBER>98302, rbx:<PHONE_NUMBER>0576527944, rcx: 3454, rdx: 3454, rsi:<PHONE_NUMBER>0156513280, rdi:<PHONE_NUMBER>94848, rsp:<PHONE_NUMBER>0576527304, rbp: 0, r8:<PHONE_NUMBER>0576527960, r9: 0, r10: 3454, r11: 1024, r12: 0, r13: 3454, r14:<PHONE_NUMBER>0576527944, r15: 0, rip:<PHONE_NUMBER>1586207022, rflags: 328194 }) debug_regs Ok(kvm_debugregs { db: [0, 0, 0, 0], dr6:<PHONE_NUMBER>, dr7: 1024, flags: 0, reserved: [0, 0, 0, 0, 0, 0, 0, 0, 0] }) events Ok(kvm_vcpu_events { exception: kvm_vcpu_events__bindgen_ty_1 { injected: 0, nr: 13, has_error_code: 1, pending: 0, error_code: 0 }, interrupt: kvm_vcpu_events__bindgen_ty_2 { injected: 0, nr: 0, soft: 0, shadow: 0 }, nmi: kvm_vcpu_events__bindgen_ty_3 { injected: 0, pending: 0, masked: 0, pad: 0 }, sipi_vector: 0, flags: 13, smi: kvm_vcpu_events__bindgen_ty_4 { smm: 0, pending: 0, smm_inside_nmi: 0, latched_init: 0 }, reserved: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], exception_has_payload: 0, exception_payload: 0 }) err Error(14)

I spent some time tracing this with BPF to try and identify where the error was being generated from, and it seems that the EFAULT error is returned during direct_page_fault -> handle_abnormal_pfn -> kvm_handle_bad_page. The kvm_handle_bad_page is called when the pfn of the page fault is masked to an error, but the reason for it being masked I was unable to figure out.
I have attempted to compare the CLH code versus the Dragonball code but they both seem pretty similar when setting up the DAX window, so I'm not too sure what could be causing this error. I understand that this is not the easiest to explain in detail over a message, but I would appreciated any thoughts you may have on figuring out possible situations that could cause an error like this to be generated in the current context.


Requesting certificates using enrollment web services. (Issue  #49)

Added functionality for requesting certificates by using Enrollment Web Service - CES and Enrollment Policy Web Service - CEP allowing external servers to request certificates.

MSFT_xCertReq.psm1

Added parameter to specify CA type e.g. Standalone or Enterprise (defaults to the latter).
Certificate Template will only be added to the .inf file if the default CA type is specified.
Added parameters for CES and CEP URLs.
certreq  -new and -submit can now use Credentials and URLs to CEP and CES (if specified) to request certificates.

MSFT_xCertReq.schema.mof

Added entries for CA type, CES and CEP

Because of the change to creating the .inf file mentioned above some tests are failing. I'm quite new to testing so I would appreciate some feedback on how to change the tests to fit the new functionality, also on how to test the new functionality :slightly_smiling_face:

This change is

@gammalost,
Thanks for your contribution.
To ensure that the project team has proper rights to use your work, please complete the Contribution License Agreement at https://cla.microsoft.com.
It will cover your contributions to all Microsoft-managed open source projects.
Thanks,
Microsoft Pull Request Bot

@gammalost, thanks for signing the contribution license agreement. We will now validate the agreement and then the pull request.
Thanks, Microsoft Pull Request Bot

Hi @gammalost - great job on submitting this and thank you! It looks like the unit tests are breaking though. Are you able to take a look? I'll review once the tests are working.

Oh. And also the merge conflicts :grin:

Hi @PlagueHO! I've tested a bit on why the unit tests fails and have concluded that it is related to the inf file generation, but I can't for the life of me understand why they are failing as it should generate the same output.
This is how the removal of the cert template parameter in the inf file is done now
xCertReq.psm1
# Create INF file
    $requestDetails = @"
[NewRequest]
Subject = $Subject
KeySpec = $keySpec
KeyLength = $KeyLength
Exportable = $($Exportable.ToString().ToUpper())
MachineKeySet = $MachineKeySet
SMIME = $smime
PrivateKeyArchive = $privateKeyArchive
UserProtected = $userProtected
UseExistingKeySet = $useExistingKeySet
ProviderName = $ProviderName
ProviderType = $providerType
RequestType = $requestType
KeyUsage = $KeyUsage
"@
    # If an enterprise CA is used a certificate template must be provided.
    if ($CAType -eq 'Enterprise')
    {
        $requestDetails += @"

[RequestAttributes]
CertificateTemplate = $CertificateTemplate
"@
    }

    $requestDetails += @"

[EnhancedKeyUsageExtension]
OID = $OID
"@

The here-string used in the unit test should AFAIK be the same as long as nothing else than enterprise is sent as a parameter
xCertReq.Tests.ps1
        $CertInf = @"
[NewRequest]
Subject = "CN=$validSubject"
KeySpec = 1
KeyLength = $KeyLength
Exportable = $($Exportable.ToString().ToUpper())
MachineKeySet = TRUE
SMIME = FALSE
PrivateKeyArchive = FALSE
UserProtected = FALSE
UseExistingKeySet = FALSE
ProviderName = $ProviderName
ProviderType = 12
RequestType = CMC
KeyUsage = $KeyUsage
[RequestAttributes]
CertificateTemplate = $CertificateTemplate
[EnhancedKeyUsageExtension]
OID = $OID
"@

I'm fairly new to testing in general so would love some pointers, both to this problem and possibly how to build relevant unit tests for the new functionality :slightly_smiling_face:

No problem @gammalost - testing can be one of the trickiest parts of any change. Once the other two outstanding PRs are through I'll take a look at this one as those will cause your one some more merge conflicts. Sorry about taking so long on this!

@gammalost - Ok, I had a look at the tests to see why they're failing. The problem is that there is a test (10 to be precise) that checks that the Set-Content cmdlet on this line is saving the correct information.
But because you've changed the information that is being saved here, you'll also need to change the tests to reflect this. The test can be found starting on this line:
https://github.com/PowerShell/xCertificate/blob/dev/Tests/Unit/MSFT_xCertReq.Tests.ps1#L291
But, the code you'll actually need to change to make the test pass is here:
https://github.com/PowerShell/xCertificate/blob/dev/Tests/Unit/MSFT_xCertReq.Tests.ps1#L225
You need to make this string match what you'd expect the real code to generate. This may also mean that you'll actually want to adjust the parameters being passed to Set-TargetResource in the test (here: https://github.com/PowerShell/xCertificate/blob/dev/Tests/Unit/MSFT_xCertReq.Tests.ps1#L302) to ensure that the full INF content is being generated. The goal is to ensure we're testing as much of our code as possible.
Does all that make sense?

Anyway, once the tests pass, I'll do a proper review on it. I think there is a couple of things in there I've noticed that we might be able to optimize (to reduce shared code or ensure we're not implementing a breaking change - as that requires a whole lot of additional things to be done to release).

It makes sense. What i find peculiar is that the INF content in the test and in the resource should be the same. The new parameter I've added that defaults to Enterprise should result in the necessary content regarding certification template to be added in the INF, and should therefore not break the tests(?).
    if ($CAType -eq 'Enterprise')
    {
        $requestDetails += @"

[RequestAttributes]
CertificateTemplate = $CertificateTemplate
"@
    }

I think what the problem could be is the CRLFs. It looks like you've added some CRLFs between the sections (above the [RequestAttributes]) which will cause the string in the unit test to not match. So you can either remove the CRLF from above the sections or add them to the comparison string in the tests.

You sir are correct. Did not even think about that, I kind of shamelessly copied how it was done to the rest of the parameters that were added to the file (newbie errors). I rewrote how it is removed to instead leave the original creation of $requestDetails as is, and instead replace portion of the here-string if something other than enterprise is passed.
    if ($CAType -ne 'Enterprise')
    {
        $requestDetails = $requestDetails.Replace(@"
[RequestAttributes]
CertificateTemplate = $CertificateTemplate
[EnhancedKeyUsageExtension]
"@, '[EnhancedKeyUsageExtension]')
    }

Regarding test-scenarios for this code; as it is reliant on credentials and the URL for both CEP and CES would the following scenario make sense?

CEP and CES URLs passed, credentials not passed

As templates will be removed from the request details when standalone is used i reckon a test for the new request details will be required as well?

@gammalost - haha I think I had the exact same trouble when I wrote the tests in the first place - so you're not alone there :grin:
I definitely think it would be beneficial to add tests for these new scenarios. Especially as I'm going to be adding CodeCov.io support soon to this repo (it evaluates our test coverage) - e.g. https://codecov.io/gh/PowerShell/xStorage
So keeping the Unit test coverage high will be good :grin:

@PlagueHO There, I've added tests for web-enrollment to both standalone and enterprise CA. The test towards standalone will also check that the inf content do not contain templates. Can you think of any more relevant tests?

Thanks for your work on this @gammalost - great job and nearly there! :grin:

Reviewed 4 of 4 files at r1.
Review status: all files reviewed at latest revision, 6 unresolved discussions.

ReadMe.md, line 79 at r1 (raw file):

### Unreleased

<<<<<<< HEAD

Can you remove these conflict markers?

DSCResources/MSFT_xCertReq/MSFT_xCertReq.psm1, line 61 at r1 (raw file):

    .PARAMETER CesURL
    The URL to the Certification Enrollment Service.

Can you please remove this blank line?

DSCResources/MSFT_xCertReq/MSFT_xCertReq.psm1, line 427 at r1 (raw file):

        ) -join '' )

    # If enrollment server is specified the request will be towards

Can you change these two lines to be a comment block? e.g.
<#
If enrollment server is specified the request will be towards
the specified URLs instead, using credentials for authentication.
#>
https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-comments

DSCResources/MSFT_xCertReq/MSFT_xCertReq.psm1, line 442 at r1 (raw file):

        )
    }
    else {

Can you move the { to the next line?

DSCResources/MSFT_xCertReq/MSFT_xCertReq.psm1, line 464 at r1 (raw file):

        if ($Credential)
        {
            # If enrollment server is specified the request will be towards

Can you change these two lines to be a comment block? e.g.
<#
If enrollment server is specified the request will be towards
the specified URLs instead, using credentials for authentication.
#>
https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-comments

DSCResources/MSFT_xCertReq/MSFT_xCertReq.psm1, line 479 at r1 (raw file):

                )
            }
            else {

Can you move the { to the next line?

Comments from Reviewable

@PlagueHO No problem :slightly_smiling_face: The necessary corrections have been made, and I've also added the new parameters to xCertReq in the README file.

Thanks @gammalost - great job!

Reviewed 2 of 2 files at r2.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable


testkube-operator-manager-role not found if testkube is installed with release name different than "testkube"

Describe the bug
When I install testkube using helm and release name is different than testkube (e.g. my-testkube) clusterrolebinding my-testkube-operator-manager-rolebinding points to wrong cluster role (it creates my-testkube-operator-manager-role and clusterrolebinding points to ClusterRole/testkube-operator-manager-role).
To Reproduce
Steps to reproduce the behavior:

Install testkube using helm with different release name than "testkube"
Check logs of testkube-operator-controller-manager pod
See error

E0310 06:29:11.857408       1 leaderelection.go:330] error retrieving resource lock my/47f0dfc1.testkube.io: leases.coordination.k8s.io "47f0dfc1.testkube.io" is forbidden: User "system:serviceaccount:my:testkube-operator-controller-manager" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "my": RBAC: [clusterrole.rbac.authorization.k8s.io "testkube-operator-manager-role" not found, clusterrole.rbac.authorization.k8s.io "testkube-operator-proxy-role" not found, role.rbac.authorization.k8s.io "my-operator-leader-election-role" not found]

Expected behavior
Every clusterrolebinding is created with expected values
Version / Cluster

Which testkube version? -- 1.9.98
What Kubernetes cluster? (e.g. GKE, EKS, Openshift etc, local KinD, local Minikube) -- local
What Kubernetes version? -- GitVersion:"v1.23.16"

Additional context
Role template -- https://github.com/kubeshop/helm-charts/blob/main/charts/testkube-operator/templates/role.yaml#L7
Role binding template -- https://github.com/kubeshop/helm-charts/blob/main/charts/testkube-operator/templates/rolebinding.yaml#L18

#419 fixes that issue

#419 fixed by


MAISTRA-1810: check timeout period to ensure the delay is relative to…

I don't understand why we are seeing such a clock drift. I don't think this issue is some sort of a race -- all calls are made from within the same thread.
Handling of timers in libevent is quite involved (I tried asking the question on their irc but no response so far), but I wonder if we are seeing an interaction between cached time, their use of gettimeofday and clock_gettime calls, and the way the drift between the last two is calculated/used. We could enable debug logging of event  handling (would show handling of timeout events) and/or reduce the interval which is used for syncing of gettimeofday and clock_gettime from 5s to maybe 1s?
I'll post the question on their mailing list. Is the cluster running under a hypervisor?

@dmitri-d I don't believe clock drift is the issue, nor threading, but rather the sequence of events with the timer and dispatch.  My theory is that the first write sets the timer, this then expires and queues up the handler, in the meantime another write event takes place (should reset the timer) followed shortly after by the timer handler.  If we are sticking to the documentation then the handler should have been cancelled and restarted.

If we are sticking to the documentation then the handler should have been cancelled and restarted.

Yep, and I can see it in the libevent code. I only just realised that we don't check the return value of the event_add call, which can be an error (-1); in case of an error the internal state of the event loop isn't changed, which could explain missed update.

/retest

/retest

/test unit

/retest

There's one test failure, but I don't think it's related to this change, but rather is a falky test.


The Documentation should contain more information

The parameteers table should contain more information. For example: type, default..

@eozbay Truly agree. But doc says You can check more info on bbb docs. Plus, check your node_modules/bigbluebutton-js. it isn't that complicated.

Valid point. I will work on updating the doc in near future.
As a temporary fix, @TERADA-DANTE suggestion is a good place to start since this library is basically a minimal wrapper to bbb api with converter from xml to json. There are no real defaults, and types correspond to those found in the bbb api.


Upload/download snapshot files between S3 and page server

Goal is that if a meteor strikes a page server, it can be fully reconstructed from the files stored in S3. S3 is authoritative source of truth; in a sense, the page server is only a cache of what's stored in S3.
It's not clear how e.g. branch creation should work in this model. Should the control plane create the branch in S3, and the page server automatically notices that the new branch appeared? Or is there an API between the control plane and page server to communicate such things. This needs some design work. See also https://github.com/zenithdb/rfcs/blob/7810fcadbcd140241cd595304df1eeb7a7dd6718/snapshot-first-storage.md#cloud-snapshot-manager-operation

There's a similar issue at https://github.com/zenithdb/zenith/issues/230. That was underspecified, and ISTM it evolved into something more like import/export. I'm opening this new issue to track specifically the idea that the uploading/downloading to S3 is based on the immutable snapshot files, as Eric alluded to in this comment: https://github.com/zenithdb/zenith/issues/230#issuecomment-865111139.

Should the control plane create the branch in S3, and the page server automatically notices that the new branch appeared? Or is there an API between the control plane and page server to communicate such things.

There were some recent discussions about branch/tenant creation and since it is mentioned in this issue I'll respond here.
Right now it is done by direct communication between console and page server ('branch_create' command in page_service.rs).
The whole concept of passing information between closely located console and page server not directly, but through remote fs-like storage instead seems weird to me, but let's imagine we are doing it this way.
Let's also keep in mind that we are trying to create compute node in a matter of seconds. Right now nothing in principle prevents us from doing this within a 1-2 seconds timeframe.

There is a tricky part about 'page server automatically notices that the new branch appeared' -- notices how? I can see 2 options here:
a. Use S3 Event Notifications service. Right on the first page of this service, there is a statement that Typically, event notifications are delivered in seconds but can sometimes take a minute or longer. We can just disqualify that options based on latency characteristics.
b. Notify page server directly from console after creating a branch in S3. That kind of invalidates any need in a previous console communication with S3 because we anyway going to contact the page server about this. And the page server anyway will need to go to S3 at some point in time.

Rails exposure to our storage format.
We would need to understand directory structure and file formats in the rails app. That is some amount of code that needs to be in sync with the same code in the page server. Also, that would complicate deployment since we would need to update them both if we need to change something in file format.

Page server-only information.
Some information about branches/tenants is needed for page server operations only (quotas, GC settings, etc) and there is no need for it to be stored at S3 or even be stored on disk in some kind of custom format since we have it in console DB.

Latency.
I just have run these commands from ec2 in us-west-1 (both VM and s3 in the same AZ):

    $ time aws s3 mb s3://2136481274361289480
    make_bucket:<PHONE_NUMBER>361289480

    real	0m0.971s

    $ echo 'hello world' > test

    $ time aws s3 cp test s3://2136481274361289480/test
    upload: ./test to s3://2136481274361289480/test

    real	0m0.383s

    $ time aws s3 ls s3://2136481274361289480/
    2021-07-19 09:03:45          8 test

    real	0m0.369s

compare that with direct communication with page server:
    admin@ip-172-31-66-237:~$ time psql -h <IP_ADDRESS> -p 6400 -c 'status'
        data
    -------------
    hello world
    (1 row)

    real	0m0.012s

So this way we would just add about a second to our compute node start which is quite noticeable.
So to summarize: S3 is cool and cheap storage but trying to use it as a substitute for a database or network connection only creates problems.

Goal is that if a meteor strikes a page server, it can be fully reconstructed from the files stored in S3. S3 is authoritative source of truth; in a sense, the page server is only a cache of what's stored in S3.

In what sense is that true, and in what sense is that not true? This seems important to me.

A few responses:

page server automatically notices that the new branch appeared

I'm not sure why we'd expect it to. If a new branch was created, I would think that was either done by the control plane, or commanded by the control plane; that control plane can then notify the right pageserver.

Rails exposure to our storage format.
We would need to understand directory structure and file formats in the rails app. That is some amount of code that needs to be in sync with the same code in the page server.

I agree, that seems tricky. My first thought would be to build an FFI rust crate that has access to the functions that understand directory structure & file formats. A less powerful option would be a standalone rust binary that can run the necessary operations as a command-line tool.

s3 Latency

I agree that creating a new timeline in s3 would add some delay, though connecting to an existing timeline would not. There might be a few workarounds for this:

Create the timeline concurrently with pageserver start.
Create empty timeline files ahead of time and assign them as new database creations are requested.

If new database creation time is a critical metric, then maybe it's OK to allow the pageserver to run only that operation in a non-durable manner. I don't think it's a good idea to give ourselves that kind of freedom (to create non-durable state) in general, though.

I'm not sure why we'd expect it to. If a new branch was created, I would think that was either done by the control plane, or commanded by the control plane; that control plane can then notify the right pageserver.

Right. But that thing creates logical trouble for me: why then console talked to S3 in the first place if it anyway need to talk to page server API after that. Why don't we put corresponding code to the page server in the same API call? This way page server would create corresponding metadata in S3 by himself. And this call could be synchronous regarding to S3 operation so we don't introduce any non-S3-backed state in the page server.

I agree, that seems tricky. My first thought would be to build an FFI rust crate that has access to the functions that understand directory structure & file formats. A less powerful option would be a standalone rust binary that can run the necessary operations as a command-line tool.

And what I'm suggesting is to put all that logic in the page server (and it need to know that formats anyway) and use http API calls to run these operations from the console.

Create the timeline concurrently with pageserver start.

I think you meant concurrently with postgres start. Yeah, that may work -- if we do that in parallel it may cover S3 latency. If that would be the case then we definitely should do metadata operations synchronously and avoid dealing with cases when pageserver crashed before creating metadata in S3.

not sure who this should be assigned to, assigning to @hlinnaka. @hlinnaka can you delegate if you're not the right owner for this issue?

@lubennikovaav presented some early thoughts on the design last week.

@SomeoneToIgnore As I can see there are issues for the next parts. I assume that we can close that one


Add view & log changes globally

.. so one could track changes made by users, restricted by role

hi @S1SYPHOS .
i added the pagemethod findRecentlyModifiedByUser recently. haha pun intended. so the following should work.
sections:
  recentarticles:
    type: recentlymodified
    headline: Recently Modified Articles
    # query: site.index(true).sortBy('modified', 'desc') # all pages here or as php setting

return [
    'bnomei.recently-modified.query' => 'site.index(true).sortBy('modified', 'desc')', // all pages
    'bnomei.recently-modified.limit' => 10000, // track 10k pages globally
    'bnomei.recently-modified.info' => function ($page) {
        $user = $page->findRecentlyModifiedByUser(); // ?User
        return ($user ? $user->name() : '') . ' | ' $page->modified();
    },
    // other options...
];

this will show a list of all pages sorted by modified timestamp and which user changed it most recenlty. if you want to list the last few editors for a single page, not just the most recent editor, then take a look at the findRecentlyModifiedByUser pagemethod and just build your own based on that.


CFBundleVersion is not valid for all platforms

New Issue Checklist

[x] I have read and understood the CONTRIBUTING guide
[x] I have read the Documentation
[x] I have searched for a similar issue in the project and found none

Issue Info

Info
Value

Platform Name
iOS

Platform Version
10.2.1

SDWebImage Version
4.0.0

Integration Method
carthage

Xcode Version
Xcode 8.2.1

Repro rate
100%

Repro with our demo prj
No

Issue Description and Steps
When submitted to iTunes Connect/TestFlight the archive is rejected because the CFBundleVersion does not have a value in the Info.plist.
"ERROR ITMS-90056: This bundle is invalid. Payload/YOUR_APP_HERE/Frameworks/SDWebImage.framework Info.plist file is missing the required key: CFBundleVersion"
I saw in the project that it is set to 1 for some platforms and is missing for others - especially iOS. It seems Apple have some new more strict rules for the provisioning and bundles metadata effective since yesterday. I'd suggest you maintain a better versioning for the project and it's targets.

Same problem here

Let's see how we can fix this. I assume it's just missing from the Info.plist, even if that plist was generated by Xcode.

Yeah. I'd suggest you just go over all of the different targets' Build Settings and chamge Current Project Version value or go to the Info.plist an change it there. Just make sure all targets have one of these set to something meaningful to your progress on the project.

Same problem...
I screamed and directly it edited. 😱
$(SRCROOT)/Carthage/Build/iOS/SDWebImage.framework/info.plist

So, do I have to change it manually?

@omatty198 I did the same thing :)

Can any of you guys create a PR with this change?

I am using Carthage and am hitting this exact error - ERROR ITMS-90056: "This bundle .../SDWebImage.framework is invalid. The Info.plist file is missing the required key: CFBundleVersion."
Is there a workaround for this?  Branch, fix - need help :)

I checked out the pull request #1784
Worked fine for me

Great guys.
@ivan-konov PR was merged into master.
Closing this.

Sure, glad to help!

I've updated with latest version, but still happening...

I've modified the Info.plist manually to make the upload as @omatty198 pointed out.

@jfsagasti by "update to the latest version" you mean cloning master branch or downloading latest release?

You are right. Carthage picked the latest release I guess (I don't have "master" specified in the cartfile). Sorry for the oversight!

@jfsagasti If you are in a rush and need to submit soon I would suggest you stick to the manual change solution and not really rely on master as a stable codebase.

That's exactly what I did, thanks! I needed to submit a new version today 👍

I had this same problem today. Manually edited the SDWebImage.framework/Info.plist file to set the CFBundleVersion


Does the Admin Add RB Form Need a Field for Campaign Run?

FEATURE OVERVIEW
User Story
Leads and the impact team use this form decently often to add reportbacks for members. Does this page need a field for run ID?
https://thor.dosomething.org/admin/users/rb/add
Why This Matters
Reportbacks must be associated with the correct run.

yes. it does.


Add timestamp of last successful attestation to verifier API (enhancement #74) and address some missing documentation

This pull request fulfills https://github.com/keylime/enhancements/issues/74 by adding a new database column (last_successful_quote) and exposing it through the verifier's REST API. Many thanks to @maugustosilva for his help with this in the Slack channel.
Additionally, I've adding some descriptions to the API documentation for the last_successful_quote field and a couple other fields that were missing documentation (namely, attestation_count and last_received_quote).
See commit messages for further details.

Codecov Report

Merging #1313 (6edc355) into master (6583d0d) will increase coverage by 0.48%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files

Flag
Coverage Δ

packit-e2e
74.59% <ø> (+0.48%)
:arrow_up:

testsuite
11.09% <ø> (ø)

unittests
1.92% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

keylime/keylime_agent.py
73.12% <0.00%> (+0.57%)
:arrow_up:

@stringlytyped thank you for the PR and welcome to Keylime! The changes in general LGTM, but I would name the option last_successful_attestation to make it more clear that it refers to the last successful attestation round.

@stringlytyped very welcome addition, especially the API documentation (which I forgot to add for attestation_count and last_successful_quote).
We are very close to a merge, but please change the column name to last_successful_attestation (@THS-on is correct on pointing out that this will avoid the eventual confusion about wether the quote was simply received by the verifier or was actually fully attested).
As for the pylint errors, you can either run make check, ./scripts/check_codestyle.sh (for pylint only) or configure the per-commit hooks (which I recommend in case you plan to contribute more)

Thanks @THS-on, @aplanas and @maugustosilva for your feedback. It has taken me a moment to get your suggested changes implemented and to fix those formatting issues because I was struggling to get the formatters/linters/etc. working in my local environment. But I think that should be all good to go now, pending CI actions.

The only outstanding issue is on fedora-rawhide-x86_64. Looking at the logs, I cannot see how the failures could be due to my changes. Is there a chance this is related to this bug and the upgrade in Fedora 38 of tpm2-tss and tpm2-tools to versions 4.0.1 and 5.4 respectively?

The only outstanding issue is on fedora-rawhide-x86_64. Looking at the logs, I cannot see how the failures could be due to my changes. Is there a chance this is related to this bug and the upgrade in Fedora 38 of tpm2-tss and tpm2-tools to versions 4.0.1 and 5.4 respectively?

@stringlytyped  yes that failure is in the agent TPM code as you correctly identified and has nothing to do with this change.
Can you integrate the last fixup commit into the two first ones? After that this it should be ready to merge:)

Can you integrate the last fixup commit into the two first ones? After that this it should be ready to merge:)

@THS-on Sure thing. Is it okay if I squash the commits (easier than rebasing manually) or would you prefer the changes to the code base in a separate commit from the updated docs?

@stringlytyped either one is fine for me because all the changes belong logical together.

As we discussed, the Fedora Rawhide failure is not related to your commit. Thank you again, will merge.


DOC: Documenting the use of subdirectory for template installation with 'langchain app add'

URL
No response
Checklist

[X] I added a very descriptive title to this issue.
[X] I included a link to the documentation page I am referring to (if applicable).

Issue with current documentation:
Description:
I recently discovered a very useful feature in the LangChain CLI that allows templates to be installed from a specific subdirectory within a repository using a URL fragment, like so:
<EMAIL_ADDRESS>However, I was unable to find any documentation on this feature in the current LangChain documentation, and I had to dig into the source code to find out how to use it. This feature is incredibly useful for managing multiple templates in a single repository and would greatly benefit other users if it were documented.
Proposed Solution:
Add a section in the documentation that explains how to install templates from a specific subdirectory within a repository using the URL fragment notation.
Example:
langchain app add<EMAIL_ADDRESS>Additional Context:
This feature allows users to manage and install multiple templates from a single repository, which is a common use case for organizing LangChain templates. Including this in the documentation would improve user experience and reduce the need for source code exploration.
Idea or request for content:
No response

Please create a sample repository for the documentation rather than using my repository used i my issue report. Thank you!


Songs don't play properly and chart editor crashes

Describe your problem here.
I've been trying to mod the source code, and to test out a character json along with its icon, I wanted to go into the chart editor and replace GF with the character in Tutorial. When doing this, however, I accidentally discovered that songs don't seem to be running their proper files (charts work, but the songs themselves don't play properly) and it's instead replaced with the Menu music. When opening up the chart editor, it crashes and I get this:

Is there something wrong with how it's built to make it act like this, or is it because of something in the files?
Are you modding a build from source or with Lua?
Source
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
The things I mostly did were changing the icon art and also Project.xml, though mostly changing the window title and a few other things in it. I also changed a few of the week jsons to be hidden from Story Mode and Freeplay.

Alright, so I changed Project.xml to have mods being allowed, and that... fixed it? No clue why that happened, but I'm at least glad that I managed to fix it.


Update dashboard

What should go in the dashboard?

This is going to be where mental models and usage comes in. Do people think about all of the data coming into their server as related? Is each form entirely separate in their minds? How often does it make sense for several forms to be grouped? @yanokwa

Maybe it only makes sense to add a dashboard once we have campaigns? I think that campaigns will be a natural grouping unit for forms, but without them, a single server's forms might not all be related. Maybe before we implement campaigns, the landing page can just be the forms list?

That is all sounding good to me.

Sounds good! I'll close the issue for now, but we can revisit once we get to campaigns.


[Hosts] Encoding setting

Summary of the Pull Request
New setting to chose the encoding of the hosts file between UTF8 (default) and UTF8 with BOM.
CC @crutkas

PR Checklist

[x] Closes: #24842
[ ] Communication: I've discussed this with core contributors already. If work hasn't been agreed, this work might be rejected
[ ] Tests: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Dev docs: Added/updated
[ ] New binaries: Added on the required places

[ ] JSON for signing for new binaries
[ ] WXS for installer for new binaries and localization folder
[ ] YML for CI pipeline for new test projects
[ ] YML for signed pipeline

[ ] Documentation updated: If checked, please file a pull request on our docs repo and link it here: #xxx

Detailed Description of the Pull Request / Additional comments

Validation Steps Performed

Start the editor
Make a change: file should be saved with UTF8 encoding
Open settings (don't close editor)
Change encoding to UTF8 BOM
Make a change: file should be saved with UTF8 BOM encoding
Close editor
Start editor
Make a change: file should be saved with UTF8 BOM encoding

Works for me! Don't even need to close the editor between changing the Setting, it just applies.


Log level

Hi Adrian,
Thanks for this work.
I  am using your API, but I cannot configure optuna log level. Every time I run a study, a lot of messages, for each try, fill the log no matter if I previously configure it :
optuna.logging.disable_default_handler()
optuna.logging.set_verbosity(optuna.logging.WARNING)
optuna.logging.disable_propagation()
¿Is there a way to avoid it?
Thanks in advance
Aura

Hi!
That's because we're using our own log handler, which at the moment does not have a nice API to configure it. For the time being, you can use a workaround:
import logging
import random
import time

import optuna
import optuna_distributed

def objective(trial):
    x = trial.suggest_float("x", -100, 100)
    y = trial.suggest_categorical("y", [-1, 0, 1])
    time.sleep(random.uniform(0.0, 2.0))
    return x**2 + y

if __name__ == "__main__":
    logging.getLogger("optuna_distributed").setLevel(logging.WARNING)
    optuna.logging.disable_default_handler()
    study = optuna_distributed.from_study(optuna.create_study())
    print("Running 10 trials...")
    study.optimize(objective, n_trials=10)
    print(f"Best value: {study.best_value} (params: {study.best_params})")

I'll try to introduce Optuna-like way to suppress log messages when there's more time.

Hi,
I've added optuna style logging helpers in 0.5.0 release. They can be used as follows:
import random
import time

import optuna

import optuna_distributed

def objective(trial):
    x = trial.suggest_float("x", -100, 100)
    y = trial.suggest_categorical("y", [-1, 0, 1])
    time.sleep(random.uniform(0.0, 2.0))
    return x**2 + y

if __name__ == "__main__":
    optuna.logging.disable_default_handler()
    optuna_distributed.config.disable_logging()

    study = optuna_distributed.from_study(optuna.create_study())
    print("Running 10 trials without logging...")
    study.optimize(objective, n_trials=10)
    print(f"Best value: {study.best_value} (params: {study.best_params})")

    optuna_distributed.config.enable_logging()
    print("Running 10 more trials with logging...")
    study.optimize(objective, n_trials=10)
    print(f"Best value: {study.best_value} (params: {study.best_params})")

Note that logging is enabled by default, so there's no need to call enable_logging if disable_logging wasn't called before.


s21-7pm-1 - sc/sk - changed heading on Analyze Slack Data/Analyze Reactions

In this PR, we changed the heading from "User" to "Message Author" on the Analyze Slack Data/Analyze Reactions page.

mvn and npm compiled, and UI on localhost does reflect change from 'User'->'Message Author'


Timestamp information or sampling rate

Thank you for this excellent work, as well as the valuable dataset. I've been trying to find any information about the timestamps for each stroke point, or a sampling rate. I think the original IAM-OnDB had timestamps, but even a sampling rate (if the stroke points are picked after regular intervals) for this dataset would be sufficient.
I'd appreciate any help you can provide in this regard. Thanks!

Hi,
We released an extended version of our dataset which includes timestamp information as well. Please visit our project page https://ait.ethz.ch/projects/2018/deepwriting/


[Docs]: Docs on replicated storage backups with snapshots

Are you reporting an issue with existing content?
I could only find backup docs for local storage and not replicated storage (may be I am blind). I am not sure if the same procedure is recommended for replicated as well, given it supports snapshots and other functionality. It would very much benefit the community if there is a doc on this topic and what the recommended tool / setup is.
Are you proposing new content, or a change to the existing documentation layout or structure?
A document under replicated storage header with the name backups explaining how to do replicated / snapshotted backup with a given tool - velero / k8up / etc.

Sure @avishnu. Also, are the other 2 i.e. "Velero Backup and Restore using Replicated PV Mayastor Snapshots - Raw Block Volumes" & "Kasten Backup and Restore using Replicated PV Mayastor Snapshots - FileSystem" applicable for 4.1?

Yes

This issue is documented in PR #496.


cxxopts::ParseResult returned by value is broken

Thanks for the nice lib,
here is an example https://wandbox.org/permlink/qp0geTxEPm6Ldjy3
#include <string>
#include <iostream>

//commit 7c468aaf68206f0f05fcf6c436ed274da138bdbc
#include "cxxopts.hpp"

cxxopts::ParseResult handleCLI(int argc, char ** argv){
  cxxopts::Options options("testapp", "");
  options.allow_unrecognised_options().add_options()
  ("o,outfile", "File name", cxxopts::value<std::string>());
  auto cliArgs = options.parse(argc, argv);
  if (cliArgs.count("outfile")) {
    std::cout << "found " << cliArgs["outfile"].as<std::string>() << "\n";
  }
  return cliArgs;
}

int main(int argc, char ** argv){
    auto cliArgs = handleCLI(argc, argv);
    if (!cliArgs.count("outfile")) {
       std::cout << "strange\n";
    }
}

the output is
found/dev/null
strange

i've also had a look in gdb on mains cliArgs and the object is in an invalid state

A ParseResult object can't leave the scope of the Options that it was created from because it holds references to the original object.

So there is a dependency, I would then recommend to share the data in shared_ptr between the objects, or make ParseResult uncopyable/unmovable (it will not avoid every misuse, but at least avoid this case)

It needs to be at least moved to return it from Options::parse, so that doesn't work.
I will look at whether I can implement this in a better way.


make attime specifications honor the requested timezone

Currently time specifications like midnight are always treated as being relative to UTC, this patch changes them to be treated as relative to the specified timezone, either from the tz parameter in the request or the TIME_ZONE setting.  This changes the counter-intuitive behavior reported by @Dieterbe in #639
This does represent a fairly significant departure from the current behavior, albeit one that I think makes things much more sensible from a user perspective.

Codecov Report

Merging #1989 into master will decrease coverage by 0.04%.
The diff coverage is 92%.

@@            Coverage Diff             @@
##           master    #1989      +/-   ##
==========================================
- Coverage    69.5%   69.45%   -0.05%
==========================================
  Files          71       71
  Lines        6961     6953       -8
  Branches     1356     1357       +1
==========================================
- Hits         4838     4829       -9
  Misses       1882     1882
- Partials      241      242       +1

Impacted Files
Coverage Δ

webapp/graphite/render/views.py
46.56% <0%> (ø)
:arrow_up:

webapp/graphite/render/attime.py
99.13% <95.83%> (-0.87%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9207d0...90fe80b. Read the comment docs.


Should set bookSettings.ValidationFlags = XmlSchemaValidationFlags.ReportValidationWarnings

Without the ValidationFlags property set to XmlSchemaValidationFlags.ReportValidationWarnings, no schema validation warnings are returned.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e99bf3b5-3eb2-a0d3-2009-4fb2b9de3b76
Version Independent ID: 8252e3d3-0718-9814-d685-852387f16afd
Content: XML Schema (XSD) Validation with XmlSchemaSet
Content Source: docs/standard/data/xml/xml-schema-xsd-validation-with-xmlschemaset.md
Product: dotnet
Technology: dotnet-standard
GitHub Login: @mairaw
Microsoft Alias: mairaw

@masterofimages Thanks for reporting this. We'll fix it the next time we update this article. If you'd like, you're welcome to submit a pull request to fix it as well.


Reduce podspec to 2subspecs and remove customized modulemap

New Pull Request Checklist

[x] I have read and understood the CONTRIBUTING guide

[x] I have read the Documentation

[x] I have searched for a similar pull request in the project and found none

[x] I have updated this branch with the latest master to avoid conflicts (via merge from master or rebase)

[x] I have added the required tests to prove the fix/feature I am adding

[x] I have updated the documentation (if necessary)

[x] I have run the tests and they pass

[x] I have run the lint and it passes (pod lib lint)

This merge request fixes / refers to the following issues: #954
Pull Request Description
This PR reduces podspec to 2 subspaces(Core and Swift) to simplify usage of CocoaPods and match to Carthage.
And PR also removes customized modulemap(They needs to be adopted at the same time).
Thanks.

@ffried
Oh, approved too fast:) Thanks.

@sushichop Looks good to me. Travis will have the last word, though. 🙂

          2 Warnings

:warning:
Any changes to library code should be reflected in the CHANGELOG. Please consider adding a note there about your change.

:warning:
The library files were changed, but the tests remained unmodified. Consider updating or adding to the tests to match the library changes.

  Generated by :no_entry_sign: Danger

Codecov Report

Merging #976 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #976   +/-   ##
=======================================
  Coverage   25.26%   25.26%
=======================================
  Files          21       21
  Lines        3451     3451
=======================================
  Hits          872      872
  Misses       2579     2579

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 01f314d...06aa9e2. Read the comment docs.

[ci skip]

I have just got to review this PR. Overall looks good, thanks @sushichop
A few comments to think about:

per https://guides.cocoapods.org/syntax/podspec.html#public_header_files

public_header_files
A list of file patterns that should be used as public headers.
These are the headers that will be exposed to the user’s project and from which documentation will be generated. When the library is built, these headers will appear in the build directory. If no public headers are specified then all the headers in source_files are considered public.

so I guess we don't need to specify the public_header_files if all the headers from source_files will be public. Just a small detail
From a certain point of view, removing the modulemap means people who used import CocoaLumberjack.DDDispatchQueueLogFormatter or similar will need to change their code. That means in semantic version words that this should be a major release. I don't think we should be marking the release with a major bump of version (meaning 4.0.0) but just wanted to state this.
It's good to have some parity between CocoaPods and Carthage, but it's still not full parity, as Carthage will create 2 frameworks CocoaLumberjack.framework and CocoaLumberjackSwift.framework. To match exactly, we would need to add a dedicated CocoaLumberjackSwift.podspec, but I'm also not sure about this.

Just would like to hear some feedback on these items. Glad that the PR is merged :)

PS: Out of curiosity, ran objc-diff between version 3.4.2 and the latest master, we don't have ObjC API differences (just a change of param types from NSUInteger to NSInteger).

CocoaLumberjack.framework/Headers/CocoaLumberjack.h

Added CocoaLumberjackVersionNumber
Added CocoaLumberjackVersionString

CocoaLumberjack.framework/Headers/DDContextFilterLogFormatter.h

Modified -[DDContextWhitelistFilterLogFormatter addToWhitelist:]

Declaration
From- (void)addToWhitelist:(NSUInteger)loggingContext
To- (void)addToWhitelist:(NSInteger)loggingContext

Modified -[DDContextWhitelistFilterLogFormatter removeFromWhitelist:]

Declaration
From- (void)removeFromWhitelist:(NSUInteger)loggingContext
To- (void)removeFromWhitelist:(NSInteger)loggingContext

Modified -[DDContextWhitelistFilterLogFormatter isOnWhitelist:]

Declaration
From- (BOOL)isOnWhitelist:(NSUInteger)loggingContext
To- (BOOL)isOnWhitelist:(NSInteger)loggingContext

Modified -[DDContextBlacklistFilterLogFormatter addToBlacklist:]

Declaration
From- (void)addToBlacklist:(NSUInteger)loggingContext
To- (void)addToBlacklist:(NSInteger)loggingContext

Modified -[DDContextBlacklistFilterLogFormatter removeFromBlacklist:]

Declaration
From- (void)removeFromBlacklist:(NSUInteger)loggingContext
To- (void)removeFromBlacklist:(NSInteger)loggingContext

Modified -[DDContextBlacklistFilterLogFormatter isOnBlacklist:]

Declaration
From- (BOOL)isOnBlacklist:(NSUInteger)loggingContext
To- (BOOL)isOnBlacklist:(NSInteger)loggingContext

CocoaLumberjack.framework/Headers/DDDispatchQueueLogFormatter.h

Added DDAtomicCountable
Added -[DDAtomicCountable initWithDefaultValue:]
Added -[DDAtomicCountable increment]
Added -[DDAtomicCountable decrement]
Added -[DDAtomicCountable value]
Added DDAtomicCounter

@bpoplauschi
Thanks for your comment. I agree with you totally.

so I guess we don't need to specify the public_header_files if all the headers from source_files will be public. Just a small detail

Yes, definitely! I'll create a PR later to do the above thing.

From a certain point of view, removing the modulemap means people who used import CocoaLumberjack.DDDispatchQueueLogFormatter or similar will need to change their code. That means in semantic version words that this should be a major release. I don't think we should be marking the release with a major bump of version (meaning 4.0.0) but just wanted to state this.

I agree with you. Developers who build with Carthage or manually needs to change their code a little about Extensions podspec.
e.g.

from import CocoaLumberjack.DDDispatchQueueLogFormatter
to import CocoaLumberjackSwift

It's good to have some parity between CocoaPods and Carthage, but it's still not full parity, as Carthage will create 2 frameworks CocoaLumberjack.framework and CocoaLumberjackSwift.framework. To match exactly, we would need to add a dedicated CocoaLumberjackSwift.podspec, but I'm also not sure about this.

Yes, you are right. To match exactly, CocoaLumberjackSwift.podspec is also needed because CocoaLumberjack project has two target(CocoaLumberjack and CocoaLumberjackSwift), you know.
Or we have another option. I think we also can merge Swift code to CocoaLumberjack target like OHTTPStubs.
https://github.com/AliSoftware/OHHTTPStubs
Or we also can choose the option like Realm(pure Objective-C or Objetive-C/Swift mixed) and RealmSwift(pure Swift) for the distant? future.
https://github.com/realm/realm-cocoa
But I'm not sure which is the best solution, and I also think it is easy that we just add CocoaLumberjackSwift.podspec for now(though I'm not sure it is strongly necessary for developers now...😅

About the last 2, I don't think we need to change anything, but wanted to at least talk about them. :)

About the last 2, I don't think we need to change anything, but wanted to at least talk about them. :)

I see😃


feat add wikipedia incremental

This will replace South Park incremental indexing example

docs(wikipedia): update readme to use latest docker image on hub
feat(wikipedia-incremental): initial commit
docs(wikipedia-incremental): clean up formatting

Thanks @theUnkownName and @Yongxuanzhang . I'll fix tomorrow and get back to you

Thanks @theUnkownName and @Yongxuanzhang . I'll fix tomorrow and get back to you

No problem! You could directly request us here to review the pr. 😄

It says I still need approval from @theUnkownName . It's weird though - I've marked all issues as resolved now