[13pt] Diagnose and resolve cases where dem_adj_elevation is much different than USGS-reported elevation

The following sites have dem_adj_elevation values (from usgs_elev_table.csv files) very far off from the official USGS elevation. I suspect thalweg notch may be to blame for many sites. To get full metadata, including HUC number, you can use the attached Workbook and filter on status using large discrepancy in elevation estimates from gage and HAND and all categories failed to map.
amym6
augg1
bawt2
bgbm6
blda1
brfw3
caxm7
crnz1
ctic1
fcwt2
flvt2
fulm6
gabt2
gtem5
hamo1
hwdn8
isvo1
kwpm7
likf1
lowm3
lsec1
luft2
midf1
mion8
mklm5
mspm5
mvom5
ncet2
nrfm5
nshn3
nulm5
plkt1
ptet2
ratm6
ripc1
smat2
supt2
tcla1
tyla1
unvc3
waak1
wfon8
wspf1
stage_based_catfim_sites_4_3_11_0.xlsx

I have gone through the points with the status message of  large discrepancy in elevation estimates from gage and HAND and all categories failed to map and taken notes about where the issue could be coming from.
Summary of current findings:

There are 44 points with this status message
Out of those 44, 21 points have a USGS altitude of zero
Another 15 points have a wildly decreased elevation disparity when the units of the USGS altitude are converted

These findings are being organized in the Site Organizer Google Sheet.

Single point evaluation: mspm5 - HUC 07010206
For this point (as well as several others), incongruent vertical datums and vertical units make it difficult to determine whether the NWS elevation or the USGS datum is more suitable for use as a zero datum in the CatFIM workflow, or if neither are correct.
We would expect the following values to be fairly close in value, but for LID mspm5 they are not:
HAND datum elevation value: 243.19 meters = 797.87 ft (in NAVD88)
NWS zero datum elevation value: 794.3 ft (in NGVD27),  795.06 (in NAVD88)
USGS gage elevation value: 710.2 ft (in NAVD88)
The NWS zero datum (794.3) has the vertical datum listed as "1938 by CITY," but elsewhere in the metadata the elevation is also listed with the projection NGVD27. If you use the NOAA Vdatum API to convert the NWS zero datum into NAVD88, the NWS zero datum elevation value is much closer to the HAND datum elevation value (listed above).
In this case, homogenizing the vertical datum and vertical unit of each data source reveals that the NWS value seems much more believable than the USGS value and the flagged difference between the two number is due to an incorrect USGS value.

Evaluating multiple points with a Python script
Background and Methods
After analyzing some individual cases to determine potential causes for the elevation disparity, I assembled a Python script to test for those potential errors in the full set of 44 error points.
This evaluation takes place on a dataset that the script compiles, which is made up of data from the stage_based_catfim_sites_4_3_11_0.xlsx file and the usgs_elev_tables.csv file for each HUC. This specific combination of data sources makes several types of errors easier to recognize.
The script tests for indicators of the following errors:

Thalweg notch: indicated by a large difference between the dem_elevation and the dem_adj_elevation columns in the usgs_elev_table.csv
Snapping error: possible indicated by a large value in the snap_distance column
'nodata' value in dem_elevation column
Vertical datum inconsistencies

The script (included below) also converts the USGS and NWS elevation values to the NGVD88 vertical datum and compares the results to the NWM HAND datum elevation value. This process reveals whether the NWS or USGS value is closer to the expected value and, in some cases, reveals whether an elevation value of zero is reasonable or erroneous.
investigate_dem_disparity_script.zip
Results and Discussion
Thalweg notch:
A DEM difference threshold of 10 ft was used to flag points potentially affected by the thalweg notch. It is also important to note that this error is checked for elsewhere in the CatFIM workflow.
Only one point (AUGG1) had a DEM difference higher than this (89.8 ft). This point displayed a number of other inconsistencies, including elevation values of -0.61 and 0 (where the expected value was about 111 ft) and a NWS vdatum of '1957', which is not one of the expected values of NGVD29 or NAVD88.
Snapping error:
A snap distance of 75 or higher was flagged as a potential snapping error. This threshold flagged 7 out of the 44 error points. However, this can be a misleading metric when it is not contextualized by the size of the stream.
For example, LID MSPM5 (from HUC 07010206) has a snap distance of 98.05 ft. This value seems high and was flagged by the threshold. However, this gage is on a part of the the Mississippi river that is over 600 ft wide (pictured below). In that context, a snap distance of 98 feet is a reasonable distance to expect between a gage and the stream centerline.
While it is not unlikely that snapping errors are occurring in some instances, applying a threshold to the snap distance alone is not a clear indicator of when it is occurring. In future analyses, it would be useful to use different snap distance thresholds for different stream orders. This would more effectively flag snapping errors because while a snap distance of 98.05ft isn't an indication of snap error in the large Mississippi River, a snap distance of 98.05 ft in a small, first order stream would be a definite cause for concern.

River width at LID MSPM5 (from HUC 07010206) on the Mississppi River in Fridley, MN
'nodata' value in dem_elevation column:
5 out of the 44 points in the error dataset have the nodata value (-999999) in their dem_elevation column. These points are LID's AMYM6, BGBM6, FULM6 from HUC 3160101, LID WFON8 from HUC 9020204,  and LID CAXM7 from HUC 10300101. These five points overlap with levee-protected areas, which explains why they have been assigned the 'nodata' value.

Points with a 'nodata' value also overlap with levee-protected areas
Erroneous zero values
In the 44 error points, there are 19 points where either the USGS or NWS elevation (or both) is zero where the HAND datum is not anywhere close to zero. In some cases, it would be possible that an elevation value of zero (or close to zero) is a valid indicator of a point that just has an elevation close to sea level. However, in these 19 cases the zero value present in the USGS and/or NWS elevation is paired with a HAND datum value that is much higher than zero.
For many points (GABT2, BAWT2, CTIC1, LSEC1, TYLA1, FCWT2, NCET2, ISVO1, GTEM5, LIKF1, WSPF1, NULM5, NRFM5), both the USGS and the NWS entries have an erroneous zero value in their elevation. In several other cases (AUGG1, CRNZ1, RATM6, RIPC1), the USGS and/or NWS entries have elevation values that is very close to zero but not quite zero (while the HAND datum is far from zero). In a few other cases ( NSHN3, PTET2), the USGS elevation has an erroneous zero and while the NWS elevation looks fairly believable.
The issue of erroneous zero values could be one of the main reasons why this error message is being recorded. It is possible that in certain some scenarios, a zero is being recorded as some type of placeholder value, but it is being mistaken as an actual value since nothing distinguishes a zero placeholder value from an actual zero elevation value.
These erroneous zero values can be traced in the annotated results dataframe.
Inconsistencies in vertical datum recording
Another pattern that is clear from the annotated results dataframe is the surprisingly high amount of inconsistencies in the vertical datums are used and how they are referenced.
For example, in the manual evaluation of LID mspm5, converting both the NWS and USGS value to the NAVD88 datum allowed us to see that the NWS value is much closer to the HAND datum than the USGS value is. This manual analysis was made possible because the NWS metadata included the elevation for two vertical datums: "1938 by CITY" and "NGVD27". The presence of the secondary elevation value in the NGVD27 datum made it possible to convert the value from NGVD27 to NAVD88 so that it could be compared with the elevation values from the other two sources. In the data from the CatFIM workflow, however, only the "1938 by CITY" vertical datum is included. The exclusion of the secondary vertical datum from the data means that this discrepancy that we caught manually was not able to be caught in the Python workflow that simply automated the same processing steps.
While this inconsistency does not seem like it would be the root cause of the the CatFIM error, the lack of consistency and missing vertical datum information decreased the efficacy of the automated error-catching system and made it more challenging to track down the correct elevation values for the flagged points.

Screenshot of annotated results dataframe
ANNOTATED_analysis_results_df.xlsx

@CoreyKrewson-NOAA @shawncrawley @ChipGobs-NOAA Please see Emily's findings in above comment.


Horizon: Add all endpoints to root '_links' response

What problem does your feature solve?
The Ruby SDK uses Horizon's hal+json content via Hyperclient, an API client for auto-discovering endpoints. It reads horizon's root endpoint response and constructs methods for each endpoint listed under _links.
However, many endpoints are simply missing, making it difficult to access these endpoints via the Ruby SDK.
What would you like to see?
All endpoints listed in Horizon's API Reference  included in the response to GET https://horizon.stellar.org under _links.
What alternatives are there?
The Ruby SDK doesn't have any alternative other than building explicitly defined requests for each endpoint, which would be a ton of work and not worth the effort compared to this fix.

Can you enumerate which endpoints have this problem?

It would be easier to list what is already present. If you click this you'll see we have properties like account_transactions under _links. The Ruby SDK takes these and automatically builds methods:
client = Stellar::Client.default
client.horizon.account_transactions(account_id:'<account_id')

But for endpoints like account/{account_id}/offers, there is no such entry under _links and therefore the Ruby SDK's client doesn't have a natural way to hit it.

Endpoints linked the root resource:

[ ] Account Details
[ ] Accounts
[ ] All Assets
[ ] Data for Account
[ ] All Effects
[ ] Effects for Account
[ ] Effects for Ledger
[ ] Effects for Operation
[ ] Effects for Transaction
[ ] Fee Stats
[ ] All Ledgers
[ ] Ledger Details
[ ] Metrics
[ ] Offer Details
[ ] Offers for Account
[ ] Offers
[ ] All Operations
[ ] Operations for Account
[ ] Operations for Ledger
[ ] Operations for Transaction
[ ] Operation Details
[ ] Orderbook Details
[ ] Strict Receive Payment Paths
[ ] Strict Send Payment Paths
[ ] Find Payment Paths
[ ] All Payments
[ ] Payments for Account
[ ] Payments for Ledger
[ ] Payments for Transaction
[ ] Trade Aggregations
[ ] Trades for Account
[ ] Trades for Offer
[ ] Trades
[ ] All Transactions
[ ] Post Transaction
[ ] Transactions for Account
[ ] Transactions for Ledger
[ ] Transaction Details

I think that adding top level endpoints (that are not deprecated) to the root resource is a no-brainer for self discoverability/hyperclient.
The nested endpoints are not show stoppers as they're linked from the subresources (I haven't actually gone through all the subresources to verify this). Though it's a bit silly that you have to go through more http calls than required just to collect links.

Thanks for the clarification @tomerweller . I will make sure that:

All the top-level links are present in the root resource
Nested endpoint links are reachable from subresources

I am running into a couple of problems:

AFAIU, HAL doesn't distinguish  between GET and POST requests (see this Stackoverflow question for more details), which makes it impossible to represent both GET /transactions and POST /transactions. @JakeUrban can you shed some light on how Hyperclient handles this? It may be enough to simply have separate entries for them, but I doubt it will be able to distinguish between GET and POST.

AFAICT, Horizon doesn't seem to do proper content negotiation for application/hal+json (e.g. when doing curl -H 'Accept: application/hal+json' localhost:8000/transactions horizon answers the endpoint normally instead of responding with the hal+json metadata. This would require fixing if we want to continue with the top-level+nested resources approach. Albeit, @tomerweller mentioned that The nested endpoints are not show stoppers as they're linked from the subresources, maybe I am missing something? Alternatively we could simply put all the links at the top level.

Regardless, I am going to create a PR with what I have now to move things forward.

@2opremio POSTing to /transactions doesn't seem to be an issue for the ruby client:
@horizon.transactions._post(tx: envelope_base64)

GET requests would look slightly different:
@horizon.transactions().get_

I don't think you need to represent POST requests. HAL only specifies the request URL paramet

I think we can close this issue since the changes have been merged. When will this be deployed?

@JakeUrban are you sure that the merged changes are enough? Note that most (if not all) the nested endpoints are not yet accessible.

I think we can close this issue since the changes have been merged. When will this be deployed?

With Horizon 1.1.0 (planned for April 6, see https://github.com/stellar/go/milestones )

@2opremio Leave endpoints (those that don't have subresources) don't need HAL since there are no more nested resources.
Do you mean the top-level resources don't have HAL describing their subresources? /accounts seems to have it.

Ahhhh, I was (wrongly) assuming that you had to be able to query HAL metadata without making an actual query (hence my confusion about content negotiation). I will review the endpoints again and reopen if needed.

@2opremio thanks for helping me get horizon set up locally. After taking a look and playing around with the client it looks good for the most part, but I found one issue:
The data in ledger should be moved to ledgers, and ledger should contain a href of http://localhost:8000/ledgers/{sequence}. This is what it is currently:
    "ledger": {
      "href": "http://localhost:8000/ledgers{?cursor,limit,order}",
      "templated": true
    },
    "ledgers": {
      "href": ""
    },

Good catch. Unfortunately this won't be fixed for horizon 1.1.0  since it's already frozen (CC @ire-and-curses )

@2opremio we're still in testing, if you want to push a bug fix for this I'm ok with it (it's low risk).

@JakeUrban it should be fixed now. Thanks for noticing!


Introduce an option to load source files in a convenient and easy way.

Addressed with PR  #169.

Done with #169.


Website refresh

Sadly the project's website has not been updated in ages 😢
I would be nice to at least update the information on it, or better completely refresh it. If someone wants to tackle that or have some ideas, feel free to comment here.

Hey @sinedied ,
I would like to contribute to this issue. Although, I am new to open source but I have good experience in Angular and web page designing as well.
Let me know if we can discuss on the same.
Thanks :)

Hi @codeNitesh that would be awesome!
Next week I'll be busy all week at a conference, but we can find some time that if that works for you?

sure @sinedied ,
I will also drop a reminder message here after next week.

Hey @sinedied ,
Let me know if we can connect anytime soon :)

Hi @codeNitesh , thanks for the reminder!
I'm sorry I did not come back sooner, I'm a bit crawling under notifications this week. Could you send me an email or PM at https://twitter.com/sinedied so we can discuss this further?


Added case Converter (Issue No :#69)

Fixes #(issue)
Description
Describe your changes in detail
I have successfully added a lowercase and uppercase button to the dino extension
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.

SWOC Participation

Name: MRIDUL MITTAL
Discord Username: MRIDUL MITTAL #8680
[x] I have included the issue number in my PR

If we are contributing in this project,
What did I submit on devfolio projects swoc?
They said all have to submit the project.

If we are contributing in this project, What did I submit on devfolio projects swoc? They said all have to submit the project.

You just have to mention the contributions you are making and submit it as a project. I do not have access to Devfolio so not sure about the process, but you can check the main channels to get more info.


Update sage.yml

Update example to follow existing example in Trellis config

🤔 this is inconsistent with https://github.com/roots/setup-trellis-cli/blob/main/examples/multi-repo.yml now

Super good point. Happy to add simple comments to the top of the other two examples!

Yeah adding the comment header + calling out what needs to be replaced would be helpful on eall 3

Thanks 🎉


Automated cordova builds and mobile improvements

This pull request makes some improvements to the user experience on mobile:

back button properly closes modals
better long touch support for multiple selections
dark/light mode detection on cordova

A new github workflow cordova-build-and-publish.yml pulls an environment from amplify and builds cordova installers. It publishes an Android release to Google Play.
Triggers on v1.* tags. cordova/config.xml must be edited with a version number than matches the tag.

Codecov Report

Merging #90 (f44df80) into master (ec86ad1) will increase coverage by 0.01%.
The diff coverage is 56.56%.

@@            Coverage Diff             @@
##           master      #90      +/-   ##
==========================================
+ Coverage   57.20%   57.21%   +0.01%
==========================================
  Files         290      294       +4
  Lines        9558     9654      +96
  Branches     2396     2417      +21
==========================================
+ Hits         5468     5524      +56
- Misses       3705     3742      +37
- Partials      385      388       +3

Impacted Files
Coverage Δ

src/hooks/useCurrentTheme.js
0.00% <0.00%> (ø)

src/index.js
0.00% <0.00%> (ø)

src/navigation/MobileNavigationDrawer.js
0.00% <0.00%> (ø)

src/redux/Reducers.js
70.09% <ø> (ø)

src/hooks/useLongPressEventContextMenu.ts
43.75% <43.75%> (ø)

src/hooks/useCordovaBackButton.ts
75.00% <75.00%> (ø)

src/redux/modalTracker/modalTrackerActions.js
77.77% <77.77%> (ø)

src/redux/modalTracker/modalTrackerReducers.js
80.00% <80.00%> (ø)

src/components/ConfirmationDialog.js
82.75% <100.00%> (+2.75%)
:arrow_up:

src/navigation/Components/UserFeedbackDialog.js
97.29% <100.00%> (+0.07%)
:arrow_up:

... and 5 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


crash when removing node_modules directory

Atom crashed when tests/node_modules directory was removed. (All my test-related packages live in a tests/ subdirectory with a separate package.json file.)
Backtrace:
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/share/atom/atom --type=renderer --enable-ex'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fd9f1a3dfe5 in InotifyTree::InotifyNode::removeChild(std::string) () from<EMAIL_ADDRESS>[Current thread is 1 (Thread 0x7fd9f1a30700 (LWP 549))]
(gdb) bt full
#0  0x00007fd9f1a3dfe5 in InotifyTree::InotifyNode::removeChild(std::string) () from<EMAIL_ADDRESS>No symbol table info available.
#1  0x00007fd9f1a3df38 in InotifyTree::removeDirectory(int) () from<EMAIL_ADDRESS>No symbol table info available.
#2  0x00007fd9f1a3c672 in InotifyEventLoop::work() () from<EMAIL_ADDRESS>No symbol table info available.
#3  0x00007fd9f1a3cbe9 in InotifyEventLoop::InotifyEventLoop(int, InotifyService*)::$_0::__invoke(void*) () from<EMAIL_ADDRESS>No symbol table info available.
#4  0x00007fda0acacfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140574038624000,<PHONE_NUMBER>148162756,<PHONE_NUMBER>13166,<PHONE_NUMBER>13167,<PHONE_NUMBER>24000,<PHONE_NUMBER>5936, -1899541000536249148, -1899235541725139772},
              mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#5  0x00007fda082f082f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

atom -v
Atom    : 1.35.1
Electron: 2.0.18
Chrome  : 61.0.3163.100
Node    : 8.9.3

Debian Sid

Thanks for the report!  Can you fill out the issue template information?  The format and information in the template is super helpful for us when triaging issues (for example, can you reproduce in safe mode, do you have exact step-by-step instructions to reproduce, etc).
Also, do you have a minimal example project that we can use to reproduce?

We require the template to be filled out on all new issues and pull requests. We do this so that we can be certain we have all the information we need to address your submission efficiently. This allows the maintainers to spend more time fixing bugs, implementing enhancements, and reviewing and merging pull requests.
Thanks for understanding and meeting us half way :grinning:

I have exactly the same problem on:
Linux OpenSuse Tumbleweed
Atom ... 1.34, 1.35, 1.36
Steps to reproduce:

Open Any JavaScript project with many dependencies in node_modules folder in Atom
Delete node_modules from the command line : rm -erf node_modules

Result:
Atom crashes.

Related to #16911


[Feature][server & ui] Add dependent tips about success or failure

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description
Add success and failure prompts in the ui and log, just like the old version 1.3.x

Use case
above
Related issues
No response
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I have the same problem.This function is very important to me.Can this be added to 2.0.2-release?@lenboo


Increased memory usage on google cloud functions on Node 8?

Hi,
After upgrading from Node 6 to 8 (Beta support in google cloud functions atm) I noticed memory increase in memory usage. I have a function that overlays 10-20 images using read and write streams.
Heres mem drop after downgrading back to Node 6 engine:

This is not a bug report, more like a question that is there something I need to be aware of or anything I could do to decrease mem usage? Of course bug also could be on Google's side.

My best guess would be that the change in runtime from Node 6 to 8 also involved OS-level changes, perhaps relating to memory allocator configuration. The Node 6 runtime is "based on Debian 8" whereas the Node 8 runtime is "based on Ubuntu 18.04" (which is based on Debian 10).
If the "memory usage" graph shows peak RSS then remember this includes free memory that has yet to be returned to the OS. It looks like you've provisioned 2GB. What happens if you provision the functions to only have access to 1GB?
Perhaps with the Node 8 runtime Google have tuned the OS to not demand free memory is returned, or tuned the memory allocator to not return free memory in an attempt to improve (perceived) performance?

@ovaris Were you able to make any progress with this, perhaps with information from Google?

Sorry @lovell I have not actually, been to busy and just stuck with Node 6 version since it seems to be running with lower memory. Getting info from Google is painfully slow process unfortunately. This function started to act weird after upgrading to Node 8 ( it only saved partial data into real time database and leaving some fields nil), but I really don't know if it was due to function running on memory limits, or was it because of some other problem in beta Node 8 runtime.

@ovaris Thanks for the update. I'd be happy to alert some Google people to this issue if you have the time to continue to diagnose the problem with them.

Yeah sure, please give heads up for Google people. We need to upgrade anyways eventually to Node 8

If possible, please can you upgrade sharp to the new v0.21.0 version first and confirm this remains a problem.

Sorry for the delay, I'm upgrading right now to v0.21.0 (need to go through the code to fix deprecated calls), then my plan is to deploy to production. First I will see how updated lib performs with Node 6, then I can switch engine to 8 from cloud console and see what happens.

Looking good so far! Updated at where blue dot is (22:30):

Hey, looks like this has been now fixed, either a) by updating Sharp or b) They've fixed something in cloud functions, env. Memory usage after upgrade (blue dot):

@lovell I'm not sure but I think I'm seeing increased memory usage after all, which I didn't see with Node 6, I can see even some 2GB peaks, here's 30 day graph,

Here's the drop after downgrading:

@lovell @DavidKuennen @ovaris same problem for me too i config vm with 1gb memory and using sharp 0.21.3 when i processing image one at at time i saw huge increase in memeory and after that
-bash: fork: Cannot allocate memory
<--- JS stacktrace ---> Cannot get stack trace in GC. FATAL ERROR: NewSpace::Rebalance Allocation failed - process out of memory  1: node::Abort() [node]  2: 0x8cd14c [node]  3: v8::Utils::ReportOOMFailure(char const*, bool) [node]  4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]  5: 0xa6717b [node]  6: v8::internal::MarkCompactCollector::Evacuate() [node]  7: v8::internal::MarkCompactCollector::CollectGarbage() [node]  8: v8::internal::Heap::MarkCompact() [node]  9: v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node] 10: v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node] 11: v8::internal::Factory::NewMap(v8::internal::InstanceType, int, v8::internal::ElementsKind) [node] 12: v8::internal::Map::RawCopy(v8::internal::Handle<v8::internal::Map>, int) [node] 13: v8::internal::Map::CopyDropDescriptors(v8::internal::Handle<v8::internal::Map>) [node] 14: v8::internal::Map::ShareDescriptor(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::DescriptorArray>, v8::internal::Descriptor*) [node] 15: v8::internal::Map::CopyAddDescriptor(v8::internal::Handle<v8::internal::Map>, v8::internal::Descriptor*, v8::internal::TransitionFlag) [node] 16: v8::internal::Map::CopyWithField(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::FieldType>, v8::internal::PropertyAttributes, v8::internal::PropertyConstness, v8::internal::Representation, v8::internal::TransitionFlag) [node] 17: v8::internal::Map::TransitionToDataProperty(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::PropertyConstness, v8::internal::Object::StoreFromKeyed, bool*) [node] 18: v8::internal::LookupIterator::PrepareTransitionToDataProperty(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [node] 19: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node] 20: v8::internal::JSObject::DefineOwnPropertyIgnoreAttributes(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::JSObject::AccessorInfoHandling) [node] 21: v8::internal::JSObject::DefinePropertyOrElementIgnoreAttributes(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes) [node] 22: v8::internal::JsonParser<true>::ParseJsonObject() [node] 23: v8::internal::JsonParser<true>::ParseJsonValue() [node] 24: v8::internal::JsonParser<true>::ParseJson() [node] 25: v8::internal::Builtin_JsonParse(int, v8::internal::Object**, v8::internal::Isolate*) [node] 26: 0x380ccc98697d Aborted (core dumped)


Implemented History

This is an implementation of issue #5 with a new page for history.

good to go


Replace image  container element to use  which is the right element

I had never seen  so it's quite a surprise that it exists.
This is however the right element to use when dealing with images.

Duplicate #100


WFLY-497 Infinispan thread pool injection

Jira
https://issues.jboss.org/browse/WFLY-497

Windows Build 1332 is now running using a merge of 7004eb6416b6ed1f501cc7954bb0c5fd64d40384

Linux with security manager Build 1195 is now running using a merge of 7004eb6416b6ed1f501cc7954bb0c5fd64d40384

Linux Build 6220 is now running using a merge of 7004eb6416b6ed1f501cc7954bb0c5fd64d40384

Linux with security manager Build 1195 outcome was SUCCESS using a merge of 7004eb6416b6ed1f501cc7954bb0c5fd64d40384
Summary: Tests passed: 822, ignored: 356 Build time: 0:05:54

Linux Build 6220 outcome was FAILURE using a merge of 7004eb6416b6ed1f501cc7954bb0c5fd64d40384
Summary: Tests failed: 1 (1 new), passed: 2840, ignored: 416, muted: 1 Build time: 0:40:40
Build problems:
Failed tests detected
Failed tests
org.jboss.as.test.clustering.cluster.web.passivation.FineSessionPassivationTestCase(SYNC-tcp).test: <no details avaliable>

Windows Build 1332 outcome was FAILURE using a merge of 7004eb6416b6ed1f501cc7954bb0c5fd64d40384
Summary: Tests failed: 1 (1 new), passed: 2841, ignored: 416 Build time: 0:49:09
Build problems:
Failed tests detected
Failed tests
org.jboss.as.test.clustering.cluster.web.passivation.FineSessionPassivationTestCase(SYNC-tcp).test: <no details avaliable>

Linux with security manager Build 1221 is now running using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63

Linux Build 6255 is now running using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63

Windows Build 1358 is now running using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63

Linux with security manager Build 1221 outcome was SUCCESS using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63
Summary: Tests passed: 822, ignored: 356 Build time: 0:05:42

Windows Build 1358 outcome was SUCCESS using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63
Summary: Tests passed: 2842, ignored: 416 Build time: 0:49:52

Linux Build 6255 outcome was FAILURE using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63
Summary: Execution timeout (new); tests failed: 102 (97 new), passed: 2544, ignored: 404, muted: 1; org.wildfly:wildfly-ts-integ-manualmode Build time: 3:00:19
Build problems:
Failed tests detected
Execution timeout
Process exited with code 137
Failed tests
org.jboss.as.test.clustering.cluster.ejb.stateful.StatefulFailoverTestCase(SYNC-tcp).jmsConnectionFactoryFailover: <no details avaliable>

org.jboss.as.test.clustering.cluster.ejb.stateful.StatefulFailoverTestCase(SYNC-tcp).noFailover: <no details avaliable>

org.jboss.as.test.clustering.cluster.ejb.stateful.StatefulFailoverTestCase(SYNC-tcp).simpleFailover: <no details avaliable>

org.jboss.as.test.clustering.cluster.ejb.stateful.StatefulTimeoutTestCase(SYNC-tcp).timeout: <no details avaliable>

org.jboss.as.test.clustering.cluster.ejb.xpc.StatefulWithXPCFailoverTestCase(SYNC-tcp).testBasicXPC: <no details avaliable>

org.jboss.as.test.clustering.cluster.ejb.xpc.StatefulWithXPCFailoverTestCase(SYNC-tcp).testSecondLevelCache: <no details avaliable>

org.jboss.as.test.clustering.messaging.ClusteredMessagingTestCase.testClusteredQueue: <no details avaliable>

org.jboss.as.test.clustering.messaging.ClusteredMessagingTestCase.testClusteredTopic: <no details avaliable>

org.jboss.as.test.clustering.xsite.XSiteSimpleTestCase.testPutToBackupNotRelayed: <no details avaliable>

org.jboss.as.test.multinode.remotecall.scoped.context.DynamicJNDIContextEJBInvocationTestCase: org.jboss.as.test.multinode.ejb.timer.database.DatabaseTimerServiceMultiNodeExecutionDisabledTestCase: <no details avaliable>

org.jboss.as.test.multinode.remotecall.scoped.context.DynamicJNDIContextEJBInvocationTestCase: org.jboss.as.test.multinode.ejb.timer.database.DatabaseTimerServiceMultiNodeTestCase: <no details avaliable>

##### there are 92 more failed tests, see build details

Retest this please.

Windows Build 1360 is now running using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63

Linux Build 6258 is now running using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63

Linux with security manager Build 1223 is now running using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63

Linux with security manager Build 1223 outcome was SUCCESS using a merge of 3ac7f7bc1849b0be6c52fd3d5e7a80649da01e63
Summary: Tests passed: 823, ignored: 356 Build time: 0:06:12

Linux Build 6260 is now running using a merge of b1c4c871e3874c09b369683eb8e5c8e4bec44f0e

Windows Build 1362 is now running using a merge of b1c4c871e3874c09b369683eb8e5c8e4bec44f0e

Linux with security manager Build 1225 is now running using a merge of b1c4c871e3874c09b369683eb8e5c8e4bec44f0e

Linux with security manager Build 1225 outcome was SUCCESS using a merge of b1c4c871e3874c09b369683eb8e5c8e4bec44f0e
Summary: Tests passed: 823, ignored: 356 Build time: 0:05:04

Linux Build 6260 outcome was SUCCESS using a merge of b1c4c871e3874c09b369683eb8e5c8e4bec44f0e
Summary: Tests passed: 2846, ignored: 412, muted: 1 Build time: 0:50:20

Windows Build 1362 outcome was SUCCESS using a merge of b1c4c871e3874c09b369683eb8e5c8e4bec44f0e
Summary: Tests passed: 2847, ignored: 412 Build time: 0:53:52

Linux with security manager Build 1259 is now running using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c

Windows Build 1401 is now running using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c

Linux Build 6294 is now running using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c

Windows Build 1401 outcome was FAILURE using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c
Summary: Cannot start build runner Build time: 0:00:32
Build problems:
Unable to find java executable at: C:\BuildAgent\work\a31d203e70e89f90\C:Javajdk1.8.0_45\bin\java.exe

Linux with security manager Build 1259 outcome was SUCCESS using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c
Summary: Tests passed: 823, ignored: 356 Build time: 0:04:54

Linux Build 6294 outcome was FAILURE using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c
Summary: Tests failed: 4 (4 new), passed: 2843, ignored: 412 Build time: 0:46:59
Build problems:
Failed tests detected
Failed tests
org.jboss.as.test.integration.ejb.remote.distinctname.JarDeploymentDistinctNameTestCase.testRemoteSLSBInvocation: <no details avaliable>

org.jboss.as.test.integration.ejb.remote.distinctname.JarDeploymentDistinctNameTestCase.testRemoteSingletonInvocation: <no details avaliable>

org.jboss.as.test.integration.ejb.stateful.undeploy.RemoveSFSBOnUndeployTestCase.testSfsbDestroyedOnUndeploy: <no details avaliable>

org.jboss.as.test.clustering.cluster.ejb.remote.RemoteFailoverTestCase(SYNC-tcp).testStatefulFailover: <no details avaliable>

Retest this please.

Linux Build 6328 is now running using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c

Windows Build 1420 is now running using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c

Linux with security manager Build 1270 is now running using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c

Windows Build 1420 outcome was FAILURE using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c
Summary: Cannot start build runner Build time: 0:00:12
Build problems:
Unable to find java executable at: C:\BuildAgent\work\a31d203e70e89f90\C:Javajdk1.8.0_45\bin\java.exe

Linux with security manager Build 1270 outcome was SUCCESS using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c
Summary: Tests passed: 824, ignored: 356 Build time: 0:05:24

Linux Build 6328 outcome was SUCCESS using a merge of 20f5c9142590b4eb2c43baaa1b84e7aee9a2738c
Summary: Tests passed: 2857, ignored: 412 Build time: 0:50:05

I am going to close this now since it got moved to 10.x and make more changes.


ui doesn't load with IE 11

java -jar zipkin-server-2.3.2-20171127.080232-13-exec.jar
loading http://localhost:9411/zipkin/ gives a blank page
IE version:

this is the loaded dom:

fixed via #1825


Get the logger to respect module hierarchy as a normal logger

Feature request to have this logger have the same multiple module inheritance present on the normal logging module.
When we create a normal log on the main.py, then subsequence invocations of logging.getLogger(__name__) on submodule will inherit the handlers and levels set on the main.py, due to the singleton nature of the logging.Logger class , hence using the logging.getLogger(name) to get the correct instance of the logger.
This allows for any log message in sub-modules to have the same characteristics a the parent logger. In this specific case, since this formatter transforms everything into a nice JSON with the possibility of adding structure logs, would be nice for those structure logs to persist and be inherited by the child logging instances.
If I need to create another logger = Logger() this will indeed create a whole new instance which has no relationship with the logger on main.py, hence all the structure logs setup on main are now lost and need to be recreated, besides being repetitive, it might not be possible.
For example, these two logs, one from the lambda handler, another one from a submodule. The runID was lost, since it was added via logger.structure_logs(append=True, runID=run_id)
{"timestamp": "2020-07-31 06:29:32,699", "level": "INFO", "location": "lambda_handler:46", "service": "myService", "sampling_rate": 0.0, "cold_start": true, "function_name": "name, "function_memory_size": "512", "runID": "1b250a6732", "message": "my message on main.py"}

{"timestamp": "2020-07-31 06:29:33,161", "level": "INFO", "location": "start_execution:49", "service": "core.handlers.handler", "sampling_rate": 0.0, "message": "my message on a sub module"}

It would be ideal that some basic keys would be inherited, such that querying the logs afterwards can be easy, such as getting all logs where runID = 'abc'.
Re setup the structure_logs on every module is not possible, since modules might be agnostic to the concept of runID

Hey @alexanderluiscampino - Thanks for opening a feature request!
Background
As you pointed out, Logger doesn't follow a compliant hierarchy because we don't use getLogger under the hood - It's not a singleton per se as we return different instances.
We subclass logging.Logger and initialize a Logger using the service name - This has the pros of us being able to return you a simplified Logger with additional methods, but it has the cons that I discovered recently that a Logger is only truly registered when using getLogger.
I've tried a few different approaches like setLoggerClass, an additional property (self.logger) and method to return a registered Logger, etc. I've had issues with all of them either on the UX we wanted to provide, or having to override how Logging captures the stack frames to correctly identify where a log statement was called.
Ideas
If you have ideas on how we could comply with hierarchy in logging while maintaining backwards compatibility in the UX, please I'd love if you could craft a RFC - This is something I want to improve but am focusing on the utility areas atm.
That said, if we can't find a solution to meets this criteria ^, we can solve this problem differently using something like Borg to share configuration between instances, while additional instances can have additional or override existing configuration -- We do this in Tracer today.
:)

I just had an idea on how to tackle this while keeping the same UX and respecting hierarchy in a nicer way - It'll involve:

Overriding findCaller from the CPython Logging implementation
Include a self.logger = getLogger... as well as info, error, debug, etc. methods
Capturing upper frames to identify where the log statement was actually executed
Create a logger name based on source frame (ala getLogger(__name__))

I'll thinker on that on Thursday or Friday.
Still happy to hear additional ideas here

Hey @heitorlessa , I'm on vacation this week, but willing to help however I can next coming week. I have created a similar version for the logger for my projects that pretty much is what you described above. Its just a class which does not inherit from the logging.Logger, allowing the normal behavior to proceed.
I do a self.logger = logging.getLogger() and apply the JSON formatter you created.  Something of the sort:
class LambdaLogger(LambdaDecorator):
    """
    General AWS Lambda Loggers Utility Tool
    Handles setting a JSON lambda formatter for logging to cloudwatch,
    utilizing lambda root logger.
    JSON format on logs its useful due to its easy indexation capabilities
    via the query language in Cloudwatch Insights.

    This log is meant to be used instead of the normal logging module,
    since it only changes its format and adds methods to update the
    JSON keys outputted.

    Usage example:
        >> lambda_logger = LambdaLogger(service='myService', level='warning', app='myApp')
        >> logger = lambda_logger.get_logger()
        >> @lambda_logger.log_everything
        >> def lambda_handler(event, context):
        >>    lambda_logger.update_attributes(runID='3cnu8930a48')
        >>    logger.info("My First Log")

    """

    def __init__(self, level=logging.INFO, **kwargs):
        """
        Create instance of LambdaLogger

        Args:
            level ([str, int, logging.Level]): Logging level to apply. Defaults to logging.INFO.
        """
        self.logger = logging.getLogger()
        if isinstance(level, str):
            level = level.upper()
        level = level or os.getenv("LOG_LEVEL").upper()
        self.logger.setLevel(level)
        self.formatter = LambdaFormatter(**kwargs)
        self.apply_formatter()

    def get_logger(self):
        return self.logger

    @property
    def log_everything(self):
        """
        Decorators for lambda handler to log event trigger, context and return value
        """
        return LogEverything

    def update_attributes(self, **kwargs):
        """
        Updates list of log JSON key attributes
        """
        self.formatter.add_log_attributes(**kwargs)
        self.apply_formatter()

    def apply_formatter(self):
        """
        Applies formatter to all handlers available
        """
        for handler in self.logger.handlers:
            handler.setFormatter(self.formatter)

I am sure there are things i'm missing out with this approach in comparison to yours.

After some research I think I found the solution while keeping backward compatibility, performance, and UX the same.
Challenge I had with implementing this safely was that the stack frame of the caller (who issued: logger.info/debug) was pointing at the Logger class, not the caller itself.
There was a lot of code to change back then to make this work which I felt it was unnecessary, as I didn't have any customer feedback on that yet to justify.
Luckily 'stacklevel' logging parameter was introduced in CPython 3.8 to solve this problem, and keep our "location" key intact -- I just need to backport this to support 3.6 and 3.7:
https://github.com/python/cpython/pull/7424/files
I'll give this a try on the weekend
Thanks for the patience so far :)

hey @alexanderluiscampino - When you get a chance, could you please have a look at the PR linked in this issue?
I've managed to find a more elegant solution without compromising UX, backwards compatibility, and the dreaded location with less code.
I'll be writing the docs next if that works for you.
Thanks!

@heitorlessa , I'm in the process of reviewing it!

Awesome! I’ve updated the UX section to show how the experience looks like
without breaking existing customers
On Tue, 11 Aug 2020 at 17:47, Alexandre Campino<EMAIL_ADDRESS>wrote:

@heitorlessa https://github.com/heitorlessa , I'm in the process of
reviewing it!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/awslabs/aws-lambda-powertools-python/issues/97#issuecomment-672028842,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZPQBHLSF6RPZPJIOAUNRLSAFRZ3ANCNFSM4PPSXNXQ
.

@alexanderluiscampino This is now released under 1.1.0 version in PyPi 🥳 🎉
Docs: https://awslabs.github.io/aws-lambda-powertools-python/core/logger/#reusing-logger-across-your-code
Thank you once again for the suggestions and reviewing ;)

@alexanderluiscampino This is now released under 1.1.0 version in PyPi 🥳 🎉
Docs: https://awslabs.github.io/aws-lambda-powertools-python/core/logger/#reusing-logger-across-your-code
Thank you once again for the suggestions and reviewing ;)

Link is not working; here is the fixed link.


Fix DictDataset to work in Python 3.

Fix #3236.

Jenkins, test this please.

Hey Jenkins, test this please.

@mitmul Can I include a test for TupleDataset which checks it works with slice (like this codes) in this PR?
Should I make another PR for that?

@bonprosoft I think adding the test to this PR is OK. Thanks!

@mitmul I added the test for TupleDataset. Please check it. Thank you.

Jenkins, test this please

Jenkins, test this please

Jenkins...Jenkins... test this please

LGTM


Release version 0.0.16

Codecov Report
Patch and project coverage have no change.

Comparison is base (0e5313f) 100.00% compared to head (6eae91a) 100.00%.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #293   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          316       316
  Lines         9904      9904
  Branches       961       961
=========================================
  Hits          9904      9904

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[OSF-7264] Test and fix some of citeproc-py wrong citations.

Purpose
Improve the accuracy of citations provided by the api, and create a testing suite to use for future improvements.
Changes
Addition of a json file populated with citations made using citeproc-js, a test suite comparing citeproc-py to the test file, and some small improvements to our render citation.
With the improvements, 368 citation types differ between our APi and the project page, down from 700+
Ticket
https://openscience.atlassian.net/browse/OSF-7264

@hmoco Why didn't you just use this for https://openscience.atlassian.net/browse/OSF-7479

@brianjgeiger thats pretty much what I ended up doing, but without the tests. I guess if the tests are also desired, I can solve merge conflicts and link this to the ticket instead

Ah, merge conflicts. Ugh. No, just do a demo for product.

@hmoco Flake8: ./admin/base/settings/defaults.py:9:40: W291 trailing whitespace

@brianjgeiger, taking a look at this, not sure how it's failing flake. Passes it locally and that path points to a file that was not touched by this pr. Any ideas?

Maybe develop just had a bad patch right when those commits were pushed? When travis works again we'll restart the tests and see what happens.


bundle.generate returns a promise

Not really sure you should merge this as I didn't test if this really fixes things. Making this change in the installed version in my project did seem to eliminate the error:
ERROR [preprocessor.rollup]: bundle.generate(...) now returns a Promise instead of a { code, map } object

Also, the plugin seems to no longer be maintained. Do you know of any alternatives?
Cheers!

After some more fiddling, I can confirm this fixes the build issue I was having.


strangely, does not work with sudo

I'm having a very strange issue here, hopefully someone can shed some light on the subject.
go 1.14, raspberry pi 4 B+
using the blinker.go example:
./blinker works just fine
sudo ./blinker makes nothing happen.
Why is this?  Normally that would be fine, except that I'm trying to use rpio along with a dht11 library that requires root access to run.  I find it perplexing that running rpio with heightened privileges causes it to not work.

Same here - had to switch from root user to pi user to get blinker.go working. Not sure what is the reason for this.

Same here - had to switch from root user to pi user to get blinker.go working. Not sure what is the reason for this.

I also encountered the same problem.

This might be some OS-related change.
You can try to change this part of code:
https://github.com/stianeikeland/go-rpio/blob/master/rpio.go#L630-L637
So that /dev/gpiomem is always used (but only basic gpio functionality will work)

Also #50 this might solve your issue, can you please try?

Same issue here... can not get it to work on root on and RPI4, go 1.15.3
Only trying to control a single pin at the moment, program works with pi

related https://github.com/stianeikeland/go-rpio/issues/68 [fixed]

Should be fixed now - thanks to #70 by @rami-dabain.
Feel free to reopen or create a new issue in case of a problem.


Upgrade to<EMAIL_ADDRESS>

Fixes #750, fixes #651.

The latest version of Docusaurus requires to be on Node >= 8.x and Yarn >= 1.5.

Nice work! Hopefully they release V2 soon as well :)


What to do with ambiguous results?

We have lints that that may apply or that may pass/fail, however the runtime does not have quite enough information to prove it.
An example is in #491 wherein a certificate would fail if-and-only-if one of its parents has the TLS auth EKU. We have a discussion in #428 on enabling operators to include chains into runs of the tool in order to disambiguate such cases. However, what if the operator does not include the chain? How do we let them know that there is a lint that might benefit from seeing the full chain (as simply hiding it in the docs is a great way for these sorts of lints to go unnoticed)?
And certificate chains aside, I am sure that there will be (are?) additional lints that are otherwise ambiguous (perhaps they need access to some CSV or database that's not always going to be there, for example certificate transparency records).
Some Questions

How "loud" are these notices of ambiguous runs? Is an ambiguous Error just as loud as a proven Error? That seems noisy, however Errors are tied to MUSTS - if I were banking on this tool for my job I would personally appreciate it being this strict. Maybe --strict/--nostrict type flag (let's say that by default an unproven Error results in a non-zero exit code and you can disable that behavior via --nostrict)?
Ambiguous Warnings, however? At first blush, these actually seem even less important than an actually Warning.

Another scenario where the ambiguity is difficult is anywhere in the BRs where compliant participants MUST have a process for X or so forth, aka areas where no additional context would be able to help zlint provide a more concrete determination.
For context, that was the issue leading to #493 in which the BRs state (<IP_ADDRESS>):
the CA MUST establish and follow a documented procedure[^pubsuffix] that determines if the wildcard character occurs in the first label position to the left of a “registry‐controlled” label or “public suffix”

@cardonator So for those lints, I'm taking it as you'd support a CookieMonster level, because just like cookies are a sometime food, these are certificates that are sometimes good? 😜

@sleevi haha! I was thinking it's more like classic CookieMonster where you get all the cookies you want unless you're being a bad boy. 😉

Jokes aside, on a pragmatic level, I would think that for situations like that - where something is bad UNLESS the CA has taken steps X/Y/Z (such as a documented process) - it probably makes sense to assume the negative case entirely, and then allow the CA to suppress certain lints on a case-by-case basis (e.g. if they have a documented process).
The downside to this approach is the common downside we see throughout the CABF, which is that if the requirements change in, say, v1.x.y to 1.y.z, and we reuse the same lint, the CA may have disabled the lint for the new requirement. However, since a new requirement generally means a new effective date, and a new effective date generally means a new lint, I suspect the liklihood is that a new lint for the new requirement(s) would be introduced, and the CA would squelch the new lint if it doesn't apply.

I like that approach, it's kind of like an attestation for specific requirements that are Error scenarios without an attestation.

Exactly. Whether that attestation is implicit (i.e. the CA disabled the lint) or explicit (e.g. a "ZLint config" which can contain all of the relevant attestations a CA is making) I'm somewhat ambivalent to. I suspect that either way, there will be plenty of ways for things to go wrong, so I lean on the KISS and leave the work up to the CA to manage those attestations themselves, without any overly complex guardrails.


[4.0][Bug] isColumnNullable throws error on column not found

Bug report
What I did
Added a field to crud and setup a mutator to manipulate the field and insert data in other table.
What I expected to happen
My mutator runs smooth! :+1:
What happened
isColumnNullable() is throwing error when column does not exist in database.
What I've already tried to fix it
Hey there guys. Dunno if i am misunderstand something, but isColumnNullable() should not throw error if column does not exist in database right ? I could add a field to crud and don't have a field in this table to store it ? Nor in any table, i could just want to setup some cron job or send some email or wtv.

    public static function isColumnNullable($column_name)
    {
        // create an instance of the model to be able to get the table name
        $instance = new static();

        $conn = DB::connection($instance->getConnectionName());
        $table = Config::get('database.connections.'.Config::get('database.default').'.prefix').$instance->getTable();

        // MongoDB columns are alway nullable
        if ($conn->getConfig()['driver'] === 'mongodb') {
            return true;
        }

        // register the enum, json and jsonb column type, because Doctrine doesn't support it
        $conn->getDoctrineSchemaManager()->getDatabasePlatform()->registerDoctrineTypeMapping('enum', 'string');
        $conn->getDoctrineSchemaManager()->getDatabasePlatform()->registerDoctrineTypeMapping('json', 'json_array');
        $conn->getDoctrineSchemaManager()->getDatabasePlatform()->registerDoctrineTypeMapping('jsonb', 'json_array');
        try {
            //check if column exists in database
            $conn->getDoctrineColumn($table, $column_name);
            return ! $conn->getDoctrineColumn($table, $column_name)->getNotnull();
        } catch (Exception $e) {
            return true;
        }
    }

So i changed this function to a try block. In case the try fails (column does not exist in database) we return true for nullable. We are not checking, nor we require that the column exists in database, that's a developer option right ?
Best,
Pedro

I agree! It should not throw an error - just fall back to "nullable" => true if no column is there.
Question: what if there's a mutator to a real database column? What would isColumnNullable() return then?

@tabacitu i think i don't really get your question but here is my answer from my understanding:
return ! $conn->getDoctrineColumn($table, $column_name)->getNotnull();

isColumnNullable does not care about about mutator or whatever. It's all about database column. isColumnNullable just checks if the column in DATABASE is nullable.
We are throwing error if it does not exists.
If it exists this function will return just that info, is nullable in database or not.
We are not accessing any ->attribute to fire mutators or whatever here. If there is a mutator it will fire upon attribute set/get, does not matter if column is nullable or not.
Best,
Pedro

Great! So it's just a matter of not throwing an error.
Can you please help put together a PR to fix this?
Cheers!


Changing the dataset

Hi Andre, I hope you’re doing well! I’d like to get your advice on something.  Are there specific things I should keep in mind, and would I need to resize the training images to a particular size? And what changes might be required in the final execution script?
Also what changes would be there in the flow if I use Tensorflow lite? And what version would be required?

Hello again Aryan!
If you would want to use a different dataset, I suppose you also want to use a different number of classes to detect. For example, you could use a dataset for face detection. If that is the case, you would need the following changes on the workflow:

On the model conversion to Tensorflow, you need to make sure you are using the YOLOv3 model but trained for the particular dataset and detection you want. For face detection you need to have the Darknet model or already the Tensorflow model with the parameters trained on the face detection dataset.
On the quantization part, you would need to repeat the process with the desired dataset, instead of the COCO dataset. The input image size for the YOLO accelerator remains the same (416x416) and the callibration script already resizes the images of your dataset for you. You would also need to confirm the input and output node names to then use them on the quantization script, but that is well described on the tutorial.
On the deployment part, you would only need to change the number of classes and the name of them on the c++ script. You would anso need to name the input and output nodes on the script but all of that is really easy.

Note: I think I have somewhere this project done for face detection for other student if you need it I can provide to you!
About the Tensorflow Lite part, I don't have a lot of knowledge about it. Assuming it has some tools to reduce the model size using quantization or prunning, I don't think it would be much help. For instance, DNNDK already quantizes the model so that the weights and activations use 8 bits (I can provide you the scientific paper about the DPU if you want). It is mandatory for it to use 8 bits because that is how the accelerator is configured.
DNNDK does what we call "quantization after training" which is less effective than "quantization aware training" which is probably what Tensorflow Lite is able to do.
Prunning the model could potentially help but would technically reduce the efficiency of the accelerator. Anyhow, it might help reduce the latency!
I would need more details about Tensorflow Lite to help you more on that. But in summary, if it can prunne the model parameters, it might help reduce the latency on the pynq-z2. Quantization should not be done.
I hope this helps in any way. If you need any help or the material I talked about on this post, make sure to tell me!
Good luck!

Okay, understood, I am myself exploring how to deploy tensorflow lite and
looking for ways to deploy it on the board. I was just experimenting with
the process and reviewing the scripts to see where adjustments might be
needed. Also, if you could, please share the face detection project you
mentioned, and send over the scientific paper on the DPU—just attaching the
link or paper name would be perfect.

Explore Tensorflow Lite and see if you can start by just prunning the existing model to see if that makes any difference. If such option does not exist or the Tensorflow Lite flow is more abstract, let it do the quantization normally and continue the process to deploy on the pynq-z2 to see if there us any difference in performance.
The article about the DPU is this one. They also relaesed a pattent for the DPU and this article is essencially the same information.
About the face detection project, It is available here. The compilation is already done, you can just modify the c++ file with the class name "face" (only one class) and the input and output nodes. If I manage to find the file I will send you. Sorry for that inconvenience,

Thank you very much! Could you do one last thing to help me? By any chance,
do you have the generated annotations file mentioned in your metrics
section 2: detections-on-the-pynq-z2 for Tiny YOLO and YOLO? I’d like to
compare it.
On Mon, Nov 11, 2024, 3:06 AM André Araújo @.***> wrote:

Explore Tensorflow Lite and see if you can start by just prunning the
existing model to see if that makes any difference. If such option does not
exist or the Tensorflow Lite flow is more abstract, let it do the
quantization normally and continue the process to deploy on the pynq-z2 to
see if there us any difference in performance.
The article about the DPU is this one
https://dl.acm.org/doi/10.1145/2847263.2847265. They also relaesed a
pattent for the DPU and this article is essencially the same information.
About the face detection project, It is available here
https://drive.google.com/file/d/1vkuZ81Lqq0QNTAsLgoH8qUJY2pP14rMv/view?usp=sharing.
The compilation is already done, you can just modify the c++ file with the
class name "face" (only one class) and the input and output nodes. If I
manage to find the file I will send you. Sorry for that inconvenience,
—
Reply to this email directly, view it on GitHub
https://github.com/andre1araujo/YOLO-on-PYNQ-Z2/issues/18#issuecomment-2466936815,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5RZEIWLCBGRZYF4CDKSLUTZ77GUDAVCNFSM6AAAAABRO6IUSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRWHEZTMOBRGU
.
You are receiving this because you authored the thread.Message ID:
@.***>

Yes yes, I can certainly help!
You can consult the annotations using this google drive link.
I am sorry for the delayed response but it took a lot of time to upload all those files.
I hope it helps!

Awesome, thank you so much Andre!
On Mon, Nov 11, 2024 at 7:00 PM André Araújo @.***>
wrote:

Yes yes, I can certainly help!
You can consult the annotations using this google drive link
https://drive.google.com/drive/u/2/folders/1GI-VVlpxcfqYh19qbeoHdnOPpmZWMiqn
.
I am sorry for the delayed response but it took a lot of time to upload
all those files.
I hope it helps!
—
Reply to this email directly, view it on GitHub
https://github.com/andre1araujo/YOLO-on-PYNQ-Z2/issues/18#issuecomment-2468185636,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5RZEISRUCTKKDWQVHC6YLT2ACWOFAVCNFSM6AAAAABRO6IUSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRYGE4DKNRTGY
.
You are receiving this because you authored the thread.Message ID:
@.***>

Hi Andre,
I’ve trained a YOLOv3 model on a different dataset, which is relatively
small, consisting of just 1000 images with 2 classes. The annotations for
this dataset are currently in .txt files (in the YOLO format), but I
noticed that your quantization process requires annotations in a .json
format, specifically following the COCO standard.
Yolo format : <class_id> <x_center> <y_center>
Could you advise on the best way to convert my .txt annotations to the
required .json format? If there are any tools or scripts that can help
streamline this process, I'd really appreciate your guidance. Thank you!

Message ID: @.***>

Hi, Aryan!
Sorry for the misunderstanding, but now that I reviewd the process I am 90% positive you don't acctually need the annotations for the quantization process. It just needs the images.
In any case, if you could test the quantization process to see if it works without the annotations, it would be much appreciated!
Let me know if you got it. Good luck!

Hey Andre, yes it doesn't need annotations i.e. the json file for
quantization, only images are needed. I have completed my quantization and
compilation for a custom dataset, I'll send you the files once I completely
deploy it on the board. I have used a custom fighter plane dataset, and
have trained for only 2 classes for now, which now I have successfully
quantized and compiled.
On Sat, Nov 16, 2024, 12:41 AM André Araújo @.***>
wrote:

Hi, Aryan!
Sorry for the misunderstanding, but now that I reviewd the process I am
90% positive you don't acctually need the annotations for the quantization
process. It just needs the images.
In any case, if you could test the quantization process to see if it works
without the annotations, it would be much appreciated!
Let me know if you got it. Good luck!
—
Reply to this email directly, view it on GitHub
https://github.com/andre1araujo/YOLO-on-PYNQ-Z2/issues/18#issuecomment-2479747525,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5RZEIS3XVJHRKLRGQFKNCT2AZBPVAVCNFSM6AAAAABRO6IUSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZZG42DONJSGU
.
You are receiving this because you authored the thread.Message ID:
@.***>

Good to hear!
Thank you and good job! :D

I’m encountering the same error I faced earlier, but today’s issue is particularly puzzling. Initially, everything was working fine—I successfully ran Tiny YOLO in real-time and on images. However, when I tried doing the same thing again later, everything stopped working.
The error first appeared when I used my custom dataset. I thought the issue might be with the object files, but then I swapped my custom .o object file (e.g., image.o) with the YOLOv3 COCO dataset object file. The code ran, but since the object file was different, it predicted everything except the planes I trained for.
Here’s what I’ve done for my custom dataset:

I modified the image.cpp file to update the classes and labels.
I adjusted the input and output nodes accordingly.
Despite this, the board still predicted labels from the COCO dataset, which is odd given that the labels and classes in my .cpp file were entirely different. This makes me suspect that the model was predicting COCO classes because I used the COCO .o object file.

Since .o files are generated by the Makefile, I assume I need to keep the objects folder empty in MobaXterm. However, the Makefile isn’t generating my .o files either—it’s throwing an error saying something like image.o doesn't exist.
When I tried running Tiny YOLO again, I faced the same error, even though it worked previously.
Here’s what I’ve tried so far: Formatting the SD card multiple times and copying the files back onto it.
My questions are as follows:

For the custom dataset, what could be going wrong?
You previously mentioned that my SD card image might be incorrect. How can I ensure it’s correct?
Why would Tiny YOLO suddenly stop working after working fine initially?

root@pynqz2_dpu1:~# cd yolo_pynqz2
root@pynqz2_dpu1:~/yolo_pynqz2# make
make: Warning: File 'Makefile' has modification time 1200 s in the future
g++ -c -O2 -Wall -Wpointer-arith -std=c++11 -ffast-math -mcpu=cortex-a9 -mfloat-abi=hard -mfpu=neon programs/yolo_image.cpp -o objects/yolo_image.o
make: g++: **Command not found**
make: *** [Makefile:40: objects/yolo_image.o] **Error 127**

Hi.
If it worked previously, then the SD card image is the right one, and you have the DNNDK libraries installed.
The object files are generated for each given c++ script. If you used the ones generated for the YOLO COCO dataset, then that is the expected result.
I suggest you also delete the executable file, as well as the object files, and try to compile again with the Makefile. If it does not work, try a couple of times.
By the gcc error, it is just a little confused with the object file. I might also suspect the object file name is not the same as mentioned in the Makefile. Also, make sure it is in the objects folder!
Let me know if it works!

sorry but which ones the exectuable file??

It is not mentioned on that image but after you compile the program successfuly with the Makefile, you will see it created a file called "yolo" or "yolo_image" without any extension. It will be present on the same folder as the Makefile and the dog.jpg image.

Hey Andre, everything is working okay now, tiny yolo is running, there was probably some issues with the zynq7020_dnndk_v3.1 file and it has been resolved now, but when I try to run my custom dataset, it working all but the board is not detecting anything. Do you know why would this happen?
It's showing this

We need to make sure it is not the accuracy of the model so, could you try to make detections on images used for training? Pick one of the training images and see if it detects. Try with various training images.
I suspect it can be low accuracy and so it migh need more training. Do you obtained any mean average precision value after the training process?
If the problem is not accuracy, then it might be something about the program. For instance, the image size on the output might be wrongly configured. When I did the project, some image sizes were a little weird when dsiplayed.

Ok, I'll take a look. I'll have to check. But the images used for training
were pf size 640x640 , but in the quantization we did it for 420x420, does
it make a difference?
On Mon, Nov 18, 2024, 10:16 PM André Araújo @.***>
wrote:

We need to make sure it is not the accuracy of the model so, could you try
to make detections on images used for training? Pick one of the training
images and see if it detects. Try with various training images.
I suspect it can be low accuracy and so it migh need more training. Do you
obtained any mean average precision value after the training process?
If the problem is not accuracy, then it might be something about the
program. For instance, the image size on the output might be wrongly
configured. When I did the project, some image sizes were a little weird
when dsiplayed.
—
Reply to this email directly, view it on GitHub
https://github.com/andre1araujo/YOLO-on-PYNQ-Z2/issues/18#issuecomment-2483572353,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A5RZEIX763YWNLQGIND4NCL2BIKVLAVCNFSM6AAAAABRO6IUSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBTGU3TEMZVGM
.
You are receiving this because you authored the thread.Message ID:
@.***>

Hi.
No, it will not make a difference because the YOLOv3 neural network has 416x416 images as input. So, on the training process you probably had to resize the training images. There are other YOLO versions on the site with different input image sizes but 416 seems to be the most common.
Sorry for the late responses!

Hi Andre, I hope you’re doing well!
I have tried using different datasets (in my case, time series of communication signals), and I also used TensorFlow to build a network, trained it, and got a .pb file. I have successfully gone through the quantization step for the model, and the next step is compilation. However, when I tried to compile, I encountered the error in the picture:

I removed the Softmax layer from the model, so the warning about Softmax no longer appears, but the issue with [The value is not supposed!..] still exists. I reduced the size of the model—from the original 400KB to 200KB—but the same issue still occurs. And according to your tutorial, the YOLOv3 model has much more parameters.
I am curious if you have ever encountered this issue, or if you have any relevant documentation about using DNNDK. Thank you very much!

Hi!
I never came across that particular error. Errors like that are more common when we start using other custom CNNs because DNNDK was still in early development at the time, and they probably did not conduct enough tests. I would suppose that YOLOv3 is a very deep neural network, so it should be a good test to compress.
In any case, I found this issue on the AMD forum and the comment redirects to this other forum question. If those are indeed related, then the problem is flatten and dense/FC layers on your network. I saw that sometimes, the DNNDK compiler takes initiative to execute the demanding FC layer on the CPU, but it probably happens when those are the last layers of the network. So you might avoid those (important) layers or try to replace them or convert them with a convolutional layer. Also take a look at the size, as mentioned in the comment.
I am sorry to report what seem to be the limitations of the DPU, but it really might be a little hard to deploy custom models. I once tried to implement the VGG19 model, and it had a lot of difficulties, even though it is a very well known model.
I don't know about the current progress on the Vitis AI framework, but now it might support a lot more layers. The problem is that, many times, we still don't have comprehensive tutorials for platforms like this. You might want to explore that if you cannot change the layers of your network.
I hope you can proceed, and I am sorry I could not fully help.
Good luck!

Thank you so much! I'll look into what you have referred!

Hi Andre!
Thank you very much for your help. It seems that the DPU doesn't support the Flatten layer. I have modified the model and completed the compilation, resulting in a new .elf file. However, during testing, I encountered an issue where the .elf file could not be loaded, with the following error:
[DNNDK] Fail to load DPU kernel: dpu_signal_0.elf, check if:
1- Specified DPU Kernel name "dpu_signal_0.elf" is right.
2- DPU Kernel "dpu_signal_0.elf" is compiled and linked into "/home/root/yolo_pynqz2/yolo_image" as expected.
3- DPU shared library "libdpumodeldpu_signal_0.elf.so" is located in the system library paths.
I checked the name of the .elf file and it seems correct. Have you encountered this issue before?

Hi!
I still don't know why that problem happens but I have seen it before while helping someone. I just took some time to look on the messages and this is what he did to solve it:

First you go to the MobaXterm and access the folder you want: cd yolo_pynqz2
Then you execute the following commands: gcc -shared -o libdpumodeldpu_signal_0.so and then dpu_tiny_yolo.elf. This will create a .so file with the name they refer to on the error.
Then apparently you go to "user/bin" and add the recently created .so file.
If the error presists, you delete the .so file from "user/bin" directory.

If you could do this process and then report to me how it went it would be much appreciated!
I hope this solves the problem!
Good luck :)

Thank you very much for your help. I followed your method and successfully solved the problem. Also, in the .cpp file, the file name loaded for the .elf should be changed to signal_0 (the full name of my .elf file is dpu_signal_0.elf).
Thank you again for your help. I will proceed to modify the .cpp file, and I would appreciate any suggestions you have for writing the .cpp file.

I am relieved it worked for you!
About the .cpp file, that is the complicated part haha. I think that first you should consult the examples of the DNNDK v3.1 package you downloaded on Ubuntu. They had some small implementation examples of object detection and image classification tasks. You should look at the example according to your model task. For example, if your model does image classification, you should consult the ResNet50 image classification example they have. Other examples should be available online or specifically on GitHub if you search for it.
With the gathered examples, you can try using an LLM to try and speed up the process of developing your own .cpp file.
I am sorry I don't have specific files here to help you, but this part should not be completely difficult because you just have to integrate a program with the DNNDK libraries.
I hope this helps to guide you a little to the desired solution. If you have any more problems, just tell me and I will try to help as best as I can!


5.3.x

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.16 out of 18 committers have signed the CLA.:white_check_mark: mmoayyed:white_check_mark: sbearcsiro:white_check_mark: tsschmidt:white_check_mark: pdrados:white_check_mark: viswatejan:white_check_mark: NgSekLong:white_check_mark: gregamu:white_check_mark: gagarski:white_check_mark: jtgasper3:white_check_mark: swlyons:white_check_mark: oxcafedead:white_check_mark: jgraglia:white_check_mark: ruckc:white_check_mark: mindcrime:white_check_mark: deejamin:white_check_mark: Golestandt:x: miller79:x: leleujYou have signed the CLA already but the status is still pending? Let us recheck it.


Do not expose mongo to public

It works fine with the following lines deleted.
https://github.com/karfly/chatgpt_telegram_bot/blob/main/docker-compose.yml#L9
https://github.com/karfly/chatgpt_telegram_bot/blob/main/docker-compose.yml#L29
If someone deploy on public cloud, it will be danger...

root@wowz:/home/wowz.co/chatgpt# docker-compose --env-file config/config.env up --build
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.mongo.ports contains an invalid type, it should be an array
services.mongo_express.ports contains an invalid type, it should be an array

I have this error after deleting these 2 lines

Same error if I comment these lines here.


Missing /etc/nginx/network_internal.conf for alpine Docker image

I tried setting the env variable INTERNAL_ACCESS to containers and confirmed that in default.conf, server blocks contains the line include /etc/nginx/network_internal.conf; however for alpine image, /etc/nginx/network_internal.conf is missing.
In the non-alpine image, it is copied in here: https://github.com/jwilder/nginx-proxy/blob/d6042d08f15ea5d99d9b80cdc496bdfc69629274/Dockerfile#L27

+1


Ci/release

Implements the release workflow & npm public publish.
To make the Gradle project compatible with semantic release there is a trick 😢 :
The project base version is stored in the kotlin/gradle.properties file. The version will then be formatted by default to the snapshot format, and to the release format if the -Prelease flag is passed to the Gradle command.
If you have any suggestion to make it better, I couldn't be happier.

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


ImportPlugin

PlugIn to import vectorGeoResources, which are provided via Drag&Drop, FileInput etc.

resolved by #730


unordered_map instead of map for tables?

Hi,
As TOML v0.4.0 specify that "Key/value pairs within tables are not guaranteed to be in any specific order.", wouldn't be better if tinytoml represented tables internally using std;;unordered_map instead of std::map?
Thanks.

Hi, since this library is not focusing on performance but handiness, I believe either is OK.
But yeah, std::unordered_map seems natural for this case.

Since I rely upon writing toml tables as well as parsing them, I would rather have them outputted in insertion order than random order.

@veeg yeah, it is also reasonable. However, to keep insertion order, we need to introduce something like LinkedHashMap in Java, however it's not available in STL. Another option is to use std::vector<std::pair<std::string, Value>>> for Table, but it won't be what we want to use.

Anyway, insertion order cannot be kept without introducing a bit fancy data structure.

Ahh, I'm quite forgetfull. I played around with insertion order on my local fork instead. I must have mixed up my experiences and memories regarding the matter.

To reduce the impact, a solution could be to limit the use of std::unordered_map to the parser? This way, the writer (that we are not using), would be unaffected. I understand it may be harder to implement though, as some code/definitions couldn't any more be shared between the two components of the library.


New theme: essential

A simple, console-compatible Bash-it theme.
Features:

Reactive coloring for:

Privileged and standard users.
Local and remote hosts.
Exit status for prompt strings 1 and 2.

Console-compatible!

No unicode "tofu" in console sessions.

Matching alignment for prompt strings 1 and 2.

Preview:

Cool - thanks!


[Discussion] JOSS review point: installation instructions

Hey spleeter devs, I'm a reviewer on your (JOSS submission](https://github.com/openjournals/joss-reviews/issues/2154)
Working down the checklist, I had a question about your installation instructions.  They appear to be correct (ie it works), but I'm confused by the instruction to both clone the repo and install from conda-forge.  I assume this is only written this way to ensure that the demo track is available for running the test command.
I might suggest breaking this up into more explicit sections for installing the package (for use), installing for development (git clone), and testing on an example audio file.

Hi @bmcfee
First of all, thanks a lot for taking upon your time to review our submission, especially considering the worldwide situation going on.
Indeed you're right that the cloning is not really needed now. In a first version we were also locally building a conda env from a cloned file. But since we've ported spleeter to conda-forge we should probably update the documentation accordingly.
I'm letting this issue open until we do so.
Stay safe and I hope you'll have some fun in using spleeter.

Hi @bmcfee
Thank you for the suggestion. Indeed, it was unclear that the git clone was only for downloading the audio demo file.
We updated the readme for making the Quick start instructions more explicit by adding comments and replacing the git clone by a wget.
Let us know if you feel it may be improved further.


Change the geospatial demo to the open source taxi data set

To accelerate testing, change the queries to use the open source taxi data set included as part of the ADX demos.

will not continue with the geospatial demo


DX: namespace casing

Just spotted this - not sure if it's worth to write an AutoReview test to prevent it in future.

Thank you @kubawerlos.


Update build.gradle (compile 'com.facebook.react:react-native:+')

We don't have to limit our react-native version to a specific one. This will only give us an error after the RN upgrade.

@caiosba merge plz

Awesome, thanks!

Thanks, I just released a new version on NPM.


Asset filtered out and will not be served: add `Rails.application.config.assets.precompile += %w( radio.png )

hi all.
when i add the gem and bundle it and modify my application.js and application.css in rails i face the error below :
Sprockets::Rails::Helper::AssetFilteredError in Pages#home
Asset filtered out and will not be served: add Rails.application.config.assets.precompile += %w( radio.png ) i add a lot like the error to config/initializers/assets.rb` but i think its not the way?does anyone face that before ?

I think this problem isn't a Flat-UI problem, because the project doesn't have any radio.png file. Maybe you want to ask in stackoverflow for more visibility.


Script execution Oracle

Hi.
Will be great if the tool supports oracle scripts separated by semicolon.
This a important feature.

Closing this as duplicate of #727 .


Rework IO#gets

This fixes the following issues:

Accept lines of more than 1024 bytes
Accept files not ending with a newline
Improve performance by reading chunks instead of separate bytes

My long term plan was to fix the remaining spec incompatibilities of IO#gets, then implement IO#readline and IO#readlines using IO#gets. I may have gotten sniped by #1314 for that one. I will still try to finish this method and have another look at all the duplication we have now.

I may have gotten sniped by #1314 for that one.

that's what I'm here for 😂


[DataStructure] observing Elasticsearch error type:mapper_parsing_exception

When new yang is delivered for Ericsson device, MWDI unable to save the cache into existing index entry.
Manually we need to delete the index and need to recreate.
Version: 1.1.2.a_impl
Logs:
[24.04.2024 06:26.52.016] [ERROR] Elasticsearch error occurred: {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse field [core-model-1-4:control-construct.backup-and-restore-1-0:backup-and-restore-pac.backup-and-restore-status.ready-for-starting-new-operation] of type [boolean] in document with id '529554222B'. Preview of field's value: 'backup-and-restore-1-0:READINESS_STATUS_TYPE_READY'"}],"type":"mapper_parsing_exception","reason":"failed to parse field [core-model-1-4:control-construct.backup-and-restore-1-0:backup-and-restore-pac.backup-and-restore-status.ready-for-starting-new-operation] of type [boolean] in document with id '529554222B'. Preview of field's value: 'backup-and-restore-1-0:READINESS_STATUS_TYPE_READY'","caused_by":{"type":"illegal_argument_exception","reason":"Failed to parse value [backup-and-restore-1-0:READINESS_STATUS_TYPE_READY] as only [true] or [false] are allowed."}},"status":400}
[24.04.2024 06:26.52.016] [ERROR] ResponseError: mapper_parsing_exception: [mapper_parsing_exception] Reason: failed to parse field [core-model-1-4:control-construct.backup-and-restore-1-0:backup-and-restore-pac.backup-and-restore-status.ready-for-starting-new-operation] of type [boolean] in document with id '529554222B'. Preview of field's value: 'backup-and-restore-1-0:READINESS_STATUS_TYPE_READY'
    at onBody ([/home/openbackhaul/microWaveDeviceInventory/node_modules/@elastic/elasticsearch/lib/Transport.js:367:23](mailto:/home/openbackhaul/microWaveDeviceInventory/node_modules/@elastic/elasticsearch/lib/Transport.js:367:23))
    at IncomingMessage.onEnd ([/home/openbackhaul/microWaveDeviceInventory/node_modules/@elastic/elasticsearch/lib/Transport.js:291:11](mailto:/home/openbackhaul/microWaveDeviceInventory/node_modules/@elastic/elasticsearch/lib/Transport.js:291:11))
    at IncomingMessage.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1358:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  meta: {
    body: { error: [Object], status: 400 },
    statusCode: 400,
    headers: {
      'x-elastic-product': 'Elasticsearch',
      warning: '299 Elasticsearch-7.14.2-6bc13727ce758c0e943c3c21653b3da82f627f75 "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.14/security-minimal-setup.html to enable security."',
      'content-type': 'application/json; charset=UTF-8',
      'content-length': '892'
    },
    meta: {
      context: null,
      request: [Object],
      name: 'elasticsearch-js',
      connection: [Object],
      attempts: 0,
      aborted: false
    }
  }
}

With 1.1.2d , we are still seeing this Issue after Ericsson Mediator upgrade.
We tried to re-deploy with new index , but still unable to retrievd the mwdi cache for Ericsson devices.
Uploading logs_946.txt…

The MWDI should just take what it gets from the Controller and store it in ElasticSearch as long as the highlevel structure of the CC matches the schema provided in the MWDI OAS.
However, at the moment this is not possible, if the CC contains an unexpected data structure (e.g. due to new object model on some mediators). (Easy typecasts from e.g. String to Int are no problem). This is a shortcoming of ElasticSearch.

if such cases (the same property has different types) are expected then different DB should be used to store data - NoSQL DB like MongoDB or SQL DB like PostgreSQL with JSONB column type, etc.

Info_946.docx

is there an update?

Monitored till 1.1.2.g, issue not observed, hence closing this issue


Add licence comment headers to source files

Solved. Can now be done with mvn license:format. The build checks if the Apache 2 license has been added, and fails otherwise.


error in flash size for Adafruit Feather rp2040 and ItsyBitsy rp2040?

Hi,
while  trying to add the Adafruit Feather rp2040 and ItsyBitsy rp2040 boards to Micropython I noticed a difference in flash size in the board definitions at pico-sdk/src/boards/include/boards/adafruit_itsybitsy_rp2040.h and pico-sdk/src/boards/include/boards/adafruit_feather_rp2040.h.
According to the web site feather rp2040 and ItsyBitsy rp2040 they have 8MB flash, but the board definitions (in the master branch at least) have just 4MB defined :
#define PICO_FLASH_SIZE_BYTES (4 * 1024 * 1024)
Is this a leftover from pre-production samples that did not have the 8MB flash?
I do have the boards but did not try to write more than 4MB to the flash yet, but Circuitpython shows the additional 4MB as present.

This was fixed in #348

For the Adafruit Feather RP2040 PICO_FLASH_SIZE_BYTES is defined as 8 MB in master and in pico-sdk v1.2.0.
For the Adafruit ItsyBitsy RP2040 PICO_FLASH_SIZE_BYTES is defined as 4 MB in master and in pico-sdk v1.2.0. This was fixed in the develop branch with PR #293.

What MicroPython sees will depend of the version of pico_sdk it uses.

I admittedly did not search through the dev branch. Sorry for the clutter.


protoc-gen-go: don't depend on input file ordering

Remove generate code dependencies on the order in which source files were
provided to the compiler. In other words, "protoc a.proto b.proto" produces
the same output as "protoc b.proto a.proto".

Include the proto package version assertion in all files.

Use the source file name and contents to generate unique var names for
file descriptors, rather than the file index. In other words,
"fileDescriptor_imp_81275c260ac30f8b" rather than "fileDescriptor0".

Makes the generated code more stable in the face of unrelated changes
(i.e., adding a new file to a package won't cause generated code for
other files in irrelevant ways), as well as trivial changes in the
protoc command line.
Removes the requirement that all files in a package be compiled at
the same time. Compiling each file individually will produce the same
results as compiling all at once.

Sounds good. As long as the code generation also stays deterministic for
repeated runs.
On Thu, 1 Mar 2018, 21:51 Damien Neil<EMAIL_ADDRESS>wrote:

Remove generate code dependencies on the order in which source files were
provided to the compiler. In other words, "protoc a.proto b.proto" produces
the same output as "protoc b.proto a.proto".

Include the proto package version assertion in all files.
Use the source file name and contents to generate unique var names for
file descriptors, rather than the file index. In other words,
"fileDescriptor_imp_81275c260ac30f8b" rather than "fileDescriptor0".
Makes the generated code more stable in the face of unrelated changes
(i.e., adding a new file to a package won't cause generated code for
other files in irrelevant ways), as well as trivial changes in the
protoc command line.
Removes the requirement that all files in a package be compiled at
the same time. Compiling each file individually will produce the same
results as compiling all at once.
You can view, comment on, or merge this pull request online at:
https://github.com/golang/protobuf/pull/542
Commit Summary

protoc-gen-go: don't depend on input file ordering

File Changes

M protoc-gen-go/generator/generator.go
https://github.com/golang/protobuf/pull/542/files#diff-0 (43)
M protoc-gen-go/testdata/deprecated/deprecated.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-1 (30)
M protoc-gen-go/testdata/extension_base/extension_base.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-2 (26)
M protoc-gen-go/testdata/extension_extra/extension_extra.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-3 (16)
M protoc-gen-go/testdata/extension_user/extension_user.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-4 (66)
M protoc-gen-go/testdata/grpc/grpc.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-5 (28)
M protoc-gen-go/testdata/imp/imp.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-6 (38)
M protoc-gen-go/testdata/imp/imp2.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-7 (24)
M protoc-gen-go/testdata/imp/imp3.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-8 (20)
M protoc-gen-go/testdata/imports/fmt/m.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-9 (6)
M protoc-gen-go/testdata/imports/test_a_1/m1.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-10 (6)
M protoc-gen-go/testdata/imports/test_a_1/m2.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-11 (12)
M protoc-gen-go/testdata/imports/test_a_2/m3.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-12 (6)
M protoc-gen-go/testdata/imports/test_a_2/m4.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-13 (12)
M protoc-gen-go/testdata/imports/test_b_1/m1.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-14 (6)
M protoc-gen-go/testdata/imports/test_b_1/m2.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-15 (12)
M protoc-gen-go/testdata/imports/test_import_a1m1.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-16 (16)
M protoc-gen-go/testdata/imports/test_import_a1m2.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-17 (22)
M protoc-gen-go/testdata/imports/test_import_all.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-18 (22)
M protoc-gen-go/testdata/imports/test_import_public.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-19 (22)
M protoc-gen-go/testdata/multi/multi1.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-20 (6)
M protoc-gen-go/testdata/multi/multi2.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-21 (16)
M protoc-gen-go/testdata/multi/multi3.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-22 (16)
M protoc-gen-go/testdata/my_test/test.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-23 (86)
M protoc-gen-go/testdata/proto3/proto3.pb.go
https://github.com/golang/protobuf/pull/542/files#diff-24 (12)

Patch Links:

https://github.com/golang/protobuf/pull/542.patch
https://github.com/golang/protobuf/pull/542.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/golang/protobuf/pull/542, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABvsLS5yQNDVwLBPwQ1wk3p4GEsmxQglks5taF9egaJpZM4SY-QC
.

Yes, the code generation is deterministic so long as the FileDescriptorProto is. It uses the filename and a hash of the descriptor to ensure uniqueness.

Yay, gofmt changes break tests! :)

Yes, the code generation is deterministic so long as the FileDescriptorProto is.

That's a important assumption. We should get confirmation from the protoc owners.

Why does this fingerprint the entire FileDescriptorProto, rather than just, say, hashing the filename? I'm having a bit of a nuisance due to these hashes changing for apparently no good reason (the .proto doesn't change, and the gzipped FileDescriptorProto bytes don't even change!).
Actually, now that I've written that, I can't understand what I'm seeing. If the FileDescriptorProto bytes don't change, then the computed hash can't change. But the only difference is that the source_code_info field is removed before marshaling+gzipping the descriptor, so the difference must be in there.
Actually, I tracked it down. Between protoc 3.5.1 and 3.6.1, the source_code_info data changed somehow. This explains what I was seeing.
Anyway, to return to my initial question:

Why hash the encoded FileDescriptorProto instead of just the filename?
If that hash remains, can we skip the source_code_info component in the same way that generating the gzipped byte slice does so that protoc changes don't affect the hash?

Why hash the encoded FileDescriptorProto instead of just the filename?

I don't remember. Just the filename is probably fine.

Changed to use a hash of the filename alone.

Thanks @neild!


new Date(new Date().toLocaleDateString()) return  Invalid Date

Which version of PhantomJS are you using? Tip: run phantomjs --version.

2.1.1

What steps will reproduce the problem?

new Date(new Date().toLocaleDateString())
return "Invalid Date"
chrome is ok

Which operating system are you using?

mac os 10.12.2 (16C67)

Did you use binary PhantomJS or did you compile it from source?

no

Please provide any additional information below.

i am from china

Could you try again with 2.5 binary please? Thanks!

Worked for me on 2.5 beta!
I was using new Date().toLocaleDateString('nl-NL') , expecting '28-12-2017' for example but was getting '28 December 2017' instead.


Using real-world galaxies

List of nearest galaxies https://en.wikipedia.org/wiki/List_of_nearest_galaxies
https://en.wikipedia.org/wiki/Observable_universe

off-topic
https://exoplanetarchive.ipac.caltech.edu/ NASA confirmed planets
youtube
Size Comparison of the Universe
ABSOLUTE Size Comparison 2023 (FULL) 4K 60FPS 3D

Hello! We only accept feature proposal via discord see https://github.com/Limit-Theory-Redux/ltheory/blob/main/CONTRIBUTING.md so i´ll have to close this issue

Hello! We only accept feature proposal via discord see https://github.com/Limit-Theory-Redux/ltheory/blob/main/CONTRIBUTING.md so i´ll have to close this issue

Sorry. I don't use Discord, and I think it's bad to discuss feature requests on social media

Hey now worries, i´m sorry that it is that way, but we have thorough discussions on discord, which we chose over GH discussions, regarding the project and all implementation discussions. It´s just easier to do community related things in one place. What i can do is post what you wrote here over there so it´s properly tracked.