unicharset_extractor not generating some characters for Burmese Language

Environment

Tesseract Version: tesseract 4.0.0-rc1-458-g7ddc, leptonica-1.75.3
Commit Number: 7ddca79f1254c2f3dd930f560076d202e6fc88e2
Platform: Ubuntu 18.04.1 LTS 64-bit

Current Behavior:
I noticed that some Burmese characters are not recognized/extracted by unicharset_extractor from the Burmese training text. As I am new to this topic, I honestly have no idea how the unicharset_extractor works. However, I do noticed that there are several characters missing in the .unicharset file. Some of the important characters I noticed missing are ၍, ၌, ၎, ၏ and two main punctuation characters ၊ and ။. Note: they do exists in the training_text file. Even if I provided those characters in mya.punc and mya.wordlist, they don't get generated/recognized.
Any thoughts/pointers on the cause?
P.S. There are other ways to extract the unique character of our language. I could contribute the solution if I know how the columns of the generated unicharset file is structured.

The unicharset is extracted from the trainiing text. Words may be dropped from the text if they are unrenderable in the fonts used for training.
Please see https://github.com/tesseract-ocr/langdata_lstm/tree/master/mya
okfonts.txt has a list of fonts used for training. Please confirm that the missing characters are supported in these fonts. Also suggest unicode fonts which have full coverage for Burmese.
Characters may also be dropped if they don't meet the normalization rules. Please see
https://github.com/tesseract-ocr/tesseract/blob/master/src/training/validate_myanmar.cpp
https://github.com/tesseract-ocr/tesseract/blob/master/src/training/validate_indic.cpp
https://github.com/tesseract-ocr/tesseract/blob/master/src/training/validate_grapheme.cpp

also see PR https://github.com/tesseract-ocr/tesseract/pull/2070 by @pndaza

Hi @Shreeshrii
Thank you so much for the directions.
And I just found the pull request #2070 which should probably solve the issue.
However I got Normalization failed for string error on all the six characters ၍ ၌, ၎, ၏, ၊, ။ and a bunch of other characters.
I am getting the same type of error as your issues #1452 and #1135.
Here is some of the output.
Word started with a combiner:0x103a
Normalization failed for string '်'
Invalid start of Myanmar syllable:0x104d
Normalization failed for string '၍'
Word started with a combiner:0x102e
Normalization failed for string 'ီ'
Word started with a combiner:0x1038
Normalization failed for string 'း'
Word started with a combiner:0x102c
Normalization failed for string 'ာ'
Word started with a combiner:0x1038
Normalization failed for string 'း'
Word started with a combiner:0x103a
Normalization failed for string '်'
Word started with a combiner:0x1038
Normalization failed for string 'း'

I think I got it.
I was relying too much on the tesstrain.sh. After looking at the unicharset_extractor source, I found there is a different norm mode for different languages. I ran the unicharset_extractor on its own and the unique characters are now generated as expected.
Note; it still shows the same error
Word started with a combiner:0x103a
Normalization failed for string '်'
Invalid start of Myanmar syllable:0x104d
Normalization failed for string '၍'
Word started with a combiner:0x102e
Normalization failed for string 'ီ'
Word started with a combiner:0x1038
Normalization failed for string 'း'
Word started with a combiner:0x102c
Normalization failed for string 'ာ'
Word started with a combiner:0x1038
Normalization failed for string 'း'
Word started with a combiner:0x103a
Normalization failed for string '်'
Word started with a combiner:0x1038
Normalization failed for string 'း'

There are two possible issues here.
One is that the normalization rules for Burmese are wrong or incomplete.
@pndaza Does your PR fix the issue in your own testing?
Second is that the training text is not normalized or is not in correct Unicode format.
From what I understand, Burmese uses another non-Unicode compliant mapping for the same range of Unicode chars. Since the training_text is scraped from internet, it is possible that there are errors in the training text. So, you can check for that.

Hi again,
I am back on track for generating with the data.
Yes, PR #2070 fixed this issue. There is still two characters left out though tall aa and aa ာ  ါ. They are included in the code but I still couldn't debug why it is not working yet.
I noticed one odd thing here though. The unicharset file in the original langdata has all the right characters.
Because the unicharset_extractor is written differently for that file?
After looking at the various issues, and the source codes itself, there are a few more things that needed to be fixed. So, I am wondering how the unicharset in the langdata folder is generated and if possible how could I port it for lstm training.

Also, can you please point me to where I can see the meaning behind these numbers?
Joined 7 0,255,0,255,0,0,0,0,0,0 Latin 135 0 127 Joined	# Joined [4a 6f 69 6e 65 64 ]a
|Broken|0|1 f 0,255,0,255,0,0,0,0,0,0 Common 2 10 2 |Broken|0|1	# Broken
ယ 1 64,64,254,255,336,359,20,21,378,402 Myanmar 3 0 3 ယ	# ယ [101a ]x
ခ 1 64,64,254,255,190,190,20,21,232,234 Myanmar 4 0 4 ခ	# ခ [1001 ]x
ဩ 1 0,255,0,255,0,0,0,0,0,0 Myanmar 40 0 40 ဩ	# ဩ [1029 ]x
၈ 8 61,64,254,254,190,193,20,21,232,237 Myanmar 58 0 58 ၈	# ၈ [1048 ]0

The unicharset file
https://github.com/tesseract-ocr/langdata/blob/master/Myanmar.unicharset in
the original langdata has all the right characters.

That is the script reference file.
unichar_extractor goes through the given training_text and generates a
unicharset based on that. It can have characters from multiple scripts
(Common, Latin, Myanmar etc.)
https://github.com/tesseract-ocr/tesseract/wiki/Training-Tesseract-3.03–3.05#the-unicharset-file-format
has general info about the file - from old 3.0x version.
Program now generates the required fields.
On Tue, Feb 12, 2019 at 6:05 PM May Pearl Aung<EMAIL_ADDRESS>wrote:

Also, can you please point me to where I can see the meaning behind these
numbers?
Joined 7 0,255,0,255,0,0,0,0,0,0 Latin 135 0 127 Joined	# Joined [4a 6f 69 6e 65 64 ]a
|Broken|0|1 f 0,255,0,255,0,0,0,0,0,0 Common 2 10 2 |Broken|0|1	# Broken
ယ 1 64,64,254,255,336,359,20,21,378,402 Myanmar 3 0 3 ယ	# ယ [101a ]x
ခ 1 64,64,254,255,190,190,20,21,232,234 Myanmar 4 0 4 ခ	# ခ [1001 ]x
ဩ 1 0,255,0,255,0,0,0,0,0,0 Myanmar 40 0 40 ဩ	# ဩ [1029 ]x
၈ 8 61,64,254,254,190,193,20,21,232,237 Myanmar 58 0 58 ၈	# ၈ [1048 ]0
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/tesseract/issues/2232#issuecomment-462742623,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o2rCxT5CnEIzVI3Qr50vokaYkqcbks5vMrT2gaJpZM4azmwi
.

--

भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com

See https://github.com/tesseract-ocr/tesseract/pull/2236/files
for some more changes that maybe needed.

You can use merge_unicharsets followed by combine_lang_model to add any additional characters to the generated unicharsets.

After looking at the various issues, and the source codes itself, there are a few more things that needed to be fixed

@maypearlaung  Have you made any progress on this?

Please test the traineddata at https://github.com/Shreeshrii/tessdata_shreetest/blob/master/mya430000.traineddata
and let me know whether it is an improvement.

Please test the traineddata at https://github.com/Shreeshrii/tessdata_shreetest/blob/master/mya430000.traineddata
and let me know whether it is an improvement.

Caught up with my school projects and couldn't able to look at that.
@Shreeshrii  Yes, I can see there are a few improvements.
I can confirm that your modifications along with #2070 does solve the validation issues.
May I know your training settings?
I was using my own training_text consists of around 400K sentences. I use best traineddata from the repo as a checkpoint and follow Fine Tuning for characters approach. However, my result is a bit worse than yours. The model couldn't learn new characters like ။ at all.

According to your comment, do I need to change normalization mode for Burmese to 3?

do I need to change normalization mode for Burmese to 3?

Yes. You can change language_specific.sh
 case "${LANG_CODE}" in
    ara | div| fas | pus | snd | syr | uig | urd | kur_ara | heb | yid )
      LANG_IS_RTL="1"
      NORM_MODE="3" ;;
    bod | tha | mya)
      LANG_IS_RTL="0"
      NORM_MODE="3" ;;
    asm | ben | bih | hin | mar | nep | guj | kan | mal | tam | tel | pan | \
    dzo | sin | san | ori | khm | lao | jav  | jav_java )
      LANG_IS_RTL="0"
      NORM_MODE="2" ;;
    * )
      LANG_IS_RTL="0"
      NORM_MODE="1" ;;
  esac

May I know your training settings?

I ran some more tests after that.  I did replace top layer training using multiple training texts . I am attaching a zip file with the bash script used and resulting traineddata (which is different from the one I had uploaded earlier).
mya-layer.zip
Also, my latest changes are in a pending PR - https://github.com/tesseract-ocr/tesseract/pull/2244 - this has not been merged. You can try it and if it helps, I will resubmit. Thanks!

@Shreeshrii that's awesome... I will try out your script too.
Yes, I tested codes from PR #2244 just a few hours ago. It seems to be working, although I haven't run it on large training texts. I will let you know again.

@zdenop This issue can be closed.


Create new bug report starter

[x] Create starter project on CodeSandbox

should have all packages and deps installed
test that any component can be added
fork/save
add simple readme with links etc on how to report

Used StackBlitz instead due to some odd issues on CodeSandbox


Fix flakey "_endpoint" tests

Example of failing tests:

test_rio_app_endpoint: https://drone-pr.rancher.io/rancher/rio/425/1/2 , https://drone-pr.rancher.io/rancher/rio/423/1/2
test_rio_svc_endpoint1

Closed as we are moving to go tests, so no point in spending time to fix.


fix SSL error

This just disables signature verification completely, why do you need to do it? What error are you experiencing?

I can confirm. Before this "fix" urllib2 complains about expired SSL certificates (at least on an up to date Windows 10), and imo compiling with headers of "unverified sources" is still better than not compiling at all.

I ran into a similar issue yesterday. For context, I'm using Git bash on Windows. I got the following error while running the script: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired
The way I solve it was to use a different SSL context:
import certifi
import ssl

...

certifi_context = ssl.create_default_context(cafile=certifi.where())
web = urllib2.urlopen(urllib2.Request(url, headers={'User-Agent': 'Mozilla/5.0'}), context=certifi_context)

(That requires pip install certifi)
I'm writing here only to share the information. I have no opinion regarding if, and how, the main project should handle the issue.


I use LCMXO2-2000HC-6G100 in the test that report “Error: data error during the speed test on count 0”

I check the USB3343 has 26MHz clock input and  it is producing 60 MHz clock. But the IFCLK have no 30MHz clock.
In the other hand, I use Lattice Diamond build a new JED file and program to FPGA. But it is as same as not work.

IFCLK is the output of the Cypress MCU, so there is some issue there.

Thanks I double check the Cypress MCU. It has two pin not connect.
It is OK now.
Thanks you very much!

On Mon, Sep 02, 2024 at 07:06:24PM -0700, MrLeekon wrote:

It is OK now.

Those who can close this issue, please do.


Improve the GitHub landing page

Closes #213

Titles and links look good! I'll leave final section ordering for the content overview, but I see you shifted a few things around for now.

What I did was align the titles on the landing page with their respective ordering in the table of contents. I didn't introduce any new shifts. I agree we can review the ordering of topics during the #214 revision cycle.

I'll merge this based on Drew's review. We are about to start the revision cycle so we can fine-tune this landing page at that stage.


Merge test branch into dev

Few info is added and tested. Merge test branch into dev.

Test branch is merged  into dev.


Cron jobs specified by datahub-ingestion-cron in helm chart hanging forever

Describe the bug
Having specified a datahub-ingestion-cron job in the helm values.yaml, the cronjob is installed on the platform.
But running it leads to never ending execution. No error message. No log output.
To Reproduce
Steps to reproduce the behavior:

add datahub-ingestion-cron to helm values.yaml
update/install the deployment (and wait for having finished)
trigger cron job execution
execution starts, never terminates and shows no further output after telling of having started the ingestion (no data arrives in the sink)

Expected behavior
Ingestion should happen, job terminate afterwards, logs should indicate some progress
Screenshots
Additional context

Caused by missing the "run" keyword in the default being used in datahub/subcharts/datahub-ingestion-cron/templates/cron.yaml, line 46
Without that the command is doing nothing, does not abort nor is doing anything.
Workaround: Add (same indentation as "schedule" line)
command: '"datahub ingest run -c /etc/recipe/
to the values.yaml using for deployment.


Sharp Readable Stream Piped With A Transform Stream Emits One Massive Chunk

Problem
I am trying to use sharp as a readable stream to pipe it with a custom transform stream.
The purpose of using stream approach instead of generating buffer data is to have least possible memory usage while processing an Image
The issue is when a readable sharp stream is piped with a transform stream, it ends up emitting single huge chunk.
Normally, maximum chunk size provided in a transform stream is 65536, but in this case I am getting the whole image as a massive chunk.
Sample Code
I have created a git repository to reproduce this behaviour - https://github.com/manojc/sharp-stream
In this repository, I have -

sharp as readable stream
custom transform stream to calculate the sum of the lengths of all incoming chunks
a large jpg image consumed by sharp with size of 30MB

As an output, the code prints -

total length of all the buffers provided in transform stream
number of chunks processed during the operation

Code
const { readFileSync } = require("fs");
const { Transform, Writable } = require("stream");
const sharp = require("sharp");

const imageFilePath = readFileSync("./30mb.jpg");

class SizeCalculator extends Transform {
    constructor(options) {
        super(options);
        this.size = 0;
        this.count = 0;
    }
    _transform(chunk, encoding, callback) {
        console.log(++this.count); // count the chunks processed
        chunk && chunk.length && (this.size += chunk.length); // increment size by chunk.length` amount
        callback();
    }
    _flush(callback) {
        this.push(this.size.toString());
        callback();
    }
}

function getSize(image) {
    return new Promise(async (resolve, reject) => {
        const sizeTransform = new SizeCalculator(); // transform stream
        sizeTransform.on("finish", () => resolve(sizeTransform.size)); // listen to finish event
        image.pipe(sizeTransform); // readable | transform
    });
}

async function sharpTest() {
    let image = sharp(imageFilePath);
    console.log(await getSize(image)); // get cumulative size of all the chunks passed in _transform function
}

sharpTest();

Output

Expected output was to see more than 1 (image size in bytes / 65536) chunks
Actual output is 1 chunk

Additional Information
Why Streams
I started using sharp with .toBuffer() API and ended up getting huge memory spikes in the application. That's where I came across this node issue which explains how node can sometimes ignore used buffer memory while triggering the GC. one of the answers in the same issue & my stackoverflow question provide more explanation
Please let me know if I'm missing anything here. Thanks!

Hello, are you able to provide any more information or links to documentation about "maximum chunk size provided in a transform stream is 65536" as I'm not aware of this limitation.
You may be interested in subscribing to #179

Hello lovell, thanks for the link to the issue, I will subscribe to it.
Well as you pointed out correctly, there is no limitation to the maximum chunk size provided in the stream.
However, streams are generally used to allow data processing without having to hold the entire data in memory at once (so normally chunk size is 65536).
Keeping this point in mind I went for the streams implementation but because sharp streams emit single chunk equal to the size of original image, I found no difference (in terms of memory usage) between buffers and streams implementation.
The whole purpose of this POC was to use streams for image processing so that amount of data held in the memory at a time is less compared to buffers.
Please let me know if I'm missing anything here.


Not working on smaller mobile screen

The Microsoft docs website and .net api browser does not work on smaller mobile device screens

Forgot to mention the screen resolution not working was 640 x 960 pixels

@tb-mtg, thanks for your feedback. could you share additional details, including screenshots of what issue you are experiencing to help us investigate further

Operating System is iOS 9.3. I know this is old but is still used by a lot at my workplace.
The following browsers are not working properly using www.whatismybrowser.com as a reference:

Safari 9
Opera Mini 16
Chrome 63

Screenshot below is of the top of the page. None of following links/buttons work:

Share this document link
Theme dark/light switcher
Contents menu (the main one of interest)

Screenshot below is of the bottom of the page. None of these links/buttons work and the:

This Page link/button
"Loading feedback" does finish and the icon keeps spinning.


JDBC drivers v1.0.0 release failed

FYI the GitHub action to release v1.0.0 of the JDBC drivers failed, so the JARs are currently not available among the v1.0.0 assets on GitHub nor at Maven Central.

JARs now available here: https://repo1.maven.org/maven2/org/duckdb/duckdb_jdbc/1.0.0/


UnsupportedOperationException: Unsupported band type (15, 1)

I have translated the red band of a 10mt sentinel 2 image using gdal_translate:
%sh */gdal_translate */T23KKS_20210723T132241_B04_10m.jp2 /T23KKS_20210723T132241_B04_10m.tif -of COG.
When I try to read this tif file using rasterframes I get  the folowing error:
df = spark.read.raster('/T23KKS_20210723T132241_B04_10m.tif')
Error:
IllegalArgumentException: Error fetching data for one of: JVMGeoTiffRasterSource(/dbfs/FileStore/shared_uploads/Remote_sensing_trial_data/Sentinal_1/Sentinel_2_tiff/S2_gdal_converted_tifs/T23KKS_20210723T132241_B04_10m.tif)
Caused by: UnsupportedOperationException: Unsupported band type (15, 1)
Any help regarding what is causing the error or what am I doing wrong.

Thanks @pomadchin for the reply. I am having trouble connecting the installed GDAL with Raster frames. That is why I was trying to convert the jp2's to tiff and then read via Rasterframes.
Below I have pasted  the gdalinfo of the converted tiff.
Could you please point out which one is the problem. Is it one of  "Type=UInt16" or "NBITS =15"?
Apologies if this is a very trivial question. I am a newbie in this domain.
gdainfo X.tif:
Size is 1830, 1830
Coordinate System is:
PROJCRS["WGS 84 / UTM zone 23S",
BASEGEOGCRS["WGS 84",
DATUM["World Geodetic System 1984",
ELLIPSOID["WGS 84",6378137,298.257223563,
LENGTHUNIT["metre",1]]],
PRIMEM["Greenwich",0,
ANGLEUNIT["degree",0.0174532925199433]],
ID["EPSG",4326]],
CONVERSION["UTM zone 23S",
METHOD["Transverse Mercator",
ID["EPSG",9807]],
PARAMETER["Latitude of natural origin",0,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8801]],
PARAMETER["Longitude of natural origin",-45,
ANGLEUNIT["degree",0.0174532925199433],
ID["EPSG",8802]],
PARAMETER["Scale factor at natural origin",0.9996,
SCALEUNIT["unity",1],
ID["EPSG",8805]],
PARAMETER["False easting",500000,
LENGTHUNIT["metre",1],
ID["EPSG",8806]],
PARAMETER["False northing",10000000,
LENGTHUNIT["metre",1],
ID["EPSG",8807]]],
CS[Cartesian,2],
AXIS["(E)",east,
ORDER[1],
LENGTHUNIT["metre",1]],
AXIS["(N)",north,
ORDER[2],
LENGTHUNIT["metre",1]],
USAGE[
SCOPE["Engineering survey, topographic mapping."],
AREA["Between 48°W and 42°W, southern hemisphere between 80°S and equator, onshore and offshore. Brazil."],
BBOX[-80,-48,0,-42]],
ID["EPSG",32723]]
Data axis to CRS axis mapping: 1,2
Origin =<PHONE_NUMBER>00000000000,7700020.000000000000000)
Pixel Size = (60.000000000000000,-60.000000000000000)
Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  199980.000, 7700020.000) ( 47d52'53.69"W, 20d46'31.81"S)
Lower Left  (  199980.000, 7590220.000) ( 47d54' 3.41"W, 21d45'58.74"S)
Upper Right (  309780.000, 7700020.000) ( 46d49'39.05"W, 20d47'23.88"S)
Lower Right (  309780.000, 7590220.000) ( 46d50'23.31"W, 21d46'53.53"S)
Center      (  254880.000, 7645120.000) ( 47d21'44.87"W, 21d16'45.03"S)
Band 1 Block=1830x2 Type=UInt16, ColorInterp=Gray
Image Structure Metadata:
NBITS=15

This issue is solved by setting : outputType=gdal.GDT_Int16 while converting jp2 to tiff via gdal_translate.


baseUrl config parameter

What do you think about adding baseUrl config parameter?
var github = axios.create({baseUrl: 'https://api.github.com/'});
github.get('/users/nickuraltsev');

I'd be happy to create a PR for this.

+1

I agree with @nickuraltsev, I've made a little wrapper around axios for my own usage:
https://github.com/iam4x/isomorphic-flux-boilerplate/blob/master/shared%2Fapi-client.js

:+1:

@nickuraltsev happy to accept a PR

Great, I'll create a PR soon.

Awesome!

is that why this doesn't work:
axios('/user')
and this does?
axios('http://localhost:3000/user')

@SkateFreak Do you have this issue when using axios in the browser or node.js? If the former, relative URLs should work fine as far as I know.


WinUI application closed by 'X' but remains as dormant proc

Describe the bug
I close the WinUI application by pressing 'X' on the window but remains dormant and must kill it in the Task Manager.
Steps to reproduce the bug
Create new app and derive App : PrismApplication
Expected behavior
No response
Screenshots
No response
NuGet package version
WinUI 3 - Windows App SDK 1.1 Preview 3: 1.1.0-preview3
Windows app type

[ ] UWP
[X] Win32

Device form factor
Desktop
Windows version
Windows 10 (21H2): Build 19044
Additional context
Request the information you need and ll provide.

@Sergio0694 this is what you were kind of seeing with ComputeSharp too, eh?

Sounds similar, yes. In my case I could narrow this down to specifically when a swap chain was rendering.
If that's not the case, the app closes just fine for me. I've opened an issue tracking that issue here: #7153.

Can you still repro this in 1.1 Stable?

Yes, still in 1.1 persist. Click on 'X' and the process becomes a backgrdound proc , which I can wintess in Task Manager.
It moves from the "Apps" list to the "Background Processes" list.

@marb2000 feel free to request whatever additional information you may need.

@marb2000 still able to reproduce it in version 1.1.3
Feel free to request any more specific context.

I am looking at this issue now. Btw, cool shader program. it is fun to investigate 😊

@pratikone That's awesome, and thank you! 😄
Also let me know if you need any help with the repro - that branch I shared is a bit out of date, so you might also want to just checkout to main (which is on WASDK 1.2 latest now) and see if the issue still repros there anyway once removing the workaround in App.xaml.cs to just force terminate the process. I can also help with that if needed 🙂

@Sergio0694 sure, i will check it out.

@Sergio0694  i have some lead.
I have a workaround (you can also continue the workaround you have).
the bug doesn't appear if your app is self-contained.
if you edit csproj of your app and set
<WindowsAppSDKSelfContained>false</WindowsAppSDKSelfContained>, this problem goes away. I am trying it on ComputeSharp.SwapChain.WinUI.csproj in your main repo.  This is good news. though switching to self contained will increase your package size as everything will be bundled together.
So, the bug only appears if you are using winappsdk as a runtime rather than a self contained binary. This is an interesting lead. Do you have a swap chain winui sample project which doesn't include compute sharp library ? It will simplify debugging for me.
(@bpulliam / @gabbybilka   we should add a swapchain sample winui app in winappsdk samples)

That is very interesting, good find! I'm not too concerned about the package size right now since I'm not currently publishing that app in the WinUI 3 version anyway (I'm only publishing the UWP version right now, as it's noticeably faster, consumes less resources, ships as an AOT binary and also has a much smaller package size), but I might as well leave the workaround in there for now to keep things simpler. I hope this discovery will help the investigation though! 😄

"Do you have a swap chain winui sample project which doesn't include compute sharp library?"

Unfortunately I don't, no 🥲
Getting a full swap chain sample running from scratch is a fair amount of work, and I've had no motivation to try to do that without reusing my library, given those samples are all meant to be a showcase of it. It's certainly possibly to do so (and the code in the library can also be used as a reference, in part), it's just not trivial work and it might require some time 😅

"Do you have a swap chain winui sample project which doesn't include compute sharp library?"

BTW, I had posted a sample with the Media Engine :
CMediaEngine.cs
(I have also done others with Direct2D, that I did not upload yet...)

You are right. in successive debugging sessions, the bug reproduced even when the app was self contained. I guess, in earlier times, VS killed the app a few times. Don't know how that happened. Definitely this issue appears whether app is packaged or self contained.
Also thanks for that media engine link. I will check it out.

@gegao18 can you take a look at this ? it looks like on using swap chain with D3D , memory is being kept alive which is causing the app to not quite during exit. WinUI correctly goes through all its cleanup rountine but the app continues to run.
easy repro : download https://github.com/Sergio0694/ComputeSharp, run ComputeSharp.SwapChain.WinUI sln.
It has the code to forcibly null ShadeRunner variable as well as a shutdown timer. You can comment those out for the bug to repo. Appears in both runtime as well as self contained configs.

Upon 'X' I witness this :

Upon 'X' I witness this today , does that help ?

Every piece of debugging information helps. Thanks for doing it. 👍🏾

With 1.4 the application deployed, X but the proc remains dormant.

Hi sigmarsson, do you have a minimal repro of this?  Sorry if I missed it somewhere in the thread.  Thanks!

Hi sigmarsson, do you have a minimal repro of this? Sorry if I missed it somewhere in the thread. Thanks!

@JesseCol the other similar issue mentioned had a repro repo here: https://github.com/microsoft/microsoft-ui-xaml/issues/7153#issue-1250217420
Though there wasn't clarity on if it was the exact same issue, it did get closed as a duplicate of this one.

Thanks!  I built and debugged this project locally: https://github.com/Sergio0694/SwapChainWinUI3Repro
From what I can tell, the CLR is keeping the process alive because there's a managed thread that's still running -- a thread that ComputeSharp had created.
I fixed the sample by doing this:

I added an Unloaded handler for ShaderPanel that calls StopRenderLoop().
I upgraded the project to WinAppSDK 1.4 (it was on an older version where the Unloaded events weren't always raised correctly).

The Xaml looks like this:
<winui:AnimatedComputeShaderPanel x:Name="ShaderPanel" Loaded="AnimatedComputeShaderPanel_Loaded" Unloaded="ShaderPanel_Unloaded" />

The C# is like this:
    public void StopRenderLoop()
    {
        ShaderPanel.ShaderRunner = null;
    }

    private void ShaderPanel_Unloaded(object sender, RoutedEventArgs e)
    {
        StopRenderLoop();
    }

As far as I know this is all by design, so I'm closing this one out.  Feel free to comment and let me know if I missed something.  Thanks!

Hey @JesseCol, thank you for looking into this! My AnimatedComputeShaderPanel already subscribes to Unloaded internally to stop the render thread (here), so I wonder if the issue there wasn't just caused by WinAppSDK using an older version where that event wasn't being raised, and that StopRenderLoop() call was not actually the one fixing the issue.
I tried again in my sample app (should just be able to F5 deploy it if you open the solution in VS 17.8), and I am not entirely sure about what I'm seeing. When I just run the app normally, I get this (which is fine):

Closing the window, the process immediately becomes this:

With just the runtime broker being left there, with 0.1 memory use. It seems to then just remain like that for a bit (say, 1-2 minutes), and then it eventually goes away. Is this the expected behavior? The thing is that I'm not entirely sure how this could be explained by eg. the CLR remaining alive because a thread was still running, because if that was the case, then why would the actual process for the app still go away, and the memory use go down to just 0.1 MB? 🤔

Agreed, if the process app process has exited, it doesn't sound like anything due to the CLR (or WinAppSDK / WinUI).  I'm sorry, I don't have much experience with RuntimeBrokers, so I don't have a good answer to your question.

I still experience that the process remains dormant upon 'X' and stays as "Background process" in Task manager. Could either procmon or windbg lend a clue and if so, what specific filter shall I apply to narrow this down ?

(522c.65fc): Windows Runtime Originate Error - code 40080201 (first chance) (522c.65fc): Windows Runtime Originate Error - code 40080201 (first chance) (522c.65fc): C++ EH exception - code e06d7363 (first chance) (522c.65fc): C++ EH exception - code e06d7363 (first chance) Microsoft.Web.WebView2.Core.dll!00007FFEC3CCCC92: ReturnHr(1) tid(65fc) 8007139F The group or resource is not in the correct state to perform the requested operation.     Msg:[winrt::hresult_error: The group or resource is not in the correct state to perform the requested operation.]  (522c.65fc): C++ EH exception - code e06d7363 (first chance) ModLoad: 00007fff9bc90000 00007fff9c0d5000   C:\WINDOWS\SYSTEM32\UIAutomationCore.DLL ModLoad: 00007fffd1500000 00007fffd15a3000   C:\WINDOWS\SYSTEM32\sxs.dll mincore\com\oleaut32\dispatch\ups.cpp(2126)\OLEAUT32.dll!00007FFFD31E470C: (caller: 00007FFFD31E49FA) ReturnHr(1) tid(65fc) 8002801D Library not registered. Microsoft.ui.xaml.dll!00007FFEBC06FFEB: 8001010E - RPC_E_WRONG_THREAD
... as windbg suggests the WebView2 isn't gracefully terminated.
            Window.Closed += (s, e) =>
            {
                var eventAggregator = Container.Resolve<IEventAggregator>();

                eventAggregator.GetEvent<TerminateEvent>().Publish(new object());
            };

_eventAggregator.GetEvent<TerminateEvent>().Subscribe(OnDispose, ThreadOption.UIThread);
        public void Dispose()
        {
            MyWebView.Close();
        }

        private void OnDispose(object x)
        {
            Dispose();
        }

how shall I shutdown it ?

May relate to https://github.com/MicrosoftEdge/WebView2Feedback/issues/2398 ?

however the webview related processes seem to have deceased, only the parent proc remains dormant.

Hi @sigmarsson -- your scenario doesn't seem to involve RuntimeBroker (like @Sergio0694 is seeing), is that correct?
Can you explain what the "x" process is in your screenshot?
Also, can you check to see if the behavior here is the same when running under a debugger vs not?  Thanks!

In the debugger, if you look at the threads that are still running, does it give any clues to why the process is still alive?
If you remove the WebView2 does the process still linger?  Adding @DmitriyKomin in case he's aware of cases where WebView2 keeps the process alive longer than expected.

@JesseCol - Thank you for your message. I confirm #7193 deserved to be closed. I removed the control WebView2 and problem has not vanished.


Don't print exceptions when run without -v

Which test failed why is good enough, we don't need all the exceptions unless the user requested that level of detail with -v.

I agree about AssertionErrors, but it seems like we'd still want exception stack traces related to framework/node issues or even non-assertion test errors (bugs, versus failing behavior).

Right!

We need to decide how we're going to handle various exception types that may or may not be a test failure versus test error.

On second thought, this is not very important, so I'm closing this. As we learn how to use FediTest, we may or may not reopen.


Unify syntax for styles, handlers and other props

Properties for a given component should be a single list, just like in elm-html. Currently only styles can be set for components, and event handler is a separate parameter. Ideally, the component API should be very familiar-looking:
text
  [ style stylesList
  , onPressHandler
  , numberOfLines 1
  ]
  "Press me"

This was done in #25.


handling the mul "language"

Wikidata is adding a "mul" language, to be used for labels, in particular, when many languages have the same label for an item.
I think that this means that using @en@rdfs:label will not work well, as many items will  not have an en label, but instead a mul label.
Is it possible to string this construct together, so that @en@mul@rdfs:label will get the en label if there is one and the mul label otherwise?  Or is there some other construct that would work (aside from the three-line construct that gets both and does a COALESCE)?

@pfps Can you give an example? What is the semantics of FILTER(LANG(?literal) = "en") when ?literal has the language tag @mul? And what is the motivation for this?

Wikidata is trying to cut down on the number of triples in the RDF dump.  One thing that contributes to the large number of triples is repeated labels, e.g., for https://www.wikidata.org/wiki/Q892 where you can see the repeated labels.  At https://www.wikidata.org/wiki/Q42 you can see the new way, with a mul "language" label (showing up under "default for all languages")  and many of the other languages just using that.  (The grey ones.)
What this means is that to get the English label for an item one has to do something like
OPTIONAL { ?x rdfs:label ?xLabelm.  FILTER ( lang(?x) = "mul") }
OPTIONAL { ?x rdfs:label ?xLabele. FILTER (lang(?x) = "en" )}
BIND (COALESCE(?xLabele, ?xLabelm) AS ?xLabel)
The issue is that there will often not be an "en" label if there is a "mul" label.
I'm not saying that this is a good thing at all.

I'm not saying that this is a good thing at all.

As I understand it, "mul" is being introduced because of Wikidata's internal reasons, with no other way forward found regarding Wikidata's scalability. I don't think anyone wanted to break compatibility with all the existing queries and tools, but this is the current situation. Also, my current impression is that the semantics haven't been fully figured out and it will depend on how Wikidata's editors will start to use this new feature in the software.
One way to handle this in QLever is to preprocess the dumps by copying "mul" labels to "en" labels where there isn't one already. This would restore compatibility with existing queries. (The weird thing is that you won't know which of "mul" labels will work in English and which won't, but apparently this is as designed. There may be some useful heuristics such as "copy the labels only if the writing system is Latin or the item is an instance of Q5 (human).". )
The other way is to try to do the same as WDQS and keep a representation of Wikidata's internal model. This is useful for "maintenance" queries by people and tools that edit Wikidata ("I want to see all the items where the en label and mul label disagree."). To keep query writing practical, QLever would need some new syntax. It might make sense to support the same label service as WDQS (for compatibility and for query performance).
(Now that I think of it, you could combine these two approaches by providing yet another language code "en without mul" for those queries that want to keep en and mul separate.)


[x/gamm][stableswap]: New tests to be added

Background
We should add the following tests for better coverage:

Add inverse relation mode for direct swaps in addition to just calcs, and checking that swap fee (squared) is charged
Add tests for spot price with scaling factor differences and exact spot prices
Add tests that swap amounts' precision does not decay as liquidity increases

Acceptance Criteria

All new and existing tests should pass

Hi @AlpinYukseloglu, can I handle this task?


Enhancement request: Option to overwrite the current customized prompt

Issue Description
According to the documentation:

posh-git will not replace the prompt function if it has detected that you have your own, customized prompt function

Ok I have a customized prompt for my general usage. But when I am doing git operations I am completely ok to have a posh-git prompt replacing my current prompt. It better fits to the current situation.
Would it be possible to have the choice with a switch or something like that, so that people with a customized prompt could have the posh-git prompt too?
In #624 you say that passing a parameter to a module is a bit unusual.
But currently, when I open a new tab/console to make git operations, I am doing this, which is a bit unusual too.
Remove-Item Function:\prompt
Import-Module posh-git

At least I have a not too annoying workaround :)

If you have a customized prompt, you can ask posh-git for what it would display by executing this in your prompt function:
& $GitPromptScriptBlock

Be sure to import posh-git after you have defined your prompt function in your profile.  See the README for more details: https://github.com/dahlbyk/posh-git#gitpromptscriptblock

Thanks for this second workaround :)

Would it be possible to have the choice with a switch or something like that, so that people with a customized prompt could have the posh-git prompt too?

Our solution to this is to invoke $GitPromptScriptBlock as shown above inside your custom prompt function.

Our solution to this is to invoke $GitPromptScriptBlock as shown above inside your custom prompt function.

Thanks @rkeithhill  for your tip! I improved my prompt function with it :-)
Function prompt {

    if (Get-Module -Name 'Posh-Git') {
        & $GitPromptScriptBlock
    }
    else {
        #My custom prompt comes here
    }
}

When I import the posh-git module, I have the posh-git prompt.
And when I remove the posh-git module from the memory I have my custom prompt back :-)


fix the wrapper selector, small fixes in subfield selectors

The changes needed to remove the selectors from repeatable field.
It goes with https://github.com/DigitallyHappy/backpack-pro/pull/45 and https://github.com/Laravel-Backpack/demo/pull/396

Tested, doesn't seem to break anything. I'll trust you on this one 😀 but please provide more context on what it fixes next time.

I didn't even know what to write, technically it's not a "fix" . It's updating the code to account that we have now the Dom element as input and the jQuery one as $input.
But yeah, that 👍


Remove use of 'package' as a variable name in install.js

package is a reserved word in strict mode. Therefore, node-sass cannot be packaged by a bundler that forces strict mode.
This pull request removes the package keyword, and renames the variable to pkg, as it is in scripts/build.js.

Thanks @ajp427 !


docs: update readme, fix typo in example

In one of the examples, req and res were transposed.
Otherwise, I just fixed up the precompilation example a bit to use async/await, but that is optional. I can revert if you'd like.

Coverage remained the same at 86.694% when pulling 0119f0514ee577c013d5c6cdad6f7f526f3ce5e9 on julianlam:master into a718ae5870d00d4d0f3c18e02d48420d811c21b8 on benchpressjs:master.

The reason I updated the precompilation pseudocode is because I found it confusing to implement it because the jst engine setting tripped me up, especially because NodeBB doesn't do it the recommended way.
I just converted it to example code that should (ideally) work out of the box once the views path is updated.


Add bounding box option to metrics module

It would be useful to be able to calculate local image metrics in addition to global ones. We could include a bounding box keyword argument in get_intensity_metrics and get_rgb_hists, which would allow the user to evaluate those functions on a local box of pixels surrounding a blob, rather than the entire image. This way we can reject noise on both a global and local level.

I totally agree that more local image metrics would be a nice addition!
Currently, we're using the regionprops function in scikit-image (http://scikit-image.org/docs/dev/api/skimage.measure.html#regionprops) to calculate various quantities on zoomed blob group images found using extract_blobs. But we don't have this functionality built into any functions in the metrics module.

We could include a bounding box keyword argument in get_intensity_metrics and get_rgb_hists, which would allow the user to evaluate those functions on a local box of pixels surrounding a blob, rather than the entire image

Just a thought, but another option would be to have get_intensity_metrics and get_rgb_hists take a list of image arrays instead of a list of files. Then users could just pass a list of the local images or a list of the full images, depending on their use case. That way the functions in metrics are separate from any image manipulation.


Tracking issue for the greatest data-refresh in the history of FTM

Also, the only data-refresh ever attempted.
Many of these can be broken into their own Issues as we go.

[ ] Assess the number of remaining cells in Sheets. (#776)
[ ] Are there non-lead moderators who might be willing to own the refresh process?
[x] @4kbt , enable outbound gmailing as<EMAIL_ADDRESS>[x] Add expiration columns to Combined.
[x] Add follow-up questions to the intake/update form.
[x] Craft manual refresh email
[x] Send manual emails
[x] Estimate expected response/takedown rate
[x] Create outbound repository address (#771)
[x] Determine the right time of day to send refresh email (Strawman: 8 am Pacific, weekdays)
[ ] Add Sheets dashboard entry to assess the number of entries that are 7, 14, 21, and 28-days stale.
[x] Prototype a mail-merge (#770)
[x] Test mail-merge in prod with dummy addresses (#773 )
[ ] Round up enough moderators
[ ] Identify emails that we can from the early 1500 entries / Ethan's database. (#772)
[x] Launch emails in blocks of ~1-200 in order to keep the rate manageable as we learn/err. (#774 )
[ ] Settle on expiration cycle (Strawman: warn at 14 days, warn again at 20 days, expire at 21) (#688)
[ ] Refresh all the entries.
[ ] Make refreshing an automated continuous process. (#775)
[ ] Win.

This is basically done. I don't think adding an automation step is likely/practical.


problem PHAF file PMID:21561865

https://curation.pombase.org/dumps/latest_build/logs/log.2018-05-24-22-57-12.phenotypes_from_PMID_21561865_phaf

PMID:21561865

The PHAF loader doesn't handle "residue(S423)" in extensions.  Let me know how urgent it is to fix that.

not hugely urgent, it was curated by Pombase staff, not community
https://www.pombase.org/reference/PMID:21561865
and it's only ~200 annotations. So just when you get to it.
Right now, go to bed ;)

Q should we allow this as an option on some phenotype terms in canto too?
or have I misunderstood what it is?

I did the Canto configuration for these extensions last month when I curated the paper (admin-only at present).

I'm using this in Canto a lot...how did we manage without it?

I have a question, do we have special syntax for the CDT repeat residues (see bottom one above).
I'm sure we discussed it, but I can't remember.....

found it https://www.pombase.org/gene/SPBC28F2.12
CDT_S2

syntax for the CDT repeat residues

https://github.com/pombase/curation/issues/139

The PHAF loader doesn't handle "residue(S423)" in extensions.

I've fixed the loader to handle that.  The results will be available on Friday.  In the meantime I'm running a Chado load locally to check for problems.

I think this is done, but while checking I noticed the residue column is missing: #943

#943

That was a typo.  It's: pombase/website#943


Verbnet corpus is out of date

The nltk data index (https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml) points verbnet to version 2.1. The latest verbnet definition is 3.2.
The latest version has updated frame descriptions that provide much more information about the phrasal structure. For example, the primary description of a frame from class future_having-13.3 in the latest version is NP V NP-Dative NP, describing the frame's structure as (noun-phrase, verb, noun-phrase(dative), noun-phrase) while in version 2.1 it just reads Dative.

@agodbehere, thanks for reporting this issue. I've verified that the existing verbnet 2 corpus reader breaks on verbnet 3 data, so both will need to live alongside each other in the corpus collection.
The next step is for someone to contribute a corresponding corpus reader nltk.corpus.verbnet3, which can hopefully share some of the existing code.
We'll need to support both for a while.

@stevenbird, what breaking case did you find for using the existing corpus reader with verbnet 2? I didn't run the test suite after updating the corpus, but for my use-case (requesting classids and frames), the existing corpus reader works just fine.

The problem exists with verbnet 3. We need a different corpus reader for
that.
On Wed, 16 May 2018 10:45 am Andrew Godbehere<EMAIL_ADDRESS>wrote:

@stevenbird https://github.com/stevenbird, what breaking case did you
find for using the existing corpus reader with verbnet 2? I didn't run the
test suite after updating the corpus, but for my use-case (requesting
classids and frames), the existing corpus reader works just fine.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nltk/nltk/issues/2015#issuecomment-389363521, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AADYbsgbjtFTKsyamRPG0OpJuWnF3UJ0ks5ty33DgaJpZM4Tzc0d
.

@amosleokim: thanks, that would be welcome!
You can see that we have verbnet (2) and verbnet3 data here.
I propose we add an entry for verbnet3 here
And then work out how to extend verbnet.py to support both verbnet and verbnet3.
How does that sound? We need to support both simultaneously, and (ultimately) deprecate verbnet 2.
We have an NLTK slack channel where we can discuss details if necessary. Thanks!

@stevenbird That sounds good to me! If you can send me an invite code to the slack channel, I'll hop on so we can get started on the nitty gritty.

@amosleokim I hope you received my invitation?

Any progress on this topic? I am trying to use verbnet for a research and the output I get from the classids methods seems weird.

Please see https://github.com/nltk/nltk/issues/2015#issuecomment-390826015

Thanks @stevenbird, the older version seemed to be the cause of the problem. I was able to manually download verbnet3.zip and read it with the reader for verbnet 2.1 that is in nltk.

@salompas Just like to check again, does the verbnet API in NLTK work with verbnet3?

It does work for what I am using it for. Let me show you my code:
import nltk
v3 = nltk.corpus.util.LazyCorpusLoader(
    'verbnet3', nltk.corpus.reader.verbnet.VerbnetCorpusReader,
    r'(?!\.).*\.xml')
v3.classids('add') # returns ['mix-22.1-2', 'multiply-108', 'say-37.7-1']

For that to work you need to download verbnet3 from here. Unzip this file in the folder ~/nltk_data/corpora~. When unzipped it should create a new folder ~/nltk_data/corpora/verbnet3which contains all the Verbnet3 definitions. Then you should be able to run the code above. Notice that for Verbnet 2 (the default) the codev3.classids('add')` only returns the first class (mix-22.1-2).
Since that is basically all I am using Verbnet3 for I have not tested the other APIs, but the classids method has been tested on maaany different words and they all work. I hope this helps!

@Salompas Hi, thank you for your solution! What version of verbnet3 is your 'verbnet3'? Is it version 3.3 or 3.2?

@Salompas Hi, thank you for your solution! What version of verbnet3 is your 'verbnet3'? Is it version 3.3 or 3.2?

Hey Songhee, it's been 2 years since I worked on this, so I do not know exactly which version I had.


Remove deprecated oci and qualified-device APIs

partial fix for #206

@klihub @elezar I left registry APIs intact as they require a bit more work. I'd prefer to remove them in a separate PR.


Cannot install dependencies

Hi team,
Thanks for the interesting program. I am having trouble installing the dependencies using the provided requirements_conda.txt.
This is the log:
conda install -c conda-forge --file requirements_conda.txt
Channels:
 - conda-forge
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - kaleido==0.2.1=pypi_0
  - plotly==5.6.0=pypi_0
  - scripttest==1.3=pypi_0
  - tenacity==8.0.1=pypi_0

Current channels:

  - https://conda.anaconda.org/conda-forge
  - defaults

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Do you know how to solve this problem?
Thanks!

Actually you just need to install the packages listed in "Requirements".

Thanks!


Ships To filter on Store Page Filtering Out Valid Listings

On main screen by default (Any Country) I see multiple listings.

If I choose a specific country they all go away.

It appears that @drwasho has somehow saved physical goods with no shipping information?
That should get rejected, unless he's using one of his experiments to populate his listings.
@cpacia do we validate that on the server? The server should be rejecting any listings with a type of physical good but no shipping options.

This feature works correctly with listings that have the expected data.

This has been fixed on the server.


Add text color to nk-error/success messages

Wondering if we can add text color to the messages. I've see a few sites with dark background and the message color is inherited so it's not visible/white on the light message background.

Thanks for taking care of this  🙌


Add option to Cesium3DTileset to set custom appearance

Currently it seems not to be possible to to change appearance of a PointCloud. There exists already a PointAppearance but this will be always set fixed in Points3DTileContent and it is not possible to pass parameter to this appearance. But we need ability to attach other shaders to a PointCloud and to modify point size (PointAppearance provide an option to modify point sze, but I didn't found a way to access PointAppearance from an app)

Thanks for the input.  Have you seen the declarative styling work-in-progress for 3D Tiles: https://github.com/AnalyticalGraphicsInc/3d-tiles/issues/2
What are you going to base point size on?  One of the point's attributes, e.g., temperature, etc.?
Also, have you already built a project on 3D Tiles?  If so, let me know - https://groups.google.com/forum/#!topic/cesium-dev/2NlWw_vqVVs

Currently we just need point size to increase general appearance. E.g. if you are really far zoomed in you have a point each 20px. If a pixel is just 2px you don't really see them good. Another nice effect of increased pixel size: You can decrease number of necessary loaded points. E.g. if you have a 4K Display you  need 1920*1080 points with a size of 2 pixel to fulfill screen. Each point currently needs 15 Bytes - so you already have to load a ~30 MB (with gzip ~26,5MB) Point cloud to hide terrain beyond it. With a pixel size of 4 Pixels you need only 7,5MB. But indeed to adjust fixed point is just a possibility to increase visual appearance. A better idea is an adaptive point size (see examples of Potree)
@declarative style: Didn't saw it until, but not sure if it makes sense on point clouds. It's a good idea for 3D Models, but with point clouds you have massive data, so you need to calculate all important things in a shader instead of with CPU. So for us it is just important to have ability to use different shaders to render a point cloud in different styles. Just take a look on examples of Potree to see what a user could do.

The base implementation of point clouds will have adaptive point size based on distance, and hopefully occlusion culling based on that.
For declarative styles, when used with point clouds, I expect that we will generate GLSL from the AST so everything will be done in a shader, see https://github.com/AnalyticalGraphicsInc/3d-tiles/issues/2

[ ] GPU implementation for tile formats that would really benefit, e.g., point clouds

Declarative styling is now implemented for point clouds; it is possible to set the point size.


Abstract a lot of code, clean up and improvments

This PR serves to abstract a lot of code, clean up and add some improvements

[x] Rewrite command & event loading
[x] Rewrite logger class

Requesting a review: @kyranet @Dev-Yukine @YorkAARGH
I now feel this PR has done enough/served it purpose to rewrite/refactor/abstract/improve/clean the code, This has been a fun time, Doing something for a bot that doesn't use a framework is a little refreshing. I love writing my own things and not using framework methods, Misaki is a good example of how good a discord bot can be without a framework.

@kyranet

CommandStore, EventStore and many other things (Command and Event aswell) are very similar to Klasa's old code with the RAM optimization PR patch applied. Nonetheless it's under a license that is not included.

In fact, The EventStore was made by me. Originally it came from PenguBot where it had no loading stuff in the class but was the same (except it actually extended Base a class with some basic properties like client but nothing else) which I obviously removed that in this implementation but its really not a copy of Klasa's ways, Its inspired but it no way is it Klasa's code.

There's also a lot of things that are strongly inspired by Klasa, for example, the misakiReady event (from klasaReady), our stores and pieces (without the Store and Piece abstract classes).

Funny thing is, I did this before I knew klasa was a thing

A lot of our style is applied in this PR, in case @YorkAARGH isn't aware, this code is way much better than the previous, yes, but it's so similar to Klasa, porting it would literally take hours as the way we handle the pieces, stores, loading, initialising, reloading, loading... are all identical.

I'm quite sure York is aware but, The whole point of this PR was to make Misaki more understandable. That is very hard when you have A Client class in one file that handles the loading over commands & events with a directory scanning module (klaw) A couple of weeks ago I was whing about York using klaw you suggested the use of fs-nextra scan (Got to admit I had no clue on how to use this function) so me being me I looked at Klasa's store walk code (I knew it was used here) and used it. As I most likely would've messed up something if I didn't use a snippet so I did. But I really can't regret that as from the moment I say klaw in guidebot I disliked it so the use of a module we already have and use is put to use 🎉

Another thing that I highly dislike about this code is the amount of uncaught promises, there are many message.channel.send that aren't either awaited nor has a .catch callback. What does this mean? This leads to unhandled promise errors.

I can totally agree

Another thing that I have noticed is that this code also lacks of documentation. Reducing lines of code isn't always a good thing, if it's undocumented, it's hard to read and understand what's going on. While the old code (and GuideBot) are highly documented, Misaki's seems to get less and less documentation. This leads to the project being harder for an external contributor to understand what is the code doing.

Yeah I can also agree on this, I'll work on adding documentation in the future + JSDoc

I must include that I have talked privately with you about MisakiConsole being ripped off from KlasaConsole (from Klasa 0.5.0) with the difference that colours are rather... hardcoded. But the design is very similar to our KlasaConsole rewrite made by bdistin. (#227 in dirigeants/klasa). I have, however, kindly asked for our license to be put in the top of the file. Instead, you renamed a method, removed the shard support, and also removed the notice about the code being inspired.

Yes, we spoke, I don't see how this can really be a rip off honestly... Everything was off the top of my head except write, Now yes I had to look at snippets of code, no I didn't only look at klasa... I spent about 30 minutes reading a couple guides on the Console class & NodeJS stream's interesting, to say the least, but I really can't say this is a rip-off, I'd say write was inspired by KlasaConsole but that's it honestly

There's also a lot of things that are strongly inspired by Klasa, for example, the misakiReady event (from klasaReady), our stores and pieces (without the Store and Piece abstract classes).

Funny thing is, I did this before I knew klasa was a thing

Funny thing is, that almost all of the code used here in EventStore/Event (and more) is line for line (variable name for variable name even), identical code to a mixture of current and past implementations of klasa's version and by extension klasa's Piece/Stores. A coincidence you don't get without copying.

@bdistin I've been looking at Event, Command pieces in klasa for the last 15 minutes, I can't find anything that even hints that them classes are a copy. Considering Command was a thing before I PRed that's a little hard to believe, the same variable names? So calling a variable name or file or client is copyright now? other then walkFiles the code is my own

I realize that I am not heavily involved in the development of Misaki, but york requested some code review so I will give it my best shot.
My main criticism of this project lies in ineffective application of objects and object prototypes.  While I will not take the time to make a fully-fledged, line-by-line review of this PR, I can generalize my findings and hopefully provide some insight on how to better utilize Javascript.  I realize that a lot of these issues existed prior to this PR, so don't take this as a criticism of only the OP.

Unnecessary instance properties/methods: while browsing the code in this PR, I noticed numerous classes that had instance properties that were constant or otherwise completely unrelated to the state of the class instance.  Class properties and methods should be reserved for things that either rely on or modify the instance state.  client.methods is a prime example of this, where every property is not reliant on the state of the client or any of it's properties.  Most people define methods on the client for convenience, which is what I imagine happened here.  However, it's is incredibly simple to just import the static util methods you require.  Oftentimes, these properties/methods should in fact be defined on another prototype, which lends the same utility in a more idiomatic and accessible fashion.
This is by far my most significant criticism: throughout every command there seems to be at least one access of some client property that is actually static.  Besides being bad practice, you could see increased memory usage (from declaring multiple copies of the same static information) and bug occurrence (from inconsistent object mutation across instances).  My suggestion would be to enable the eslint rule class-methods-use-this; while it is not a rule that should be followed religiously, it will help you discover where you can cleanup some code and will cause some pause for thought before implementing new methods.

Incorrect instance property/method location: stuff like message.settings is immediately an example of such a problem.  A message cannot have settings, and so it's strange to see a property of them defined on the prototype.  Again, this frequently occurs in an effort for convenience, but is in fact confusing and could easily be defined on a more appropriate prototype.

Basically, my suggestion is as follows.  Don't be afraid to declare new classes as you need them; if you find yourself doing a lot of boilerplate to implement a feature, a class may be the thing for you.  For example, reminders might be something to consider writing a simple class for: perhaps you could consider defining Member#reminders as a store of reminder classes.  However, really think about why you're adding that particular instance property/method to that particular class and if it's not immediately related to what that class is about, perhaps you're better off finding another class (or making a new one).
Personally, I've found that defining a Command class (accepting a message in the constructor) with all your validation and command instance validation methods has been the most effective solution.  This also leads to your command execution being more object-oriented and thence friendlier.
When looking at a method signature, oftentimes the prototype that it should be defined on is accepted as one of the first few parameters.
On a personal note, I believe this PR achieves very little in the way of significant change.  Most of the work has been on the cleanup side of things, and calling this an abstraction is somewhat of a misnomer.
Other suggestions:

The console dependency is entirely useless (and the internal console module is another example of point 1).  If you want decent logging, I would recommend switching to an async logging library like winston rather than one that just checks for console methods.
I noticed that you're throwing custom error classes, which is great error handling practice.  However, there are some cases where you're using if (!condition) return message.response(undefined, 'blah');.  I'm unsure what the difference is, and I would recommend switching everything to throw.

Please fix that method so you don't have to send undefined as the first parameter.  It hurts.

You should consider abstracting the argument system a little bit more.  I see that you have awaitReply, but numerous places duplicated code in the argument resolution.

@appellation you're better to file a issue as half of this is not related to this PR what so ever

This PR being titled "abstract code, cleanup, and improvements", I assumed that my comment was well within scope.


vcpkg macros not defined when using v140 toolset in vs2019

Describe the bug
The macros $(VcpkgRoot) is not defined.
When the toolset is v140 the $(VcpkgTriplet) macro is appropriately defined to be x64-windows-v140 by my vcxproj project file, and correctly propagates as such when looking at macro settings inside visual studio; however build fails. If the toolset is  v141 or v142  extra variables are automatically defined including $(vcpkroot), appropriate paths are selected and build succeeds normally.
The v140 choice does not define any other Vcp* named macros - in particular to root for the vcpkg.
the content of the v140 triplet is
set(VCPKG_PLATFORM_TOOLSET v140)
set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic)
Environment

OS:windows 10
16.2.2

To Reproduce
Steps to reproduce the behavior:
0. Have VS set up with all three toolsets v140 v141 v142

Edit the project files as recommended (see attached)
Examine the defined macros in the build system when different toolsets are selected
Try to build the project with the different toolsets (having first installed via vcpkg the correct libraries - the build succeeds for 142, 141 but fails on a path error for 140

Expected behavior
the build should succeed for all three toolsets
Failure logs
TestVspkg.log
Additional context
TestVspkg.vcxproj.txt

add tag  or
$(VcpkgRoot)\$(VcpkgTriplet)\lib;$(VcpkgRoot)lib;%(AdditionalLibraryDirectories)</AdditionalLibraryDirectories>
add tag
$(VcpkgRoot)\$(VcpkgTriplet)\include;$(VcpkgRoot)include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>

Looks good, but how does this differentiate between 140, 141,142. Is it missing somewhere?
VcpkgRoot is not defined in 140 and is in the other two.
Terry
On 29 Aug 2019, at 14:31, Voskrese <notifications<EMAIL_ADDRESS>wrote:
add tag or
$(VcpkgRoot)$(VcpkgTriplet)\lib;$(VcpkgRoot)lib;%(AdditionalLibraryDirectories)
add tag
$(VcpkgRoot)$(VcpkgTriplet)\include;$(VcpkgRoot)include;%(AdditionalIncludeDirectories)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vcpkg/issues/7957?email_source=notifications&email_token=ABQWN6YA5HZW33KY5RSOSNLQG66SHA5CNFSM4IR6EIAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5OJ3SI#issuecomment-526163401, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQWN67I6ZMEGHDKDVPWMTDQG66SHANCNFSM4IR6EIAA.

$(VcpkgRoot)

most likely I did not understand the question, do I need to reassign TOOLSET?
but why if you use set (VCPKG_PLATFORM_TOOLSET v140)
if you do not have MSVC 2015 installed, then why indicate v140?

Sadly, I have most versions of visual studio installed. 2008 and 2010 and 2015 for python extensions; 2013 for certain cuda based apps. This is the windows developer way... simpler for the user.  Older software still works.
Fortunately visual studio 2019 handles all this easily. I can build them all inside 2017/2019, just changing the toolset and windows target. I routinely use 2015/v140 /8.1 to build some of my console programs inside VS2019.
I followed the vcpkg example where  by modifying the vcxproj file visual studio selects and defines the triplet name to x64-windows-v14x as I change toolset between 140 141 and 142.
The V141 and v142 settings in VS2019 both work, select correct libraries and build; the v140 setting should also work I suppose. But it falls over; the most obvious error seems to be that the paths seem not to have the vcpkgroot inserted and the macro and several others are not defined. They are defined for the other two toolset that work.
I hope this makes sense.
Terry
On 29 Aug 2019, at 16:08, Voskrese <notifications<EMAIL_ADDRESS>wrote:
most likely I did not understand the question, do I need to reassign TOOLSET?
but why if you use set (VCPKG_PLATFORM_TOOLSET v140)
if you do not have MSVC 2015 installed, then why indicate v140?
As I said, I do have the v140 toolset installed (and vs2015 although I am using and building with the toolset in vs2019).
I indicate v140 because I want/need to build with v140. Just as I might want/need to build with clang, or an XP_compatible toolset...
One cannot deduce the toolset used in a build from the version of visual studio one is using. The options are limited by the toolset installed (usually installed by installing different visual studio versions or SDKs; there is a special 2008 toolset maintained for python 2.7).
If the code assumes it is being called by vs2015 when toolset v140 is used I can see potential for malfunction?

v120
v140
v141
v142
should work through TRIPLET
vcpkg integrate install
automatically integrated into the project during assembly

Would you like me to try v120 from vs2019.  I would need to create triplets and install the libraries first. I have run
vcpkg integrate install
Should I run it again; as I said the integration works fine for 141 and 142.
Terry
On 29 Aug 2019, at 22:40, Voskrese <notifications<EMAIL_ADDRESS>wrote:
v120
v140
v141
v142
should work through TRIPLET
vcpkg integrate install
automatically integrated into the project during assembly
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vcpkg/issues/7957?email_source=notifications&email_token=ABQWN6YAD6JLSLZUP2Q5GBLQHAX4DA5CNFSM4IR6EIAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5PYAUI#issuecomment-526352465, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQWN67A7OUYMXWYMLBUAVLQHAX4DANCNFSM4IR6EIAA.

honestly I do not understand the essence of the issue
no headers found in your project
I already wrote a solution
about compiler compatibility, this is a separate issue

Just to say I did do
vcpkg integrate install
at the beginning.
I would be delighted if I have simply misunderstood something. The tool seems fantastic.

Thank you for your attention!  Who said anything about headers?
My situation is that, in vs2019 with a new console project with a simple boost dependency, and with boost installed via vcpkg I cannot successfully build it if I set my toolkit to v140.
The error seems to be that the automation fails to inject some variables into the ms-build build process. In particular the root directory for the vcpkg is not set.
The error does not occur in vs2019 if the setting is v142 or v141.
To the best of my knowledge the triplets used for installation etc are set up appropriately. One for each toolset. Is that a mistake?
Apologies if it is still not clear.
On 29 Aug 2019, at 23:00, Voskrese <notifications<EMAIL_ADDRESS>wrote:
honestly I do not understand the essence of the issue
no headers found in your project
I already wrote a solution
about compiler compatibility, this is a separate issue
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vcpkg/issues/7957?email_source=notifications&email_token=ABQWN6ZVEEIN44AGWRJ7AI3QHA2G5A5CNFSM4IR6EIAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5PZXTI#issuecomment-526359501, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQWN67MHMMFAZJSRWEMUDDQHA2G5ANCNFSM4IR6EIAA.

Failure logs
TestVspkg.log

TestVspkg.cpp(21): fatal error C1083: Cannot open include file: 'boost/filesystem/operations.hpp': No such file or directory

in General, I don't know what else to recommend, maybe someone else will tell you

Yes, that error breaks the build. The other choices (141 142) instead prepend a path built out of the root directory and the triplet information to this path and run correctly.
I don’t know the design of the code however it seems to me likely that somewhere in vcpkg there is some vs2015 specific code that is (wrongly) triggered by the v140 option which VS2019 does not like. The corresponding VS2017 code triggered by the v141 option might be close enough that the vs2019 does the anticipated thing.
Thinking about it, I am not at all sure how the default integration works in vcpkg when one has multiple versions of vs installed. Does vcpkg autoinstall a different version of a package for each version of VS present? That would be cool.  If so I wonder what is the naming convention and path convention.
Thanks for your interest!!
On 29 Aug 2019, at 23:55, Voskrese <notifications<EMAIL_ADDRESS>wrote:
Failure logs
TestVspkg.loghttps://github.com/microsoft/vcpkg/files/3553998/TestVspkg.log
TestVspkg.cpp(21): fatal error C1083: Cannot open include file: 'boost/filesystem/operations.hpp': No such file or directory
in General, I don't know what else to recommend, maybe someone else will tell you
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vcpkg/issues/7957?email_source=notifications&email_token=ABQWN64QYRNSMHCCJ66EZA3QHBATBA5CNFSM4IR6EIAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5P6BKQ#issuecomment-526377130, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQWN6ZHWJPT6PLQJ7NUQZ3QHBATBANCNFSM4IR6EIAA.

I have exactly the same problem and it is very frustrating....

I've simply defined and set VCPKG_ROOT in my environment, and that works.

@terrylyons: Where is or MSBuild for v140 located? You can check <vcpkgroot\toolsrc\src\vcpkg\commands.integrate.cpp> how the integration basically works. It could be that it is not properly installed for the v140 toolset when the others are available? The idea is that it installs a redirection in one of the MSBuild files which loads the file in %LOCALAPPDATA%/vcpkg/vcpkg.user.targets which itself redirects to a file in the  <vcpkgroot\scripts\buildsystems\msbuild\vcpkg.targets>
The variable which gets installed is:
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <!-- version 1 -->
  <PropertyGroup>
    <VCLibPackagePath Condition="'$(VCLibPackagePath)' == ''">$(LOCALAPPDATA)\vcpkg\vcpkg.user</VCLibPackagePath>
  </PropertyGroup>
  <Import Condition="'$(VCLibPackagePath)' != '' and Exists('$(VCLibPackagePath).targets')" Project="$(VCLibPackagePath).targets" />
</Project>

so if your project does not have/show you the variable VCLibPackagePath it is not integrated into the v140 toolchain.

This looks helpful. I have not had time to go back, but I will. I find that vcpkg is fundamentally changing things for me – dramatically reducing the number of prop files I need – for boost etc. But the lack of different platforms is a big issue so I hope we can get to the bottom of it.
In case you see something my VS2015 toolkit in VS2019 comes via the installation of VS2015 enterprise.
I currently have Microsoft Visual Studio Enterprise 2015
Version 14.0.25431.01 Update 3
And among the many work packages those that seem potentially relevant are
Visual C++ 2015   00322-80000-00000-AA535
Microsoft Visual C++ 2015
Clang with Microsoft CodeGen   14.0.25516
Clang with Microsoft CodeGen
Intel® Parallel Studio XE 2019 Update 4 Composer Edition for C++ Windows*   Package ID: w_comp_lib_2019.4.228
Intel® Parallel Studio XE 2019 Update 4 Composer Edition for C++ Windows* Integration for Microsoft* Visual Studio* 2015, Version <IP_ADDRESS>, Copyright © 2002-2019 Intel Corporation. All rights reserved.

Other names and brands may be claimed as the property of others.
Intel® Parallel Studio XE 2019 Update 4 Composer Edition for Fortran Windows*   Package ID: w_comp_lib_2019.4.228
Intel® Parallel Studio XE 2019 Update 4 Composer Edition for Fortran Windows* Integration for Microsoft Visual Studio* 2015, Version 19.0.0052.14, Copyright © 2002-2019 Intel Corporation. All rights reserved.
Visual Assist   10.9.2341.2
For more information about Visual Assist, see the Whole Tomato Software website at http://www.WholeTomato.com. Copyright (c)1997-2019 Whole Tomato Software, LLC

Visual C++ for Cross Platform Mobile Development (Android)   14.0.25401.00
Visual C++ for Cross Platform Mobile Development (Android)
Visual C++ for Cross Platform Mobile Development (iOS)   14.0.25401.00
Visual C++ for Cross Platform Mobile Development (iOS)
Visual C++ for Linux Development   1.0.5
Visual C++ for Linux Development
From: Alexander Neumann<EMAIL_ADDRESS>Sent: 11 September 2019 12:03
To: microsoft/vcpkg<EMAIL_ADDRESS>Cc: terrylyons<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [microsoft/vcpkg] vcpkg macros not defined when using v140 toolset in vs2019 (#7957)
@terrylyonshttps://github.com/terrylyons: Where is or MSBuild for v140 located? You can check <vcpkgroot\toolsrc\src\vcpkg\commands.integrate.cpp> how the integration basically works. It could be that it is not properly installed for the v140 toolset when the others are available? The idea is that it installs a redirection in one of the MSBuild files which loads the file in %LOCALAPPDATA%/vcpkg/vcpkg.user.targets which itself redirects to a file in the <vcpkgroot\scripts\buildsystems\msbuild\vcpkg.targets>
The variable which gets installed is:

<VCLibPackagePath Condition="'$(VCLibPackagePath)' == ''">$(LOCALAPPDATA)\vcpkg\vcpkg.user</VCLibPackagePath>

so if your project does not have/show you the variable VCLibPackagePath it is not integrated into the v140 toolchain.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vcpkg/issues/7957?email_source=notifications&email_token=ABQWN6YEGCEV3TNBPRWMIMTQJDF6ZA5CNFSM4IR6EIAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6ODQBQ#issuecomment-530331654, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ABQWN62JHS7YVAXK5ZSS6LDQJDF6ZANCNFSM4IR6EIAA.

This should not be a problem now, My VM have vs2019 16.7.14 with vs2015 toolset installed, and the project build with v140 successfully. Please get latest source and try again, please report a new issue if the problem persists for you.


Reactify the ErrorPanel

This uses react to render the component rather than doing it with calls to the DOM. It makes it far more straightforward.
However, as mentioned in #13 I could only do this by attaching it to the widget node.
The only way I could get this to propagate updates appropriately was to use the text setter, which seems somewhat clunky… but this is at least a decent first pass.

oh I'm going to push to reactify and make a new pr because the css broke


fix: 🤔 add "Menu" in syn-prio-nav to translatable strings and deprecate priority-menu-label

Component
syn-prio-nav
Description

Currently the word menu is not part of the translation files. Therefore everyone would have to translate theirself by adding a prop. Instead we should use the translation files to make this translatable, so that it can switch depending on lang=de etc.
In addition the attribute should be deprecated, so that we can remove it in the next major release.
Acceptance criteria

[ ] All DoD items successfully completed
[ ] syn-prio-nav uses translations (german and english)
[ ]  priority-menu-label is deprecated
[ ] deprecation info shows up in types and Storybook and informs that it will be removed in the next Major Release (@deprecated)

I am still not sure if it is really wise to go this way. When using fixed strings from the translation files, we are bound to use en and de only (currently). This means that a user that has an already translated main application using his locale "cn" would see an application that is completely rendered in chinese - except for the string "Menu" that we hard coded. One could argue that we could just provide translations for chinese. However, this does not scale as there are so many languages out there and we do certainly not want to maintain a large pack of language files.
My suggestion would be:

Leave the property alive to make it possible to translate into languages we do not support.
Add the "Menu" string to the german and english translations files
Use this static translation as a default if no priority-menu-label attribute was set.

This would it make possible for us to automatically set Menu string in each languages we officially offer support for and override on a per need basis for foreign languages we do not have any support for.
@mariohamann
@kirchsuSICKAG
@klaarseSICKAG

But the question was not if they need this for translations but if they would need the label for a different wording instead of "Menu". I can't really read from your answer, what the result was in this regard.
Changing the translation behaviour is a completely different issue and would have a much broader impact, as you said. We're currently investigating in Solid to allow instance specific i18n, which would go into that direction.

After discussion with @mariohamann and the rest of the dev team, we agree that we need a holistic solution for the underlying problem for all future and present components. Therefore, we chose to implement this fix as suggested by Mario and strife for a global solution in another ticket.


Tangent decoding support

Currently only tangent calculation on import is supported.
I'm not sure about the state of the "DRACO_TRANSCODER_SUPPORTED", but should not be an issue when using it for unity...?

Yeah, this is not supported in current DracoUnity versions and likely won't be in 5.0.0 either.
What's your use-case? Would you know where to obtain test data for development?
Thanks

Just working on UnityGltf, and found it when i tested some models :)

This is now supported by Draco for Unity, the successor of this repository/package.
Have a look at Draco.DracoDecoder.DecodeMesh overloads with the attributeIdMap parameter.


Overloaded Prop.collect behaves surprisingly with Seq[T] argument

Summary
Prop.collect is overloaded. There is a "simple" version

collect(T)(Prop): Prop

and a "functional" version

collect(T=>P)(implicit P=>Prop): T=>Prop

Using collect in properties based on generators that produce Seq[T] values (for example Gen.pick) can lead to potentially surprising behavior. In the example detailed below, the compiler selects the functional version over the simple version when the argument is of type Seq[T] with a Type mismatch compiler error.
This function version selection occurs because Seq[T] extends PartialFunction[Int, T] which matches (in part) the signature  collect(Int => T)(implicit T => Prop).
Once the issue is understood, there are straightforward workarounds.
Improved documentation in the source file for the two collect methods seems a straightforward way to mitigate potentially complex and frustrating diagnosis by your typical developer (assuming she gets as far a clicking through to the Scalacheck sources, or pulling up the equivalent of Intellij's quick documentation).
Removing the overload completely by renaming the functional version is a more ambitious solution.
Details
I was using collect in a property along these lines
 val prop = forAll(Gen.pick(2, (0 to 9)) { ns => ns.size == 2 }
which I then augmented with collect
 val prop = forAll(Gen.pick(2, (0 to 9)) { ns => collect(ns) { ns.size == 2 }}
At which point I get this mysterious (to me, at the time) compiler error
Type mismatch: expected: Int => Prop, actual: Boolean
Eventually I figured out what was going on, but it took a while, mostly because my eyes start to glaze over when I end up looking at a declaration like:
def collect[T, P](f: T => P)(implicit ev: P => Prop): T => Prop = ...
The problem is that pick is type Gen[Seq[T]], and Seq[T] extends PartialFunction[Int,T]. This explains the compiler's decision to use collect(Int=>T) instead of collect(Seq[T]). (At least in part, I wonder why it doesn't complain about ambiguous overloading. But that is a topic for another day.)
Workarounds are relatively simple, such as

Cast to Iterable (which is not a function type)
 val prop = forAll(Gen.pick(2, (0 to 9)) { ns => collect(ns.toIterable) { ns.size == 2 }}

Convert to String
 val prop = forAll(Gen.pick(2, (0 to 9)) { ns => collect(ns.toString) { ns.size == 2 }}

Rewrite to intentionally use functional collect(Seq[Int] => Boolean)
val prop = forAll(gen)(collect{ns: Seq[Int] => ns.size == 2})

So the issue is ergonomic,  not functional. Surprising and unexpected behavior like this can be frustrating and a time waster.
In my limited experience as a Scalacheck user, my preference is for  the collect(T) form over the collect(T=>P) form, in part because the former allows more manipulation of the collected values.
Suggestions

Eliminate the overload by renaming collect(T=>P)(implicit P=>Prop)
Provide more guidance in the source documentation. Current doc is skimpy:
  /** Collect data for presentation in test report */
Documentation could be enhanced to note the potentially surprising behavior when collect's
argument is a Seq. This would have saved me quite a bit of time, since Intellij certainly took me
right to that line in Prop.scala.

I know I experienced this trauma, but completely forgot about it.  All I know is I always do ns.toString in Prop.collect to avoid the error.   It feels great to know my enemy.
I don't use the Prop.collect(T=>P) version.  There's no example of using it in ScalaCheck: The definitive guide.
I wonder how one would use it that way?  Is such usage well-known?  If proposed, I'd second the motion to rename or even drop it if it's not commonly used.  I'd argue Prop.collect is too useful to not have this usability problem (or as @pford19 rightly calls "ergonomic") be resolved.


Tracking result

Hello, thank you for your excellent work.
I am puzzled by the tracking results output in the process of reproduction. The tracking results generated by the same sequence on MOT17 data set are the same. Excuse me, where did I set the parameters incorrectly?
Hope to answer, thank you

The tracking results of the same sequence in the above figure are the same size, and the evaluation results are the same.

It's no problem, the same sequence, the same image, so the detection and tracking results are the same.

Hello, I wish you all the best and thank you again for your reply.
When I used the method based on MATLAB to evaluate the reproduced results, the following situations appeared, and the results in the same sequence showed the same results. Is this a problem with my evaluation method or other reasons?
Hope to reply, thank you.

Hello, I wish you all the best and thank you again for your reply.
When I used the method based on MATLAB to evaluate the reappeared results, the following situations appeared. The indicators evaluated are not quite the same as those in your paper. Is this a problem with my evaluation method or other reasons? What evaluation method do you use?
Hope to reply, thank you.

Hello, I wish you all the best and thank you again for your reply.
When I used the method based on MATLAB to evaluate the reappeared results, the following situations appeared. The indicators evaluated are not quite the same as those in your paper. Is this a problem with my evaluation method or other reasons? What evaluation method do you use?
Hope to reply, thank you.


What's the difference between .table() and .alterTable()?

I believe this migration works regardless of whether I use .table() or .alterTable() is there a difference?
exports.up = function (knex) {
  return knex.schema.alterTable('account', (table) => {
    table.timestamp('date_deleted');
  });
};
exports.down = function (knex) {
  return knex.schema.alterTable('account', (table) => {
    table.dropColumn('date_deleted');
  });
};

I see .table() in the documentation but I don't see .alterTable described anywhere.

they are aliases: https://github.com/knex/knex/blob/e37aeaa31c8ef9c1b07d2e4d3ec6607e557d800d/lib/schema/builder.js#L57 this issue should be moved to documentation repo or .alterTable could be deprecated to not have so many unnecessary aliases.

Thanks for the clarification it's definitely weird that an alias is in the documentation but not the actual method.

they are aliases:
https://github.com/knex/knex/blob/e37aeaa31c8ef9c1b07d2e4d3ec6607e557d800d/lib/schema/builder.js#L57
this issue should be moved to documentation repo or .alterTable could be deprecated to not have so many unnecessary aliases.

I'd prefer keeping alterTable if possible, makes it more explicit

I'd prefer keeping alterTable if possible, makes it more explicit

True and agreed +1

I add alterTable in the documentation here: https://github.com/knex/documentation/pull/352
This will be close where the documentation is merged.


Creating VFX Graph

I have managed to import the pointcloud with pcx and converted the container type to VFX. But the model is not being shown when  I tried to create a pcache file. how can I fix this?

It doesn't use a pcache file. Please check the vfxgraph-example branch that contains a sample VFX Graph.
https://github.com/keijiro/Pcx/tree/vfxgraph-example

I'm closing this issue now. Please feel free to reopen it for further problems.


enclosed type of ExitStatus::Signaled u8 vs. i32 / c_int?

I just noticed, that ExitStatus::Signaled contains a u8, which is somewhat weird, because when comparing to libc constants you have to cast them. i.e., I just wrote code like this and it was very intuitive.
if let ExitStatus::Signaled(signum) = *status {
  (signum as i32) == libc::SIGABRT
}

Please consider changing this to a i32 in the next version.

I'm a bit wary of using i32 here because there's no guarantee that the signal numbers will be precisely i32 (or any particular type) on all systems - and I'd like to avoid exposing the libc types.
Perhaps ExitStatus should provide a function to test for a particular signal:
    /// True if the subprocess was killed by a signal with the specified number.
    pub fn is_killed_by<T: Eq + From<u8>>(self, signum: T) -> bool {
        if let ExitStatus::Signaled(n) = self {
            let n: T = n.into();
            return n == signum
        }
        false
    }

You could then write your if lwr as just status.is_killed_by(libc::SIGABRT).

That new API is even better :) thanks!


Bundle release of new CLI into raw release archives

For platforms like Windows where we don't offer higher-level packages, removing the influx CLI from our release archives will be a nontrivial UX regression. See what it'd take to write a script that downloads a pre-built CLI release, for use as part of the goreleaser process. The influx binary should end up included in all the .tar.gz and .zip archives produced by the build.

A thought: influxd depends on influx-cli for some CLI utilities and integration tests. We could use this command to get the version specified in go.mod, then download the corresponding release from GitHub:
go list -f '{{ .Version }}' -m github.com/influxdata/influx-cli/v2


add like operator

#164

@jessenr could you add an integration test for it in PostgresModuleTest?

yeah sure


usb-c: Remove duplicate typedefs

See https://docs.zephyrproject.org/latest/connectivity/usb_c/ucds.html#c.policy_cb_get_src_rp_t
Not sure why this wasn't caught by CI / Doxygen, although I just had Doxygen 1.9.4 give me a warning about it on a fresh Ubuntu 23.04 install?

I'm thinking this is a regression of Doxygen v1.9.6 (introduced by the fix for https://github.com/doxygen/doxygen/issues/9552?) as I could also reproduce the warnings in Doxygen 1.9.5.
cc @gmarull


For Ocean_acid project

2 issues:
for the PH datasets, code to remove the attrbs is needed on colab only:
ds_pH = xr.open_dataset(pooch_load(url_SurfacepH,filename_SurfacepH))
ds_pH.attrs=''
ds_pH
updating the new (old) dataset. the FTP provided is temporary :
filename_CO2_CAMS= 'SSTyCO2_CAMS_Copernicus_data.nc'
url_CO2_CAMS='ftp://livftp.noc.ac.uk/noc/gmayapub/data/CMA/SSTyCO2_CAMS_Copernicus_data.nc'
ds_CO2_CAMS =xr.open_dataset(pooch_load(url_CO2_CAMS,filename_CO2_CAMS))
ds_CO2_CAMS

for the pH datasets, the lot and lan index are wrong. need to use the variables stored inside the dataset:
"Hmm..seems like the actual coordinates were stored in the longitude and latitude variables and the coordinates were just dummy values going from 0 to number of coordinates.
"
discord link (has an image, check the code in it):
https://discord.com/channels/1125460174195404870/1131254228967501886/1131497387244929054


add loading indicator when move to another page

Please add a loading indicator/shimmer when moving to another page. I think it's weird if we try to go to another page and the page is left blank like this :

Feature added.


oneof validation doesn't work for nested structs

Package version eg. v9, v10:
v9
Issue, Question or Enhancement:
oneof validation doesn't work for nested struct
Code sample, to showcase or reproduce:
type AbstractKey struct {
	KeyType             string `json:"keyType" form:"keyType" query:"keyType" validate:"oneof=ECDSASecp256k1VerificationKey Ed25519VerificationKey RSAVerificationKey"`
}

type ManagementKey struct {
	AbstractKey
	Priority int `json:"priority" form:"priority" query:"priority"`
}

// in this case validator returns valid, but should be invalid
key := &ManagementKey{}
key.KeyType = "randomString"
validate := validator.New()
err := validate.Struct(key)

// in this case validator returns invalid as expected
key := &AbstractKey{}
key.KeyType = "randomString"
validate := validator.New()
err := validate.Struct(key)

Yes indeed, it doesn't work. Is there any workaround for this please ? @deankarn


+settings for UI, less IO r/w, rm timeout from eval

Still has comm issues sometimes. I haven't looked into it too much yet since it still mostly works. Last I saw it was getting stuck between comms.js and puck.js

I have lots of new improvements made, just doing some final touches on them. I also removed the timeout for Puck.eval but now I need to handle switching to defaults differently. I think besides the communications issues I might have everything full functional (latest bug was leading zero state not reading correctly because of some state save logic that wasn't being set).
I don't see how to convert this pull request to a draft.

Top right under 'Reviewers'? At least i see it - I'll do it for you now anyway

Just committed the final touches. Seems nice and stable and fast. Let me know what you think. Thanks!

Don't have time to look into it too deeply, but it looks good - merging now!


Build and Deploy Your SAP Fiori App to SAP Cloud Platform

Tutorial URL: https://developers.sap.com/tutorials/appstudio-fioriapps-mta-build-deploy.html
Please specify the step you are referring to
In Step 2.2, we have to login with command "cf login" to cf.
In step 2.3 you wrote:
"When prompted, provide your credentials, select the API endpoint, organization, and space for your project."
I don't know which credentials I have to use here or where I can find them.
Could you please help?
Thanks!

Hi @tobiassschlosser ,
The credentials you use to access the SAP Cloud Platform account.
Regards - Raz

Thank you for your feedback. The issue seems to be resolved, so I am closing the issue.
If you still have questions, feel free to reopen the issue.


Add gateway monitor metrics and event

[ ] Make sure you have followed Kube-OVN Code Style.

What type of this PR
Examples of user facing changes:

Features

add gateway packet count metrics
add event for subnet gatewayType or gatewayNode change
add event for subnet gateway ping failed

Bug fixes
Docs
Tests

Which issue(s) this PR fixes:
Fixes #2233

The subnet metrics relies on these rules, we should consider a batch way to update these rules.


Not applied predExp when it comes to ((bin1=a1 or bin2=a2) and (bin2=b1 or bin2=b2))

Maybe I really missed and confused with number passed to and predExp
This is quite project specific, I'll try later with a simple project to show that this doesn't work.
Building predExp on two different bins
with multiple values on each bin
((bin1=a1 or bin2=a2) and (bin2=b1 or bin2=b2))
and 'and' condition between this two predExp
((bin1=a1 or bin2=a2) and (bin2=b1 or bin2=b2))
This don't do expected server side filtering.
But when I try with filter per seldom bin(any one) it works as expected.
It looks that it just don't apply this on server side. So I have to do internal silly local side filtering of data.
Thanks.

This seems more about predExp in java client other than this helper lib. With some quick mockup tests with predExp, the above query generated some reasonable results for me.
If you could provide some sample code snippets, I can take a look for further investigation.
Thanks


Weathermap doesn't load

https://github.com/user-attachments/assets/c0e34200-ed8b-403b-8e33-423bd67f8732
Device info:
iOS 18.1
iPhone 14 Pro

@jgonzales20 found the issue, i am fixing it!

There was some map fixes resolved now in both the latest (build 144) Google Play Store and the Github versions, not sure if the iOS version is up on their appstore latest iOS should be 2.9.1 Build 123 if you want to test/test this weekend or Monday @jgonzales20.


Update README.md

Hello,
I improved the spelling within the README to improve project documentation clarity.
Great project! Keep up the good work!
Peace and Love.

coverage: 93.866%. remained the same when pulling 26c4da44a1e33bc51186de03cca63dc6b6dc4cbd on helloausrine:main into 7d69dbe61396858ee7a5240199bec01dba848a1b on medplum:main.

Thank you @helloausrine !


Feature request: Improve error handling

Make acme_tiny produce error messages suitable for end user consumption. Any error message goes to stderr must be suitable for end user. For debugging and logging purposes, add the option '--log filename' to output verbose messages to 'filename'.
Any error messages returned by Let's Encrypt should be safe to be presented directly to end users. Others should generate generic error messages.

pull requests welcome, but I want to keep the script under 200 lines of read-able code.


Bug: Some acceptance tests Teardown without Open

Bug description
The acceptance tests:

TestDestination_Configure_RequiredParams
TestDestination_Configure_Success
TestSource_Configure_RequiredParams
TestSource_Configure_Success

Are calling the Teardown method without previously calling the Open method. I am assuming that the documentation is correct.
Teardown will crash because the resources that should be initialized by Open are not in place (nil).
Steps to reproduce

In this branch: https://github.com/conduitio-labs/conduit-connector-nats-jetstream/tree/fix/destination-reconnection
Remove the skip tests from acceptance_test.go
See the acceptance tests panicking because nil pointers (because Open wasn't called)

Version
v0.6.0

These tests work as intended, Teardown can in fact be called without previously calling Open. When the connector is first created, Conduit calls Configure and then Teardown to signal that the connector process will stop running. The purpose is to give the connector a chance to validate the configuration, since we don't want to create a connector with an invalid configuration. We call Teardown at the end, as we want to guarantee that it is always the last method that's called before the connector stops, regardless of what was called before, in case there is any cleanup that needs to happen.

So, that means that I need to use Configure to initialize the resources?

So, that means that I need to use Configure to initialize the resources?

Not really, all that means is that Teardown is hardwired to be the last method called before the connector stops running. You can see it as a signal that the process will stop and it's the last chance to do anything. What happens before that call depends on the context:

If the connector is being created, Conduit will call Configure to validate the configuration and then Teardown.
If the connector is being started as part of a starting pipeline, Conduit will call all methods provided by the connector as explained in the docs. That's the "happy path". If any of those functions return an error, the lifecycle can be ended early (i.e. if Configure returns an error, it goes straight to Teardown).
If the connector is being deleted, Conduit will call LifecycleOnDeleted to give the connector a chance to delete any long-living resources (e.g. replication slot) and then Teardown.