NetBeans compile on save does not work with 1.18.0

Using Maven 3.5.0 on NetBeans 8.1, or Maven 3.3.9 on NetBeans 9.0 RC1, the result is the same.

Create a file and compile it
Observe it being in the class directory
Edit the file
File is no longer in the class directory

Issue does not occur on 1.16.18 or older versions, but does on all newer versions.

So 4ac941acca350b889ae6ccd6f455c992a5e49dce / #1555 is actually the commit that introduced this bug according to git bisect

Reverting that commit fixes compile on save, but re introduces red warnings all over the IDE itself.
Will investigate further.

Only NetBeans 8.1 is fixed by reverting that commit, not NetBeans 9 RC1
Guessing compile on save uses a different file manager that is not accounted for

Completely disabling the file manager hooking produces results which work everywhere, with the side effect that the dummy force new round file is left behind and the SneakyThrows ASM processing doesn't work - neither of which are issues to me. Unfortunately I can only figure out a way to disable this via a system property and not via the Lombok config which requires an AST to already be created.
My second attempt was to rewrite the FileManager hooking to be much more minimal, and I did this quite simply in a way that worked on Java 8 and Java 10 without any special code for either, unfortunately it did not work on Netbeans compile on save, and I'm not particularly sure how to debug that.

I think the only reason that commit appears to work is because lombok doesn't even run at all when thats reverted....
I give up for now, but advice on a way to implement the option in my first paragraph would be good.

Same as #1577
I've submitted a pull request (#1775 ) just before discovering your comments :)

I had the same problems (SpringBoot Devtools not working due to missing class files) and it now works again with NetBeans 9.0 and lombok-edge. Time for a 1.18.3 release before Lombok gets a reputation of mysteriously breaking things?

I guess the bug could be closed?!

Thats ok for me


Remove pip flag when video finishes

This fixes the following bug:

Open a video on Youtube
Click on this chrome extension to put the video in the picture-in-picture mode
Wait until the video finishes and the picture-in-picture window automatically closes
Click on another video (or alternatively - wait until the next video starts playing because of Youtube's autoplay, or navigate "back" to another video)
Click on this chrome extension
Expected:
Video is in picture-in-picture mode
Actual:
Nothing happens, a console error is logged

This happens because the <video> element is not replaced when you navigate to another Youtube video, simlpy the src is changed, but the node is the same. In this case the <video> element still has the __pip__="true" attribute which is incorrectly reporting that the video is in picture-in-picture mode when it isn't.

Were you able to receive the leavepictureinpicture event?
Last time I tried, it wasn't fired for me in the extension. Let me know if https://github.com/beaufortfrancois/picture-in-picture-chrome-extension/commit/55f9218e86b01db897416799fda4a936ccc37fc8 works for you

@beaufortfrancois yes, I'm using Chrome Canary version 69. The leavepictureinpicture event is firing correctly every time.
The commit 55f9218 does not work.

It works indeed. Thank you for filing this issue @uudens!
I've updated the extension code at https://github.com/beaufortfrancois/picture-in-picture-chrome-extension/commit/412e9c2f8bbd24fd31870beeb47b242a475b3194
Does that look good to you?

@beaufortfrancois I didn't file an "issue", I created a "pull request" with the fix already here :)
Your commit also works, thanks!

Argh. Sorry, I was in the train with limited access.


Work on content of main page

Add About Me Section that's expanded on what I had in the previous portfolio


How to reuse code in pkl

like I have a yml:
targetA :
    dependencies:
        - A
        - B
targetB:
    dependencies:
        - A
        - B

I only want to write the dependencies part once and use it in different target. How can I do it in pkl?

local deps {
  "A"
  "B"
}
targetA {
  dependencies = deps
}
targetB {
  dependencies = deps
}

PS: I'd recommend to post questions under "Discussions".

@translatenix
Thank you very much.
I will pay attention next time🙇


License disappears after bundling with npm/webpack

I use gl-matrix with some own stuff I have written but the license comment disappears after bundling with webpack.
This can be quickly and easily solved by tagging the license with @license in
src/gl-matrix/common.js  src/gl-matrix/mat2.js  src/gl-matrix/mat4.js   src/gl-matrix/quat.js  src/gl-matrix/vec3.js
src/gl-matrix/mat2d.js   src/gl-matrix/mat3.js  src/gl-matrix/quat2.js  src/gl-matrix/vec2.js  src/gl-matrix/vec4.js
E.g.
/**
 * @license
 * Copyright (c) 2015, Brandon Jones, Colin MacKenzie IV.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE. */

Hi @marcelteun I've made some updates here https://github.com/toji/gl-matrix/pull/268
regarding license and version number.
Hopefully @stefnotch can have a look at them, but my idea is to remove this from the source files and tests and only append it to the dist folder which is where we really need it.

you can probably close this one @stefnotch ? :)

Hmm, I still don't see a license in my bundle. Since I am quite new to javascript, I might me misunderstanding something. Perhaps you usually have the license somewhere else, than in the code? Though the license explicitly states that it shouldn't be removed. And it is a bit strange to add it in the bundle.js, but I do think that it is what the lincense states that should be done.
What I am doing is this: http://tunnissen.eu/polyh/ClaxBako.html which uses http://tunnissen.eu/polyh/js/show-off-bundle.js
You can see that I added the license there, but I must admit, it looks a bit strange.

Hey @marcelteun everyone is new to javascript at some point or another. So don't worry.
Judging by the show-off-bundle.js file you shared it seems like you're requiring/importing several files or libraries yourself and let webpack compile them for you. If you scroll down you can see that the license regarding gl-matrix is being added (if you search that file for Copyright (c) 2015, Brandon Jones, Colin MacKenzie IV. you'll find it).
So the license is there but you're webpack is organising your dependencies in "order" so they are available in the file before any of your code calls it. This is normal.
If you have a look at the dist/ folder you can see that the gl-matrix license is generated. it can also be reviewed here.
Lastly based on the copyright version I can see on your show-off-bundle.js you are using an older version of gl-matrix. you might want to update to the latest 2.6.1
I hope it helps :)

Hmm, I still don't see a license in my bundle. Since I am quite new to javascript, I might me misunderstanding something. Perhaps you usually have the license somewhere else, than in the code? Though the license explicitly states that it shouldn't be removed. And it is a bit strange to add it in the bundle.js, but I do think that it is what the lincense states that should be done.
What I am doing is this: http://tunnissen.eu/polyh/ClaxBako.html which uses http://tunnissen.eu/polyh/js/show-off-bundle.js
You can see that I added the license there, but I must admit, it looks a bit strange.

I know the license is there in the example I gave you. It is there, because I tagged the license comment with @license. But with the new version, "gl-matrix": "^2.6.1", there is no license in the bundle, check here for the result: http://tunnissen.eu/tmp/show-off-bundle.js
My source code is can be found here: https://github.com/marcelteun/show-off
(in case you want to check yourself)
If it is right that the license should end up in my show-off-bundle.js, then tagging it with @license worked before (I noticed that lodash did that, when I added that to check whether it was generated for that module)
Best regards
Marcelteun

Hey @marcelteun well, the licence is on the source code of gl-matrix, so there's no problem there :)
What's happening is you're using webpack on your project and that picks up all the dependencies you're importing and compiles them together in one file, and because you're using mode: production on your webpack.config.js, the exported file uses a bunch of production defaults, which include code minification, removal of dead code, removal of comments etc. You can read more about it here and here.
So just to clarify, this is an issue with your project and NOT gl-matrix itself. (you're choosing to strip all comments by using mode: production. Now, if you change to mode: development you can see that all the comments and license are kept.
What I suspect you want is to minify and compile the javascript but add a license to the source code yourself on the beginning of the file. This needs to be manually defined by you and you can do it in two ways
a) You tell Uglify to keep the comments and the license will appear somewhere in the middle of the source code
b) or you can add a specific license to your project with the webpack.BannerPlugin.
You can have a look how we did it in this project here
I hope it helps your project.
@stefnotch I think you can close this issue

Thanks for taking the time to explain this to a newbie! I will have a look at the links you provided. You say there are two ways: a) and b). There is a 3rd way: you using the @license tag, but then it wont be removed. This is not my problem, though, as long as I choose either a) or b).
By not adding the @license tag you provide the means that a derivative of the software is stripped from the license. I don't know whether that has legal implications for you. Perhaps not, I really don't know.
Once again. Thank you for helping me further! I don't have an issue anymore.
/MT

Glad to hear that your issue got fixed!
@andrevenancio Thanks a lot for addressing this issue so well!
Maybe we should add a @license tag, just to prevent those issues in the future?

Hi again,
I tried your suggestions, but they didn't work.
a) I switched mode to development. The license did not appear
b1) I installed Uglify and I used:
const UglifyJsPlugin = require('uglifyjs-webpack-plugin');
..
plugins: [new UglifyJsPlugin({extractComments: true})],
the license did not appear in the separate file show-off-bundle.js.LICENSE
My own license did, thogh. And I think that is because my license is tagged
with @license.
b2) Keeping development mode I used
plugins: [new UglifyJsPlugin({extractComments: /Copyright/})],
instead. The license didn't appear.
b3)
I used
plugins: [new UglifyJsPlugin({extractComments: /Brandon/})],
show-off-bundle.js.LICENSE
It seems that the license is stripped already before Uglify is run. I tried
adding the @license tag to
node_modules/gl-matrix/dist/gl-matrix-min.js
node_modules/gl-matrix/dist/gl-matrix.js
node_modules/gl-matrix/utils/webpack.config.js
But that didn't help.
I even tried to use
plugins: [new UglifyJsPlugin({extractComments: true})],
in
node_modules/gl-matrix/utils/webpack.config.min.js
still no license
Unfortunately the situation is worse than it was before.. I will have to
add the license myself manually now.
/Marcel
On Sat, May 19, 2018 at 9:30 AM, stefnotch<EMAIL_ADDRESS>wrote:

Glad to hear that your issue got fixed!
@andrevenancio https://github.com/andrevenancio Thanks a lot for
addressing this issue so well!
Maybe we should add a @license https://github.com/license tag, just to
prevent those issues in the future?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/toji/gl-matrix/issues/304#issuecomment-390386372, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAZAnA3n73LYHx6gEALQX-6Mo7Ttv0Vzks5tz8oCgaJpZM4T8rTj
.

Hey @stepnotch let me know what’s the expected behaviour and i’m happy to make the changes. Personally when I compile the code for production I like to strip all the comments myself. But I’m not aware of legal infringement that can cause with the MIT license.
We can include a link to the licence like three.js does on their minified version, or we can put the full licence. Happy to do either. For me this is a minor issue.
You want to have a @license / @preserve tag on the files in the /dist folder and on the src/gl-matrix.js ?

@andrevenancio I think a link like three.js does would be good enough. We probably should put it in all files. A @license tag would be nice.
(I feel sorry for the person with the name @license. xD)

Ok, so this are my findings:
When you're using webpack and you're requiring or importing the gl-matrix library, you're importing the file src/gl-matrix.js as entry point. At the moment our gl-matrix doesnt include the license there. We can add it there. However, after adding it there in my local version, that comments still gets stripped out regardless if it has @license or @preserve. so its useless to have the license or just a text like three.js has pointing to the license, because webpack in its default settings will exclude the comments
The problem you're experiencing @marcelteun has to do with webpack and uglify on your own project.
If you have a look at this and this.
You'll see that there's a way of preserving the all comments.
I've been playing around with it and it seems like their documentation isn't very clear on how to extract only the license comment.
So far I've only successfully managed to grab all comments (which include the comments used to generate the docs) into a separate file.
I'm going to leave this for now as this is not an issue with gl-matrix, but with webpack itself and you can submit a bug on their repo.
If by any chance I find a solution for this, I'll come and update this issue.

Ok, after investigating this there's no way of resolving it as it is.
If you want to keep the license from the original modules you imported you need to allow "comments" in your uglify version..
Until UglifyJS updates their code, uglify-webpack-plugin and webpack will carry out this error.
More info in the links I've provided about and here
Wont fix @stefnotch @marcelteun

Yeah we can, but also we can potentially insert the @license in our license so when this issue its fixed, we dont need to do anything, and also add the licence to src/gl-matrix.js so it gets picked by webpack/rollup configurations.
I'll submit a PR tonight so we have that covered, then when they fix this license issue we are already covered :)

Thanks for your help, @andrevenancio!


Feature: Improve the UI of the High Score section of the Quiz and fix its responsiveness.

Feature description
The UI of the high score page of the quiz is not looking good as well as it is not responsive.

👀 Have you spent some time to check if this issue has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

@SauravMukherjee44 can you please assign me this issue. I would like to work on it.

Go for it!


Access the imputed dataset

Hi,
Thanks for this great work! I successfully ran the GRUD on samples (using run.ipynb).
I am interested in the imputation output (X' with the imputed values), but it seems that only the prediction performance results are stored. Could you please guide me on how to access the imputed dataset?
Thanks!

Hi moyomer,
As mentioned in the paper, The proposed method is not explicitly designed for filling in the missing values in the data, and can not be directly used in unsupervised settings without prediction labels. (https://www.nature.com/articles/s41598-018-24271-9#Sec22)
In the case that the imputed data is indeed needed, you may modify the model and take the output from the last GRU-D layer as the imputation. https://github.com/PeterChe1990/GRU-D/blob/9e1274a1ad67135137f53159eafc92c7278a931a/models.py#L56 (i.e., before the first MLP layer)


Would like to help for Burmese/Myanmar language training?

Hello,
I would like to help. I've already cloned all repository. How do I start?

What issue is there with Burmese/Myanmar language?

We have 2 types of unicode font. Non standard unicode font and standard unicode font. When I check langdata files for Burmese, most words are incorrect. I guess you have generated mixed contents with non standard unicode contents and standard unicode contents. When I try to scan an image with Burmese character written in Padauk fonts, output contents are not readable.
I would like to know method you've used to generate Burmese training files. Where did you get original data? I can check if it is standard unicode contents or not.

I think the real issue is not only about using standard or non-standard Unicode, but also the wrong method of extracting data from the source. I mean the source data need to be segmented correctly to get a correct single word.
Myanmar language users do not much care about adding a 'space' character between words; this results in false perception of two or more words as a single word, when you assume all characters between 2 'space' characters as a word. I found most word lists here ,especially bi-grams holds too long Myanmar phrases. That makes the wordlists unusable and the results of its appliction is totally unpredictable
So I think you need to extract data from a source using dictionary-lookup approach. Of course, you need to build your own wordlist manually or use those made by others.
Also Myanmar language is a syllable-based language; that is one or more Myanmar letters combine to form a syllable and one or more syllables join to form a word. So it is advisable to detect syllables so that you can gain much performance improvement in dictionary-looking up.

@herzcthu @minthanthtoo
Please add some good sources of standard unicode fonts and sample texts and word frequency lists to https://github.com/tesseract-ocr/langdata/issues/46

https://my.wikipedia.org/
All contents on wikipedia are in standard unicode font.

@zdenop Issue is with training data itself.  The person who prepared the data, does not know the Myanmar language.  Majority of the training data has misspellings and mixed with hacked version of Myanmar Unicode as said by @herzcthu .  You can imagine rice and spaghetti mixed in a bowl. Also, it is not segmented properly as @minthanthtoo pointed out.  Any suggestions to on how to prepare training data?

Please see Ray's comment at
https://github.com/tesseract-ocr/tesseract/issues/654#issuecomment-274574951
about how the training data is being built for the 4.0 LSTM training. I don't think they are using the training_text file in langdata.

Thanks @Shreeshrii   ./tesstrain.sh would automatically create .tff/box pairs from langdata directory for 4.0 LSTM training?

Yes. Tesstrain.sh creates tiff box pairs that can be used for LSTM training. Please see wiki pages regarding details. You need large amount of training data for good training. See Ray's comments about LSTM training process.

https://github.com/tesseract-ocr/tesseract/issues/654

will add the code to the github repo in due course, so experts/native speakers can offer suggestions/fixes to make them better. Myanmar in particular needs improvement, as the www data is littered with dotted circles, and the unicode book does not adequately describe the syntax for a well-formed grapheme in Myanmar (or any other language for that matter).

https://github.com/kanyawtech/myanmar-karen-word-lists/blob/master/burmese-word-list.txt?raw=true
Is this a good wordlist in correct unicode for mynamar?

These are the most common words in Myanmar, but it is not a complete list.  The definition of a word itself is tricky in Myanmar language because there are many ways syllables can be combined to form a word.  I am not so sure how Tesseract training works, but it may be better to train on the syllables instead of a word(cluster of syllables). Which is also to say that each syllable must be first detected and then do the classification.  Classifying entire word may be too difficult, unless I am not fully aware of Tesseract  capabilities.

Manually Constructed Context-Free Grammar For Myanmar Syllable Structure
http://www.aclweb.org/anthology/E12-3004

Representing Myanmar in Unicode
Details and Examples
http://unicode.org/notes/tn11/myanmar_uni-v2.pdf
http://www.tuninst.net/LINGUISTICS/myanmar-unicode/myanmar-unicode.htm
Creating and Supporting OpenType Fonts for Myanmar Script
https://www.microsoft.com/typography/OpenTypeDev/myanmar/intro.htm
Myanmar script notes
http://rishida.net/scripts/myanmar/#shaping
https://www.researchgate.net/publication/253745697_A_Rule-based_Syllable_Segmentation_of_Myanmar_Text

@theraysmith
I used a few words from the burmese wordlist and the landing page of wikipedia as a small training sample to test mynamar. Both of these are supposed to be in standard unicode for mynamar.
training text and generated unicharset are attached. I got a number of errors while building unicharset. Maybe the mynamar.unicharset in langdata needs to be updated???

=== Phase UP: Generating unicharset and unichar properties files ===
[Fri Mar 31 16:07:02 DST 2017] /usr/local/bin/unicharset_extractor -D /tmp/tmp.OzCvDLSWBp/mya/ /tmp/tmp.OzCvDLSWBp/mya/mya.Myanmar_Text_Bold.exp0.box /tmp/tmp.OzCvDLSW
Bp/mya/mya.Myanmar_Text.exp0.box
Extracting unicharset from /tmp/tmp.OzCvDLSWBp/mya/mya.Myanmar_Text_Bold.exp0.box
Extracting unicharset from /tmp/tmp.OzCvDLSWBp/mya/mya.Myanmar_Text.exp0.box
Wrote unicharset file /tmp/tmp.OzCvDLSWBp/mya//unicharset.
[Fri Mar 31 16:07:05 DST 2017] /usr/local/bin/set_unicharset_properties -U /tmp/tmp.OzCvDLSWBp/mya/mya.unicharset -O /tmp/tmp.OzCvDLSWBp/mya/mya.unicharset -X /tmp/tmp
.OzCvDLSWBp/mya/mya.xheights --script_dir=../langdata
Loaded unicharset of size 217 from file /tmp/tmp.OzCvDLSWBp/mya/mya.unicharset
Setting unichar properties
Other case È of è is not in unicharset
Other case Ë of ë is not in unicharset
Warning: properties incomplete for index 4 = ယ်
Warning: properties incomplete for index 5 = လ်
Warning: properties incomplete for index 8 = မ်
Warning: properties incomplete for index 10 = င်
Warning: properties incomplete for index 16 = မှ
Warning: properties incomplete for index 22 = ရှ
Warning: properties incomplete for index 28 = ဖွဲ့
Warning: properties incomplete for index 30 = ည်
Warning: properties incomplete for index 36 = ပ်
Warning: properties incomplete for index 37 = ဖြ
Warning: properties incomplete for index 38 = င့်
Warning: properties incomplete for index 41 = က်
Warning: properties incomplete for index 42 = နှာ
Warning: properties incomplete for index 43 = ည်း
Warning: properties incomplete for index 44 = တ်
Warning: properties incomplete for index 45 = မှု
Warning: properties incomplete for index 47 = မ်း
Warning: properties incomplete for index 50 = ခြ
Warning: properties incomplete for index 51 = င်း
Warning: properties incomplete for index 52 = ကြော
Warning: properties incomplete for index 53 = နှို
Warning: properties incomplete for index 54 = ချွ
Warning: properties incomplete for index 63 = ပွဲ
Warning: properties incomplete for index 64 = တွေ
Warning: properties incomplete for index 65 = မှာ
Warning: properties incomplete for index 66 = ဆွေး
Warning: properties incomplete for index 67 = နွေး
Warning: properties incomplete for index 73 = ထွေ
Warning: properties incomplete for index 78 = မြ
Warning: properties incomplete for index 79 = စ်
Warning: properties incomplete for index 80 = မြို့
Warning: properties incomplete for index 83 = န်
Warning: properties incomplete for index 86 = ကွ
Warning: properties incomplete for index 89 = သွ
Warning: properties incomplete for index 92 = ဖ်
Warning: properties incomplete for index 96 = ခြေ
Warning: properties incomplete for index 100 = မျှ
Warning: properties incomplete for index 101 = ဂြို
Warning: properties incomplete for index 102 = ဟ်
Warning: properties incomplete for index 103 = တွ
Warning: properties incomplete for index 110 = ရှု
Warning: properties incomplete for index 119 = ညွှ
Warning: properties incomplete for index 120 = န်း
Warning: properties incomplete for index 123 = ကြ
Warning: properties incomplete for index 124 = ည့်
Warning: properties incomplete for index 125 = နှ
Warning: properties incomplete for index 126 = ထွ
Warning: properties incomplete for index 130 = ရှိ
Warning: properties incomplete for index 132 = ကြို
Warning: properties incomplete for index 140 = ဉ်
Warning: properties incomplete for index 150 = လှ
Warning: properties incomplete for index 151 = သွား
Warning: properties incomplete for index 153 = ထွာ
Warning: properties incomplete for index 154 = ထွား
Warning: properties incomplete for index 157 = ဖွံ့
Warning: properties incomplete for index 158 = မွ
Warning: properties incomplete for index 159 = လျော်
Warning: properties incomplete for index 162 = ပြော
Warning: properties incomplete for index 163 = ထွေး
Warning: properties incomplete for index 164 = ယှ
Warning: properties incomplete for index 168 = ဘွား
Warning: properties incomplete for index 179 = လွ
Warning: properties incomplete for index 182 = န့်
Warning: properties incomplete for index 189 = စွဲ
Warning: properties incomplete for index 192 = ပြီး
Warning: properties incomplete for index 197 = မြေ
Warning: properties incomplete for index 202 = ကွာ
Warning: properties incomplete for index 210 = ရှာ
Warning: properties incomplete for index 211 = ဖွေ
Warning: properties incomplete for index 212 = တွေ့
Warning: properties incomplete for index 214 = ပြ
Warning: properties incomplete for index 215 = ကြာ
Writing unicharset to file /tmp/tmp.OzCvDLSWBp/mya/mya.unicharset

mya.Myanmar_Text.exp0.txt
mya.Myanmar_Text_Bold.exp0.txt
mya.unicharset.txt

@herzcthu @nengine @minthanthtoo
Please take a look at https://github.com/tesseract-ocr/langdata/blob/master/Myanmar.unicharset
in light of the above warning messages. Do you notice any pattern for the errors?
Tesseract does train on syllables (for Indic languages) AFAIK. Please see https://github.com/tesseract-ocr/langdata/files/885327/mya.unicharset.txt generated from the two training files - all listed in the message above.

@theraysmith do zwj and zwnj also have to be part of unicharset?
also see http://archive.mmgeeks.com/index.php?p=/discussion/379/zwnj-and-zwj

https://github.com/khzaw/awesome-myanmar-unicode

Syllabification, Normalization and Lexicographic Ordering
of Myanmar Texts using Formal Approaches
http://ir.nagaokaut.ac.jp/dspace/bitstream/10649/729/1/k709.pdf

I do not see consistent pattern.

Warning: properties incomplete for index 4 = ယ် .  ယ် by itself does not have any meaning, but when it is combined with ဘ which becomes ဘယ် it makes sense.

Warning: properties incomplete for index 16 = မှ .  မှ by itself does make sense and has a meaning, but not so sure why it is giving a warning.

[Myanmar.unicharset] (https://github.com/tesseract-ocr/langdata/blob/master/Myanmar.unicharset)  clearly does not include these syllables shown in the warnings, but just consonants, vowels, etc.
It is suppose to include all syllable combinations in Myanmar.unicharset ?  How does it work for Telugu for example?

I don't think it is supposed to include all syllable combinations in Myanmar.unicharset but it should have all vowels, consonants, vowel signs.
I see three ranges for mynamar, first seems to be there in the unicharset, part of second and none of third.
Can you please check whether all of these are required?
http://www.alanwood.net/unicode/myanmar.html
http://www.alanwood.net/unicode/myanmar-extended-a.html
http://www.alanwood.net/unicode/myanmar-extended-b.html

There are 8 major ethnic groups in Myanmar, so I believe extended A and B are added for that reason.  So, for completeness I think it should be added, but you would rarely see them on the web.  Unicode range 1000 - 104F is already good.

you would rarely see them on the web.

What about in books / documents that need to be OCRed?

excuse the brevity, sent from mobile

On 31-Mar-2017 10:35 PM, "nengine"<EMAIL_ADDRESS>wrote:
There are 8 major ethnic groups in Myanmar, so I believe extended A and B
are added for that reason. So, for completeness I think it should be added,
but you would rarely see them on the web. Unicode range 1000 - 104F is
already good.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-290770704,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o5Kd-mzKd7Mg_tmQQirf-TZ1frzWks5rrTJYgaJpZM4FRqc3
.

Yes, extended A and B should also be added for completeness as I said, but as far as for training samples, it is almost non existence on the web.

From: Shreeshrii<EMAIL_ADDRESS>Sent: Friday, March 31, 2017 1:22 PM
To: tesseract-ocr/langdata
Cc: nengine; Mention
Subject: Re: [tesseract-ocr/langdata] Would like to help for Burmese/Myanmar language training? (#13)

you would rarely see them on the web.

What about in books / documents that need to be OCRed?

excuse the brevity, sent from mobile

On 31-Mar-2017 10:35 PM, "nengine"<EMAIL_ADDRESS>wrote:
There are 8 major ethnic groups in Myanmar, so I believe extended A and B
are added for that reason. So, for completeness I think it should be added,
but you would rarely see them on the web. Unicode range 1000 - 104F is
already good.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-290770704,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o5Kd-mzKd7Mg_tmQQirf-TZ1frzWks5rrTJYgaJpZM4FRqc3
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/tesseract-ocr/langdata/issues/13#issuecomment-290774836, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAFECn5K6buH3pJrMpulDKKaWQYZNToKks5rrTZcgaJpZM4FRqc3.

Please take a look at this reference:
http://www.unicode.org/versions/Unicode9.0.0/ch16.pdf
Table 16-3.
The text says "Characters occur in the relative order shown in Table 16-3"
which I do not believe to be completely correct.
Part of the problem is that a lot of the characters are not even in this
table!
Although it is possible to guess which group the extensions belong to, I'm
not convinced I have it correct.
I have some code that implements this table plus my guesses to add the
extensions, but it isn't ready for committing to github just yet.
The problem is that I need to exclude the incorrectly formatted text (that
uses the non-standard fonts), but be sure that no correctly formatted text
is dropped.
On Fri, Mar 31, 2017 at 10:49 AM, nengine<EMAIL_ADDRESS>wrote:

Yes, extended A and B should also be added for completeness as I said, but
as far as for training samples, it is almost non existence on the web.

From: Shreeshrii<EMAIL_ADDRESS>Sent: Friday, March 31, 2017 1:22 PM
To: tesseract-ocr/langdata
Cc: nengine; Mention
Subject: Re: [tesseract-ocr/langdata] Would like to help for
Burmese/Myanmar language training? (#13)

you would rarely see them on the web.

What about in books / documents that need to be OCRed?

excuse the brevity, sent from mobile

On 31-Mar-2017 10:35 PM, "nengine"<EMAIL_ADDRESS>wrote:
There are 8 major ethnic groups in Myanmar, so I believe extended A and B
are added for that reason. So, for completeness I think it should be added,
but you would rarely see them on the web. Unicode range 1000 - 104F is
already good.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/tesseract-ocr/langdata/issues/13#
issuecomment-290770704>,
or mute the thread
<https://github.com/notifications/unsubscribe-
auth/AE2_o5Kd-mzKd7Mg_tmQQirf-TZ1frzWks5rrTJYgaJpZM4FRqc3>
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://github.com/
tesseract-ocr/langdata/issues/13#issuecomment-290774836>, or mute the
thread<https://github.com/notifications/unsubscribe-auth/
AAFECn5K6buH3pJrMpulDKKaWQYZNToKks5rrTZcgaJpZM4FRqc3>.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-290781430,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AL056cZdXfwKdFL1EpH01k8FRXDHh6NTks5rrTyNgaJpZM4FRqc3
.

--
Ray.

@theraysmith please add support for armenian
https://github.com/tesseract-ocr/langdata/issues/67#issuecomment-294216290

excuse the brevity, sent from mobile

On 15-Apr-2017 5:21 AM, "theraysmith"<EMAIL_ADDRESS>wrote:

Please take a look at this reference:
http://www.unicode.org/versions/Unicode9.0.0/ch16.pdf
Table 16-3.
The text says "Characters occur in the relative order shown in Table 16-3"
which I do not believe to be completely correct.
Part of the problem is that a lot of the characters are not even in this
table!
Although it is possible to guess which group the extensions belong to, I'm
not convinced I have it correct.
I have some code that implements this table plus my guesses to add the
extensions, but it isn't ready for committing to github just yet.
The problem is that I need to exclude the incorrectly formatted text (that
uses the non-standard fonts), but be sure that no correctly formatted text
is dropped.
On Fri, Mar 31, 2017 at 10:49 AM, nengine<EMAIL_ADDRESS>wrote:

Yes, extended A and B should also be added for completeness as I said,
but
as far as for training samples, it is almost non existence on the web.

From: Shreeshrii<EMAIL_ADDRESS>Sent: Friday, March 31, 2017 1:22 PM
To: tesseract-ocr/langdata
Cc: nengine; Mention
Subject: Re: [tesseract-ocr/langdata] Would like to help for
Burmese/Myanmar language training? (#13)

you would rarely see them on the web.

What about in books / documents that need to be OCRed?

excuse the brevity, sent from mobile

On 31-Mar-2017 10:35 PM, "nengine"<EMAIL_ADDRESS>wrote:
There are 8 major ethnic groups in Myanmar, so I believe extended A and B
are added for that reason. So, for completeness I think it should be
added,
but you would rarely see them on the web. Unicode range 1000 - 104F is
already good.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/tesseract-ocr/langdata/issues/13#
issuecomment-290770704>,
or mute the thread
<https://github.com/notifications/unsubscribe-
auth/AE2_o5Kd-mzKd7Mg_tmQQirf-TZ1frzWks5rrTJYgaJpZM4FRqc3>
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://github.com/
tesseract-ocr/langdata/issues/13#issuecomment-290774836>, or mute the
thread<https://github.com/notifications/unsubscribe-auth/
AAFECn5K6buH3pJrMpulDKKaWQYZNToKks5rrTZcgaJpZM4FRqc3>.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/tesseract-ocr/langdata/issues/13#
issuecomment-290781430>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/
AL056cZdXfwKdFL1EpH01k8FRXDHh6NTks5rrTyNgaJpZM4FRqc3>
.

--
Ray.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-294257441,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o3KU7VdK5SmxttOalVerzvm7mPeAks5rwAaWgaJpZM4FRqc3
.

I've checked characters in Myanmar.unicharset file. All characters seem correct.

Please see https://github.com/tesseract-ocr/tesseract/issues/995#issuecomment-315133403

For instance, there is a big table in the unicode standard for Myanmar, (
http://www.unicode.org/versions/Unicode9.0.0/ch16.pdf) but it doesn't cover
any of the extension Myanmar characters, and isn't explicit about whether
the table represents a specific valid order or not. The existence of a lot
of legacy Myanmar text on the web that is designed for non-compliant fonts
doesn't help make it easier to determine whether the filter is correct.

Please see code at:
https://github.com/tesseract-ocr/tesseract/blob/master/training/validate_myanmar.cpp
On Thu, Jul 13, 2017 at 10:21 PM, Shreeshrii<EMAIL_ADDRESS>wrote:

Please see tesseract-ocr/tesseract#995 (comment)
https://github.com/tesseract-ocr/tesseract/issues/995#issuecomment-315133403
For instance, there is a big table in the unicode standard for Myanmar, (
http://www.unicode.org/versions/Unicode9.0.0/ch16.pdf) but it doesn't
cover
any of the extension Myanmar characters, and isn't explicit about whether
the table represents a specific valid order or not. The existence of a lot
of legacy Myanmar text on the web that is designed for non-compliant fonts
doesn't help make it easier to determine whether the filter is correct.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-315272798,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AL056YM0MGz07l7tSTpJWUPO5bNE1W6rks5sNvrygaJpZM4FRqc3
.

--
Ray.

@herzcthu @nengine @minthanthtoo
Please test with the new traineddata in tessdata/best directory and provide feedback.

I'm testing new traineddata. It has improved a lot. Almost 98% correct. I will test more in detail and will provide feedback in detail later.

I like to test it but not so sure how to do it. I have Windows 10 installed.  Could you please point to the documentation link?

You can use new windows binaries for 4.0 linked  from
https://github.com/UB-Mannheim/tesseract/wiki
ShreeDevi

भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com
On Thu, Aug 10, 2017 at 1:56 AM, nengine<EMAIL_ADDRESS>wrote:

I like to test it but not so sure how to do it. I have Windows 10
installed. Could you please point to the documentation link?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-321371812,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o6HxqCOXKZDOZJnKpjwqm-GSf-qQks5sWhYMgaJpZM4FRqc3
.

I've attached first screenshot I've tested.
Upper part is image I've tested and lower part is OCR converted text.
Words between two adjacent same color points are missing or incorrect.
If you need code point comparison between source image and output text. I can provide later.

It would be helpful if you can point out to any pattern that you notice in
the errors.
I think one that I notice is that words are getting dropped in the OCRed
text (missing).
ShreeDevi

भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com
On Thu, Aug 10, 2017 at 7:13 PM, Sithu Thwin<EMAIL_ADDRESS>wrote:

[image: both]
https://user-images.githubusercontent.com/3231665/29173007-d4bc484e-7e07-11e7-9036-0462da3ac580.png
I've attached first screenshot I've tested.
Upper part is image I've tested and lower part is OCR converted text.
Words between two adjacent same color points are missing or incorrect.
If you need code point comparison between source image and output text. I
can provide later.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-321554871,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_oyNJ4LqZqWLODpuyuvJISRw7EC2qks5sWwkGgaJpZM4FRqc3
.

Hello, I would like to share what I found in Myanmar training data.
I used tesseract version 3.04.
I think It still need to improve a lot. Firstly I would like to tell the my test result. I tried to test with three image with myanmar language: ocr_sample_1.png and ocr_sample_2.png.
test result for ocr_sample_1 image is below. I marked with red point to see different.
image_file

Result

And the second ocr_sample_2 image result is below. It's result is completely worng. It means "how are you" in English.
Image_file

Result

And then I download the myanmar langdata from github.(https://github.com/tesseract-ocr/langdata). I found 7 files. After I check those file, most of the contents are incorrect, misspelling. I would like to show the one or two incorrect data from one of those file named mya.training_text.
For example,

#first arrow head line
It should be "ရုတ်ရုတ်သဲသဲ".
#Second arrow head line
It should be "သစ်တောများကုန်".
#third arrow head line
should be "ပညာရေးစနစ်". so on.
So I would like to contribute to make the correction for these 7 files. And I would like to ask the following questions.
-Exporting mya.traineddata is based on those file?
-How can I know which file is used for what? eg. what is mya.punc file?
-And where did you get those data?
-Is there any format or rule to put data into those files?
Could you please explain me about those files?
I am also willing to improve Myanmar language in OCR.
Thanks you for your contribution.

Please also try tesseract 4.0alpha which might have improved results.
ShreeDevi

भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com
On Thu, Oct 12, 2017 at 8:52 PM, kyawswar<EMAIL_ADDRESS>wrote:

Hello, I would like to share what I found in Myanmar training data.
I used tesseract version 3.04.
I think It still need to improve a lot. Firstly I would like to tell the
my test result. I tried to test with three image with myanmar language:
ocr_sample_1.png and ocr_sample_2.png.
test result for ocr_sample_1 image is below. I marked with red point to
see different.
image_file
[image: ocr_sample_1]
https://user-images.githubusercontent.com/4832700/31503684-54c914c6-af96-11e7-8cb5-ebfbe85dc1c4.jpg
Result
[image: screenshot from ocr_sample_1]
https://user-images.githubusercontent.com/4832700/31503452-b8ce91c2-af95-11e7-96fa-256a19394daf.png
And the second ocr_sample_2 image result is below. It's result is
completely worng. It means "how are you" in English.
Image_file
[image: ocr_sample_2]
https://user-images.githubusercontent.com/4832700/31503742-7748882e-af96-11e7-82e1-d4189ec553d0.png
Result
[image: screenshot from ocr_sample_2]
https://user-images.githubusercontent.com/4832700/31503479-c62a7e26-af95-11e7-93d5-442ddb1fd637.png
And then I download the myanmar langdata from github.(https://github.com/
tesseract-ocr/langdata). I found 7 files. After I check those file, most
of the contents are incorrect, misspelling. I would like to show the one or
two incorrect data from one of those file named mya.training_text.
For example,
[image: screenshot from 2017-10-12 21-01-05]
https://user-images.githubusercontent.com/4832700/31503553-f608e970-af95-11e7-910a-af7ceeb2852d.png
#first arrow head line
It should be "ရုတ်ရုတ်သဲသဲ".
#Second arrow head line
It should be "သစ်တောများကုန်".
#third arrow head line
should be "ပညာရေးစနစ်". so on.
So I would like to contribute to make the correction for these 7 files.
And I would like to ask the following questions.
-Exporting mya.traineddata is based on those file?
-How can I know which file is used for what? eg. what is mya.punc file?
-And where did you get those data?
-Is there any format or rule to put data into those files?
Could you please explain me about those files?
I am also willing to improve Myanmar language in OCR.
Thanks you for your contribution.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-336171733,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o2vO5yL2GI7dhu_pWBoDZhKv9iS5ks5sri7PgaJpZM4FRqc3
.

Yes, I used tesseract 4.0.0-alpha.20170804. I test with the following image files. The following is test result.
ocr_sample_1.png

Result

%%%%%
©05080×05
5082:40:82! 0=2405005$2³050

ocr_sample_2.png

Result

ပဵနႚတ္ဂဵကာဧ်တ္အီးလာသီူး

Thanks.

langdata repo has not been updated for 4.0x.
You can extract the wordlist from the tessdata_best traineddata file.
Use the commands (please lookup the syntax)
combine_tessdata -u ....
dawg2wordlist ...
to see the version of files used for 4.0
You can compare this wordlist to the wordlist in langdata for spelling etc.
ShreeDevi

भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com
On Fri, Oct 13, 2017 at 8:39 AM, kyawswar<EMAIL_ADDRESS>wrote:

Yes, I used tesseract 4.0.0-alpha.20170804. I test with the following
image files. The following is test result.
ocr_sample_1.png
[image: ocr_sample_1]
https://user-images.githubusercontent.com/4832700/31528553-fff13d48-aff9-11e7-9fca-987a0e68c90c.png
Result
%%%%%
©05080×05
5082:40:82! 0=2405005$2³050
ocr_sample_2.png
[image: ocr_sample_2]
https://user-images.githubusercontent.com/4832700/31528592-396454f2-affa-11e7-9139-f6954eba8ef4.png
Result
ပဵနႚတ္ဂဵကာဧ်တ္အီးလာသီူး
Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-336338245,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE2_o2fsRUsCSdN8ou-Yk-WMWrzlwieKks5srtR_gaJpZM4FRqc3
.

Yes, I work perfectly on tessdata_best. But after I checking wordlist, there are many misspelling and incorrect data. I point out the outstanding misspellings. Please see the following attachment.

The most of data from following link is not included in this tessdata_best wordlist.
https://github.com/kanaung/wordlists
I would like to know where did u get that data.
How can I contribute to update those incorrect data?
Thanks.

myanmar traineddata 4.0 does not recognized for the following chars.
၊(u104a)
။(u104b)
၌(u104c)
၍(u104d)
၎(u104e)
၏(u104f)
I unpacked  and checked unicharset.
I not found these char in unicharset.
Also unicharset-extractor  does not produce these chars.

I'm just reading through this thread and have a few pointers - hope this helps!
Also (I'll try to raise it as a separate issue or suggest a fix it if that's possible) the punctuation list currently doesn't include any Burmese punctuation, so while the 4.0 LSTM works very well overall, it fails on every Burmese comma and period.

Unicode range: everything in contemporary use is in the 1000-104F range. The second half of the base range is for special characters needed for writing a number of minority languages but also for Pali words that might be needed by Burmese language users. At some point, as more online data in those languages becomes available on the web, it might make more sense to train specific models for those languages (i.e.  Shan, Mon, Kayah, Western Pwo) that also include the base Myanmar range. For Burmese, a.k.a. Myanmar language, the 1000-104F might be sufficient (and it's hard to find training data beyond that range).

Word lists - the best word list available to researchers AFAIK is a 133k word list maintained by the Myanmar Language Commission (MLC). This is basically just a list of all the words from the large official dictionary of the language. I don't think the word list is publicly available, and it doesn't have any proper nouns, but it would be a good resource if it could be made available for this project. The kanaung word list linked above by @herzcthu is much better than the mya wordlist and I think it is based on an old software release of a "correct spelling" word list and has since incorporated words and place names from other sources like the postal system. I think there are a good deal of non-words and very rare words in that list, though, which maybe limits its usefulness.

Words and spacing - as mentioned above, Burmese / Myanmar doesn't use spaces between every word in its writing system. Spaces are thrown in as needed for typography and usually between multi-word phrases. So the best options for a language model are probably either a character-based language model or a model that incorporates word segmentation (prediction of spaces) into the pipeline. I'm not sure if either of these are possible with Tesseract 4.0. As somebody already mentioned above, the lack of spacing is probably why there are >500,000 words in the mya.wordlist file. As of right now I think the state of the art in Burmese word segmentation (breaking non-spaced continuous text into words) is around 99 percent accuracy -- it's not perfect but it's pretty good.

Syllable segmentation, such as Ye Kyaw Thu's script here https://github.com/ye-kyaw-thu/myPOS/tree/master/corpus-draft-ver-1.0, might also be simpler and more effective than a character-level language model in the absence of word segmentation. A few lines of regex can capture the boundaries between syllables, in which case the language model approach can be similar to what you might use for Chinese (sequences of syllables instead of sequences of words).

Unicode and Zawgyi - the choice between Unicode and Zawgyi is controversial in Myanmar, and Zawgyi is much more popular, with Unicode maybe making some inroads. The problem with Zawgyi, in addition to being non-standard, is that it takes up random spaces in the shared Myanmar unicode range, including spaces for other languages spoken in Myanmar, so it breaks not just Burmese but also the entire extended Myanmar unicode range.  It's also frustrating because it used to be difficult to cleanly convert from Zawgyi to Unicode and back, and it still doesn't work perfectly because Zawgyi hides many typing errors by superimposing the same typed letter without advancing the cursor. For this reason there is a lot of corrupted Burmese language text data on the web that may have been converted to unicode at some point (or could be converted) but it's difficult to catch all of the errors left over from the original typed Zawgyi input. Wikipedia has a good explainer here on the two encoding standards: https://my.wikipedia.org/wiki/Wikipedia:Font#Why_not_Zawgyi? The main thing to watch out for here is not to accidentally feed Zawgyi text into the training data, because with so many overlapping codepoints it could wreck the accuracy of the model.

Thank you for the detailed notes. Please review the source training data in langdata_lstm repo also.

Please test the traineddata at https://github.com/Shreeshrii/tessdata_shreetest/blob/master/mya430000.traineddata
and let me know whether it is an improvement over the existing traineddata files.

Hi Shreeshrii,
I've tested your traineddata. It is a little improved over existing traineddata in tesseract 4.0 beta.
Especially it can detect punctuation and non-burmese characters better.
BTW, I'm trying to train myself, I'm generating lots of box and tif files for only one font. Is that a good idea to have many files for single font? Or should I make it only one box and one tif file.
Currently I have more than 1000 files.
thanks and regards,
Sithu

The amount of training data you need depends on the type of training that you are planning to do. eg. from scratch, replace a layer, plus minus, etc.
I think multiple files for single font may be ok. How are you generating these files?
You should try to keep approximately the same number of lines in each file so that all samples are used in a uniform way for training.

I had used
'Myanmar Khyay'
'Myanmar Sans Pro'
'Myanmar Text'
'Noto Sans Myanmar' \
Which one is a more representative font out of these for training and testing?

I took one paragraph from wikipedia. Make screenshots with all fonts you have mentioned.
Noto Sans Myanmar has best result.
There is new fonts which will be used in officials documents. It is called Pyidaungsu
You can download here https://www.unicode.today/fonts-download/
I'm creating box and tif files using text2image binary from training.
I collected 1 millions unicode text lines from wikipedia and 3 famous news websites. Creating box files from that contents.

@herzcthu Thanks for the info about the new font.
If you do 'replace layer' type of training, you can get by with fewer lines.
Keep posting about your progress with training.

I'm stuck at unicharset extractor.
I get one unicharset file. But when I open that file, I'm seeing some unusual combination of characters which is not possible to exist in Burmese scripts. I wonder if this kind of junk can affect training.
I've attached unicharset file I got.
Here is some sample which is not usual
တ္မြ 1 0,255,0,255,0,0,0,0,0,0 Myanmar 514 0 514 တ္မြ	# တ္မြ [1010 1039 1019 103c ]x
တ္က်ေ 1 0,255,0,255,0,0,0,0,0,0 Myanmar 515 0 515 တ္က်ေ	# တ္က်ေ [1010 1039 1000 103a 1031 ]x
ဥ္တြ 1 0,255,0,255,0,0,0,0,0,0 Myanmar 516 0 516 ဥ္တြ	# ဥ္တြ [1025 1039 1010 103c ]x
င္လေ 1 0,255,0,255,0,0,0,0,0,0 Myanmar 517 0 517 င္လေ	# င္လေ [1004 1039 101c 1031 ]x
ည္ဖြ 1 0,255,0,255,0,0,0,0,0,0 Myanmar 518 0 518 ည္ဖြ	# ည္ဖြ [100a 1039 1016 103c ]x
[output_unicharset.txt](https://github.com/tesseract-ocr/langdata/files/2946477/output_unicharset.txt)

Use NORM_MODE="3" with unicharset extractor command.

Tried norm_mode 2 and 3. Both has missing vowels and medial.

Hi,
Any further progress?

Please mention or let me know if something you need help for checking/fixing Burmese datasets, I'm gladly to be part of it. I've some experience in Python & Typescript. Cheers! all for helping to improve Myanmar Language in machines.

Hello guys. My name @BykovMAN, and your hear me to the international word.

I am new to tesseract. Recently I tried Myanmar language. It is still not perfected yet. I searched the training of data set and found this thread. However, it seems to be very old and no recent updates. I am not familiar with "how to train the data sets", but I know the language. Is there anyway that we can do to improve the Myanmar language? I also wish to understand how the training is done.

The training requires training data = lots of line images (.png) with corresponding transcription (.gt.txt). The original training used generated (artificial) line images, but meanwhile newer trainings for other scripts are often based on real line images from scanned books or newspapers. It's also possible to use a mix of artificial and real line images. You need as many lines as possible, and the text must cover all relevant glyphs (characters).
With enough lines for training, you can use tesstrain for the training.
Examples of training data for Latin script: https://code.bib.uni-mannheim.de/ocr-d/GT4HistOCR/src/branch/master/dta19/1827-heine_lieder.
Examples of training steps: https://github.com/UB-Mannheim/tesstrain/wiki/.

Thank you for providing the steps related to training. I think I will have
to try a lot to understand the steps.
For the moment, I am more on to providing the training data (.png and
.gt.txt). I know that even that one will take a lot of pairs. However, I am
willing to do more on that if there is anyone who can use these to train
it.  I believe that Myanmar script (image) is not much complicated (like
the German script example). Myanmar words are generally of the same shape
except the size might proportionately increase/decrease. If the basic ones
can be identified, the result will be improved. That is what I think.
Apologies, if what I said is too simple. What I meant is that myanmar
language shapes are quite consistent and different styles are rarely used,
and also I am willing to help with (.png and .gt.txt) if I were given more
detail requirements for providing these.
thanks and best regards,
On Fri, 21 Jun 2024 at 17:57, Stefan Weil @.***> wrote:

The training requires training data = lots of line images (.png) with
corresponding transcription (.gt.txt). The original training used
generated (artificial) line images, but meanwhile newer trainings for other
scripts are often based on real line images from scanned books or
newspapers. It's also possible to use a mix of artificial and real line
images. You need as many lines as possible, and the text must cover all
relevant glyphs (characters).
With enough lines for training, you can use tesstrain
https://github.com/tesseract-ocr/tesstrain/ for the training.
Examples of training data for Latin script:
https://code.bib.uni-mannheim.de/ocr-d/GT4HistOCR/src/branch/master/dta19/1827-heine_lieder
.
Examples of training steps: https://github.com/UB-Mannheim/tesstrain/wiki/
.
—
Reply to this email directly, view it on GitHub
https://github.com/tesseract-ocr/langdata/issues/13#issuecomment-2182567714,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BBGCV7H2E22XXWCBXPN2OSDZIQE3LAVCNFSM6AAAAABJRVPIOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBSGU3DONZRGQ
.
You are receiving this because you commented.Message ID:
@.***>


Some countries are not updated for at least 2 weeks

Hello, some countries are not updated for at least 2 weeks
e.g.
CA, USA
Canada
Russia
and some more
Is it because of quotas?

Yeah, the graphql API can be a bit flaky sometimes with fetching contribution counts for big countries. I countered this by adding retries and backoff. Should be more stable now. 👍

Thanks 👍

I think this issue has not been resolved yet since "NY, USA" & "CA, USA" last updated in "2017-05-16".

@mr-eyes that is a different issue - there is no US state-level recurring update set up at the moment.

@lauripiispanen I see. Thanks for the clarification.


Auto publication from Traefik

It is possible to automatically get all .local domains from Traefik and send them to avahi as CNAME.
Issue at Traefik: https://github.com/traefik/traefik/issues/6334
Example of implementation: https://gitlab.com/viraptor/docker_mdns

Not sure if this is needed or not. Version 2 introduced support for "subdomains".

I will close it for now.


Flutter_driver deprecated causing this error: "ext.flutter.driver" is not found in "extensionRPCs"

Hi there. Since flutter_driver is being deprecated in favor of integration_test and  enableFlutterDriverExtension() is not in that module. ¿Is there any other way to solve this issue?

Do you mean even after adding flutter_driver dependencies in your deps, no the module found error happens?
https://github.com/appium-userland/appium-flutter-driver/blob/e57bd279bdda85e504406830f5c443f281241786/example/flutter_app_under_test/pubspec.yaml#L18


Expect separator after null byte during encoding (v0.2)

Currently the spec does not have the separator occuring after null bytes encoded to bottom. Below is a list of implementations that must be updated to follow this encoding change. Decoding should not need to be changed, as long as it already followed the spec.

Implementation
Encode Compliant
Decode Compliant

spec (README)
- [ ]
- [ ]

bottom-bash
- [x]
- [x]

bottom-c
- [x]
- [x]

bottom-d
- [x]
- [ ]

bottom-dart
- [x]
- [ ]

bottom-dotnet
- [x]
- [ ]

bottom-ex
- [x]
- [ ]

bottom-go
- [ ]
- [ ]

bottom-go-bindings
- [ ]
- [ ]

bottom-hs
- [ ]
- [ ]

bottom-hy
- [ ]
- [ ]

bottom-java
- [x]
- [x]

bottom-lua
- [x]
- [ ]

bottom-matlab
- [ ]
- [ ]

bottom-php
- [ ]
- [ ]

bottom-py
- [ ]
- [ ]

bottom-py-bindings
- [ ]
- [ ]

bottom-rb
- [ ]
- [ ]

bottom-rs
- [x]
- [x]

bottom-swift
- [x]
- [ ]

bottom-web
- [ ]
- [ ]

bottom-webextension
- [ ]
- [ ]

bottom-wl
- [x]
- [ ]

bottom-workers
- [ ]
- [ ]

bottom-x86
- [x]
- [ ]

oneline-bottom-py
- [x]
- [ ]

power-bottom
- [ ]
- [x]

Also see: #2, bottom-software-foundation/bottom-rs#8, bottom-software-foundation/bottom-go#1

Maybe this should be changed into a table just so both encode and decode can be validated? Null byte was something I saw was ignored in a lot of implementations so it's possible some have a compliant encode but not a compliant decode.

Here's a partial table:

Implementation
Encode compliant
Decode compliant

bottom-bash
- [ ]
- [ ]

bottom-c
- [ ]
- [ ]

Source
| Implementation | Encode compliant | Decode compliant |
|---|---|---|
| bottom-bash |

- [ ]

 |

- [ ]

 |
| bottom-c |

- [ ]

 |

- [ ]

 |

Note there are zero width spaces

It might also be worth adding the version tested
That also means you can differentiate between implementations that have been verified to be noncompliant and implementations that haven't been checked yet
Without a standard testing suite it's hard to get automated testing on all of them, so it could also be possible that an implementation changes from being complaint to noncompliant, so logging versions could help with being clear about what was tested (although tracking changes after an implementation has become completely compliant is probably out of scope for this issue)

I've added the table. Decode is assumed not compliant until I review it or someone reviews it for me.

It might also be worth adding the version tested
That also means you can differentiate between implementations that have been verified to be noncompliant and implementations that haven't been checked yet
Without a standard testing suite it's hard to get automated testing on all of them, so it could also be possible that an implementation changes from being complaint to noncompliant, so logging versions could help with being clear about what was tested (although tracking changes after an implementation has become completely compliant is probably out of scope for this issue)

Out of the scope of this issue imo.

FYI, null byte encoding works fine for bottom-hs, and has always encoded separators after null bytes.
For evidence:

Here's the encoding logic for null bytes: https://github.com/bottom-software-foundation/bottom-hs/blob/f86c1fc88a772eb8a91469deea5b7df0b6cc2f76/src/Data/Encoding/Bottom.hs#L144
Here's new test case: https://github.com/bottom-software-foundation/bottom-hs/blob/f86c1fc88a772eb8a91469deea5b7df0b6cc2f76/test/Main.hs#L40
Here's a passing run of that test: https://github.com/bottom-software-foundation/bottom-hs/runs/2251350907

a standard testing suite

You could build one of these pretty easily by adapting the bottom-hs test suite to accept adapters for other CLIs. It uses both the bottom-rs tests as well as randomized property-based testing via QuickCheck.

Thanks! I've updated the table to match


refactor(charts.js): use switch instead of multiple if else

It reduces code and it is easier to read

Or maybe a key, value Map? That seems to more extensible.

I find the switch case easier to read. But that's just a matter of opinion.


Data Saturday Vienna

Issue created as organizers are interested in using this platform.

When they have a need for an event we can use the new process


Add support for person information from RIPE

Hi,
I found your script very useful but noticed that unfortunately it does not print out the whois record of type "person". Therefore I added this (only for RIPE at the moment) and it would be great if you could look into this and validate it. Maybe you have some suggestions for improvement.
If you agree with what I have coded so far I would continue my work and also look into the other registries.
Thanks,
Kind Regards
frennkie

Tracking this feature in https://github.com/secynic/ipwhois/issues/71


changed merge_type's types type hint, from Tuple[Type] to Tuple[Type, ...]

Description

Types of Changes

[ ] Core
[x] Bugfix
[ ] New feature
[ ] Enhancement/optimization
[ ] Documentation

Issues Fixed or Closed by This PR
Resolves #1347
Checklist

[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] I have tested the changes and verified that they work and don't break anything (as well as I can manage).

had to tell mypy to ignore type._type_definitions.field. Should there be a protocol to represent strawberry.type wrapped types?

In the future, we aim to stop monkeypatching .type_definition onto the wrapped type and instead have the function return entirely new StrawberryObject types. (When this happens, this function will no longer take Tuple[type, ...], but instead Tuple[StrawberryObject, ...].)
But for now, I think the mypy type: ignore comment you added is fine. Not sure why mypy didn't complain before this change though.


Semi-important typo in readme

Inside main argc and argv are mixed up. I considered fork and merging, but that seemed a bit much.
int main(int argv, char* argc[]) {
   START_EASYLOGGINGPP(argc, argv);
   ...
}

I was going to create the same issue, could you fix it for the next users ?


Treeshake

Tried to treeshake whatever possible. But shallowEqual still fails because of commonjs. Bad dependency.

Codecov Report

Merging #370 into master will decrease coverage by 4.24%.
The diff coverage is 80.89%.

@@            Coverage Diff             @@
##           master     #370      +/-   ##
==========================================
- Coverage   92.98%   88.73%   -4.25%
==========================================
  Files          51       50       -1
  Lines         328      373      +45
==========================================
+ Hits          305      331      +26
- Misses         23       42      +19

Impacted Files
Coverage Δ

src/packages/recompose/setStatic.js
100% <ø> (ø)
:arrow_up:

src/packages/recompose/setPropTypes.js
100% <ø> (ø)
:arrow_up:

src/packages/recompose/setDisplayName.js
100% <ø> (ø)
:arrow_up:

src/packages/recompose/renderNothing.js
100% <ø> (ø)
:arrow_up:

src/packages/recompose/renderComponent.js
100% <ø> (ø)
:arrow_up:

src/packages/recompose/getContext.js
87.5% <100%> (-12.5%)
:arrow_down:

src/packages/recompose/__tests__/utils.js
100% <100%> (ø)
:arrow_up:

src/packages/recompose/componentFromStream.js
100% <100%> (ø)
:arrow_up:

src/packages/recompose/withContext.js
87.5% <100%> (-12.5%)
:arrow_down:

src/packages/recompose/onlyUpdateForPropTypes.js
90% <100%> (-10%)
:arrow_down:

... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 52e3baf...0b54774. Read the comment docs.

I see no issues, @wuct please merge if all is ok from your side.

We need to think about moving fbjs shallowEqual into the recompose, this will solve that shallowEqual is not tree shaked and included with every import.

Or better, I introduce es modules in fbjs to make happy everybody.

Should we open a new issue for shouldComponentUpdate?

In react? Or you are about shouldUpdate hoc?

Oh, I mean recompose/shallowEqual. I am typing on my phone. sorry for the mistake.

I am going to merge this PR. Let's track shallowEqual in other issue.

ShallowEqual export is tree shaked. Fbjs/shallowEqual dep is not.

Okay

@TrySound Thanks for your great work!


URL fallback option

I have added fallback option (for those of use Windows) to use the absolute URL when no relativeUrl is available.
Any questions and comment are welcome...

Ah sorry, it looks like the "Relative URL" property is new as of svn version 1.8. You should be alright on Windows with 1.8 but I agree we should use a different property or have a fallback to at least support 1.7, so we'd welcome a PR.
Do you mind cleaning up the styles (please indent with spaces, not tabs) and leaving off the webstorm folder from the ignore list?

Of course, that's no problem. I forgot to check the coding style (sorry about that).
I'll get back to you with another PR.

Thank you for pointing out that svn 1.7 does not use the relative url property. I should really update my svn client...
Anyway, I have updated my fork by removing my earlier commits and committing/pushing the changes you requested.

Thanks! The changes should be available in v0.7.1.


Function call agent.config() throws an error if agent.clear_state() is called before it

Please check that this issue hasn't been reported before.

[X] I searched previous Bug Reports didn't find any similar reports.

Expected Behavior
It shouldn't cause an error
Current behaviour
It throws the following error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\User\GitHub\loopgpt\loopgpt\agent.py", line 441, in config
    "progress": self.progress[:],
TypeError: 'NoneType' object is not subscriptable

Steps to reproduce
import loopgpt

agent = loopgpt.Agent()
agent.name = "Node Exec Test"
agent.goals = "Run the list_files command and then the list_agents command"
agent_init_config = agent.config() # save init config

agent.clear_state() # start fresh
agent.from_config(agent_init_config) # reload cleared config
print(agent.config()) # 💀💀

Possible solution
Update line 346 in agent.py to this:
self.progress = []

Compare
Which Operating Systems are you using?

[ ] Linux
[ ] macOS
[X] Windows

Python Version

[ ] >= v3.11
[X] v3.10
[ ] v3.9
[ ] <= v3.8

LoopGPT Version
latest from main branch
Acknowledgements

[X] My issue title is concise, descriptive, and in title casing.
[X] I have searched the existing issues to make sure this bug has not been reported yet.
[X] I am using the latest version of LoopGPT.
[X] I have provided enough information for the maintainers to reproduce and diagnose the issue.

Python throws the error, "TypeError: 'SomeType' object is not subscriptable," when you attempt to use a bracket [] on an object that is not a list or dict, i.e. is not subscriptable. Currently line 346 of agent.py in clear_state() is,
346: self.progress = None

The error, "TypeError: 'NoneType' object is not subscriptable," is thrown on line 441 of agent.py in config(),
441: "progress": self.progress[:],

Switching line 346 to self.progress = [] fixes this, as you found. This is the same value assigned to self.progress on line 60 of agent.py in __init__(), so it seems like an acceptable solution to me.
Thinking I will add your example as a test case in test_serde.py as test_serde_after_clear_state() and submit a PR. If you feel this test case should go into its own file, please let me know. In general, test coverage is pretty abysmal. It would be good to have a parallel file structure for unit tests and some proper mocking, imo, but that is a separate discussion.

Thanks a lot for opening the issue @iskandarreza!


[widget utils/Fastload Utils] Initial .hide() will not hide widgets when fast loading, needs a .show() first

Affected hardware version
Bangle 2
Your firmware version
2v16.49
The bug
First discussed in https://github.com/espruino/BangleApps/pull/2464#issuecomment-1375354642.
I experience this with Podcast Addict Remote and Remote for Spotify, when fast loading from BW Clock (using Quick Launch) and from Desktop Launcher (both showing widgets) using Fastload Utils installed on the watch. The bug is not present if not using Fastload Utils.
I have yet to see if this bug is present when switching to other apps than the two mentioned using .hide(). I have also not tried launching them from other clocks or launchers.
Please propose other apps utilizing .hide() for testing this bug.
Reproduce:

Install Desktop Launcher and Fastload Utils
open Podcast Addict Remote
Notice widgets are not hidden.

Installed apps
kbmulti (0.05)
widbthide (0.01)
widram (0.03)
calculator (0.07)
activityreminder (0.10)
pomoplus (0.05)
widanclk (0.02)
kbtouch (0.02)
run (0.14)
heartzone (0.01)
wohrm (0.09)
sevenmin (0.01)
banglexercise (0.03)
sleeplog (0.12)
swp2clk (0.04)
torch (0.11)
hrm (0.11)
lightswitch (0.06)
boot (0.55)
recorder (0.20)
spotrem (0.07)
podadrem (0.07)
sleepphasealarm (0.12)
annoy (0.01)
messages (0.58)
widmsggrid (0.04)
messageicons (0.03)
messagesmusic (0.05)
slopeclock (0.04)
tetris (0.01)
smpltmr (0.07)
weather (0.23)
circlesclock (0.23)
alarm (0.37)
android (0.20)
setting (0.56)
dragboard (0.07)
aiclock (0.07)
widminbat (0.02)
notify (0.12)
quicklaunch (0.08)
taglaunch (0.02)
altimeter (0.02)
messagelist (0.02)
health (0.20)
bwclk (0.28)
dtlaunch (0.22)
sched (0.19)
widalarm (0.01)
swscroll (0.02)
fastload (0.03)

/*
1. Upload this script to a file "testWU.js" via Web IDE.
2. Navigate the watch to a clock/launcher/app with remove-function specified and widgets shown (I use BW Clock).
3. In console field enter "Bangle.load('testWU.js')".
4. Notice if widgets are hidden or not (Most of the time they would not be hidden for me and be interactable ,e.g. Light Switch Widget, the red back button will be hidden but interactable).
5. Click hardware button to load the user chosen clock (for the following steps it should have a remove function, or specify another app in the back function below)
6. Notice if widgets are shown or not (On BW Clock they are shown as expected).
7. In console field again enter "Bangle.load('testWU.js')".
8. Notice if the widgets are hidden or not (They are not hidden for me).
9. In console field again enter "Bangle.load('testWU.js')".
10. Notice if the widget are hidden or not (This time they are hidden for me, however if I tap upper left corner the back widget still executes and if I tap the upper right corner Light Switch Widget will be triggered).
11. Press hardware button to load the user chosen clock.
12. Notice if widgets are shown (they are shown for me).

The expected behavior would of course be that the widgets are always hidden when testWU.js is launched.
*/

{
g.clear();

  Bangle.setUI({
    mode : "custom",
    back : ()=>{Bangle.load();},
    remove : ()=>{require('widget_utils').show();}
  });

  Bangle.loadWidgets();
  require('widget_utils').hide();
}

@halemmerich any ideas about this one?

@thyttan I have tried your steps with BWClock and I see the same behaviour there.
Tests with Anton and newly fastloadable Berlin Clock do not show the problem.
So at first glance I would suspect a problem in BWClock. It does not seem to use widget_utils for hiding and showing the widgets so that is probably the cause or at least part of it.

I'll try adding widget utils in BW Clock then, then we can check it off on #2196 as well. I looked at doing that before but didn't really understand the current handling of widgets though 🙃 Maybe I'll get it this time.

So when I add widget utils to BW Clock .hide() works directly with the test code above as well as the two aforementioned apps.
However, interactable widgets such as the back button (via setUI back function) and Light Switch Widget will still be triggered if I touch where they usually are.

Aha! :)

I don't know if something should be done so .hide() works even in the case when loading from a clock app which doesn't use widget_utils? Otherwise it's ok with me to close this.
#2492 Converts all remaining clocks to use widget_utils.

I don't think that is easily doable. The problem was BWClock attempting the same things as widget_utils (renaming and changing draw/_draw). So essentially converting everything to widget_utils and having an eye on future changes trying to do something like it should solve this.

Thank you both for checking this out!

I think that is expected since hide/show currently only handles the drawing code, not the listeners of the loaded widgets.

Yes, it is. Not ideal but I think for most cases it's not an issue.

I don't know if something should be done so .hide() in an app works even in the case when loading from a clock app which doesn't use widget_utils?

Fast load should only work for apps that have 'opted in' by adding remove listener - so really if an app has added 'fast load' and it's breaking other stuff because of something like its widget hiding, that means that whoever added it didn't test it properly.


[Bug] can not connect with a VNC

General

[X] I have updated to the latest version of Nexterm.
[X] My bug has not been reported yet.

The Bug
I can not connect with my VNC Server.
guacd[8]: INFO:	Guacamole proxy daemon (guacd) version 1.6.0 started
guacd[8]: INFO:	Listening on host <IP_ADDRESS>, port 4822
Successfully connected to the database file
Server listening on port 6989
Authorized connection to server vnc with identity Auto-generated identity
Error in Guacamole WebSocket: Error: connect ECONNREFUSED ::1:58391
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16) {
  errno: -111,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '::1',
  port: 58391
}

I am using Nexterm on a ARM64 Server.
What device are you using to access the page?
In the browser
Which operating system is your Nexterm instance running on?
Linux

now I've tried RDP and VNC on a X86 CPU.
it does not work. SSH works well.
Just as feedback by the way: I love this idea, and the implementation of the web interface so far. If it works, it's sure to be great.

Thanks! This might be related to #3 so it should be fixed with the new PR I made. If it does not get fixed within the next version, feel free to re-open this issue. :)


When no folder or file open, the debug play button should not be shown

Open a new VS Code window with no folder or file open
Go to the debug viewlet.
The gear icon is missing which makes sense as there is no workspace to create the launch.json file in
The play icon is present. Clicking on it opens the "Select environment" action

Choosing any option obviously would result in an error.
We should either not show the play icon at all or on clicking on it should show an error saying "no file/folder to debug" instead of showing the "Select environment" action

Thanks for your feedback, however some debuggers support debugging in no folder case - example node debug.
I also do not want to have a special case of not showing the play button if there are no files in the workspace. Since that could also in theory be covered by the debugger
If you are not getting a proper error message that is up to the debug adapter and please file accordingly to go or node, thanks


Which accessibility testing tool should you use?

Overview
There are more than 100 accessibility testing tools.  Figuring out which ones to use can be a black hole.  For guidance we recommend this article:  Which accessibility testing tool should you use?
Summary of Article
The author recommends using the tools in the following order fixing as you go along, since no one tool catches all the relevant issues

[x] aXe - Bruce
[x] SiteImprove - Bhagya
[x] WAVE - Nandana
[x] Lighthouse - Max

Tenon will be skipped since it is throwing errors
But if you want to test your site with other tools, here is a bigger list
List of Issues:
List of Issues:

#1094
#1102
#1112
#1122
#1107
#1106
#1104
#1101

Spreadsheet with ongoing audit results is here:
https://docs.google.com/spreadsheets/d/1-XfxuJQCg6dJb4fFyITfDvYYWvUCL4oXYnJ_rsR77EQ/edit#gid=2087742937

Spreadsheet with ongoing audit results is here:
https://docs.google.com/spreadsheets/d/1-XfxuJQCg6dJb4fFyITfDvYYWvUCL4oXYnJ_rsR77EQ/edit#gid=2087742937

Per @cnk Caltech uses Wave. Here's a link to tool choices for reference from the CMS she works with https://wagtail.io/accessibility/

@bruceplai and dev team to do additional run through to make sure there are no duplicate documented issues. Afterwards if support is needed in creating issues please tag @kevindphan and I in the comments

@ladissi @kevindphan Issues have been consolidated in the spreadsheet.  You may need to consult with @chalimar due to some of the issues being related to design (for example 1) sufficient color contrast for elements such as links and buttons 2) sequential heading text sizes to help the user navigate the page hierarchy)

@bruceplai to set some time with UI/UX and include PM's as optional to review issues

@chalimar FYI a session will be needed to determine how the accessibility updates will need to be implemented through a ui/ux lens. @bruceplai will be reaching out to you

@chalimar after more analysis for the sequential header accessibility issue, it seems aXe really does want headers to increase only one level at a time as you dig deeper into the page hierarchy.  On some pages the header size updates won't be noticeable, but on others it will.  It's not a high severity level issue like color contrast.  How should we handle it?  Leave the headers as is?

Will discuss with Chali

@chalimar please provide update

Progress
Blockers
Availability
ETA

Dev team to create issues that dont require UI/UX based off spreadsheet. To be placed in New Issue Approval for PM's to review

Comment issues that were created from this issue.

#1101

@chalimar It looks like there are design issues that need to be discussed.  Can I help you review that and make issues.  We want to lock this down before you get massively busy. lol, I know you are already busy.

#1112

If there are more accessibility issues, let's open up new tickets. Closing this issue now


Add new maintainers

As voted on per e-mail, adding
new maintainers to the list
Welcome!
- A picture of a cute animal (not mandatory but encouraged)

ping @docker/core-engine-maintainers, @dnephin @mlaventure @stevvooe @aluzzardi @vieux

LGTM

LGTM

LGTM 👼

LGTM

LGTM :tada:

🎉

❤


[feat]: Unjoin drizzle-orm (schema definition and query build)

Feature description
Hi.
This is a great project.
However using it on an existing codebases makes it very much difficult to adhere.
For example, last week I integrated on my project which uses Supabase successfully converting to (postgrest query builder). It took me a few hours though.
However, there was a huge refactoring (44 files) recently to fix the count on filters, which made it mostly impossible to keep up. Also, currently there is much dead code (for example mostly of utils.ts).
You could suggest me to use drizzle on the data-table specifically, and i would agree with that. I spent a few hours doing that, however, the code itself is also so much relied on the schema definition, which makes this almost an impossible task.
Even though i was willing to use drizzle to make it easier on the query builder part and make easier to maintain, turns down that i just cannot use it to define my table structure.
So in my humble opinion, code should be more detachable in the aspect of defining schema/and consuming the schema on query buillders.
At the moment, they are much binded.
At the end, in spite of being a great project, it mostly serves as an inspiration, because its very hard to adhere in current codebases, must be said.
What do you think?
Additional Context
Additional details here...
Before submitting

[X] I've made research efforts and searched the documentation
[X] I've searched for existing issues and PRs

it shouldn't be that hard to integrate with existing drizzle project.
the db utils stuffs were just a bunch of utils that i use for my other drizzle projects, you can just skip them if those are irrelevant to you.
you just need to update the schema, and the query, that should be enough.
in which parts are you having trouble implementing?
i can help you set it up, if you want

@tecoad want to start a fork actually? Literally in the middle of this (~80% done) right now, moving the whole thing to Supabase and came here for the same reason. LOTS of subtly required changes, not as easy as rip out Drizzle 🙁

i've been working on a primitive ui component library since the last week.
gonna push something more accessible within the next month.
also will add docs for the table and other stuffs.
@danielgwilson can you say which parts of it was difficult to swap out?


Refactor: hardhat-deploy, contractLoader, signers, ethers, rollup-readiness

Hey! Sorry this is quite a big PR...
Introduce hardhat-deploy

This is a wondrous package for managing hardhat deployments
Deployment scripts in the /deploy folder are run in alphabetical order, replacing the custom deploy.js script
Deployments are stored by network in the /deployments folder - there is a native export function, and I updated the deploy command to export to the react-app
Re-worked the publish.js script to only publish to the subgraph package

Re-work ContractLoader

Updated the ContractLoader to use the new hardhat-deploy export file
Loads contracts based on the provider or signer network chainId (though this can be overriden in the second network object parameter). Note that if (edge case) you have multiple hardhat networks on the same chainId, with the same name, they will be overwritten
Replaced the CustomContractLoader by adding another parameter to map custom addresses based on contract names
Replaced ExternalContractLoader - you can now add abis & addresses into contracts/external_contracts.js, and these will be loaded in by the ContractLoader. DAI is in there as an example. Same network rules apply (uses the provider, unless network.chainId is specified)
Hopefully it is easier than ever to manage contracts in your frontend, based on local deployments or otherwise!

Remove BurnerProvider, Replace userProvider with userSigner

Replaced BurnerProvider package with a lightweight BurnerSigner hook - just fetches the key from localStorage
We have been using web3providers forever. Felt like an ok time to move to the true way of ethers.js signers
Updated useUserProvider -> useUserSigner, so it either uses a BurnerSigner, or returns injectedProvider.getSigner()
Updated Transactor and other places accept providers or signers, so this will hopefully not break anything

Tidy up ethers versions

We were importing some random old versions for specific ethers functionality. I stripped all of them out so we just have the one ethers dependency

Rearrange packages

Move the /docker folder (currently just the graph-node docker setup) into a services package

Ready for Rollups

TBD on how we want to handle this, but I am jazzed to make scaffold-eth a super easy place to play with L2s...
Added L2 local and testnet configs to hardhat config, and to the constants file in the react-app
Added submodules to Optimism and Arbitrum
Added commands for running both rollups locally

Other

Should fix https://github.com/austintgriffith/scaffold-eth/issues/190
Small gasPrice tweaks

Can't wait to see this one merged 🦍😁🔥


Layer Article example, footer doesn't stretch 100%

Layer Article example has a footer that doesn't stretch to be 100% of the screen.
https://grommet.github.io/hpe/docs/layer/examples#3
Your Environment

Browser Name and version: Firefox 48.0 and Chrome 53.0.2785.116

Fixed by adding flush option to the article example
https://github.com/grommet/grommet-docs/commit/30be35300720c5e1279118c38dc1677ea864a298


bugfix:issue #94, typo in plugin

change log content as below:
core.log.warn("plugin balancer phase, conf: ", core.json.encode(conf))

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@zhiyong0804 merged, many thx.


Code does not work.

Please fix this repository, as the code here is not functional.

Thanks for the quick fix @NLNHSR!

No problem @ravibandaru-lab!


[Event Request] T1301 "Item Template" - procedure InsertItemFromTemplate()

procedure InsertItemFromTemplate(ConfigTemplateHeader: Record "Config. Template Header"; var Item: Record Item)
var
begin
  ----------------------- ADD EVENT ----------------------
  IsHandled := false;
  OnBeforeUpdateBaseUnitofMeasureFromItem(IsHandled)
  if not IsHandled then begin
      if Item."Base Unit of Measure" = '' then begin
            ....
      end;
  end;
  ----------------------------------------------------------

....................
[IntegrationEvent(false, false)]
local procedure OnBeforeUpdateBaseUnitofMeasureFromItem(var IsHandled: Boolean)
begin
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Stabilize -Ztimings as --timings

The -Ztimings option has existed for years, and many people use it to
profile and optimize their builds. It's one of the common reasons people
use nightly cargo.
The machine-readable JSON output may warrant further careful inspection
before we commit to a stable format. However, for the human-readable
output we don't need to make any commitment about the exact output.
Add a --timings option, as the stable equivalent to -Ztimings.
Stabilize the info and html output formats, and require
-Zunstable-options for the json output format.
Document the new option, and update the testsuite.

r? @Eh2406
(rust-highfive has picked a reviewer for you, use r? to override)

Thanks for moving this forward!
I'm reluctant to block progress, because I know some people find this feature useful as-is. However, from my perspective it was never finished. I'm just a little leery of stabilizing something that might be somewhat buggy or a poor user experience. There are some concerns I'd like to talk about:

I don't know if it is useful to have the info form. I haven't seen anyone indicate that it is useful, and it does add a lot of noise (I have never found it useful). The tracking issue also had some discussion about different ways it could be presented.
The location where the html file is saved may not be ideal (the current directory). It creates a mess of files that the user then needs to manually clean up. I've been thinking it may make more sense to save them somewhere in the target directory?
The canvas drawing mechanism has issues.  It breaks on large graphs (no graphs appear). Adjusting the sliders incorrectly can cause it to break. It is dependent on the browser and graphics card memory limits.  Perhaps it works well enough for most people?
I've noticed some people being a little confused about how to read the information.  Ideally I think there would be more built-in help to understand the information.  (This isn't any sort of blocker, just a note about it being an overall poor user experience.)

Relatedly, some users have been confused due to it not being obvious that you need to scroll horizontally.

The mouse hover drawing can be quite slow on large graphs.

Also, just FYI, I've been wanting to support generating reports after the build.  Maybe something like cargo report timings, which would generate a report for the most recent build. This would require saving the raw information every time, assuming it isn't too slow.  I think it is still useful to have something like --timings to generate reports on every build, though.
Overall I'm excited to move forward! However, I think there might be a few small things to address first (especially the first two points).

@ehuss wrote:

Thanks for moving this forward!
I'm reluctant to block progress, because I know some people find this feature useful as-is. However, from my perspective it was never finished. I'm just a little leery of stabilizing something that might be somewhat buggy or a poor user experience. There are some concerns I'd like to talk about:
* I don't know if it is useful to have the `info` form. I haven't seen anyone indicate that it is useful, and it does add a lot of noise (I have never found it useful). The tracking issue also had some discussion about different ways it could be presented.

While I think it's noisy enough to debate whether to include by default, I do think it adds value when trying to investigate build performance. Sometimes you want the full HTML graph, but sometimes you just want information at the granularity of "Did that specific crate take as long as it looked like it did? Wow, that one takes 20 seconds all by itself."
I think it adds enough value to be worth turning on by default with --timings, but I'm not especially attached to it if you consider it a blocker.

* The location where the html file is saved may not be ideal (the current directory). It creates a mess of files that the user then needs to manually clean up. I've been thinking it may make more sense to save them somewhere in the target directory?

The target directory has the disadvantage of being cleaned by cargo clean, and when benchmarking build performance you may want to do many clean builds and compare the results. Yes, the user has to manually clean them up when done with them, but this avoids accidentally losing valuable data that takes time to collect.
Plus, git status will show them, which will get the user to notice them and clean them up eventually when not needed.
I agree that it isn't the perfect place to put them, but I also don't have a better suggestion. We don't currently have anything else of the form "semi-precious but still a build output, don't clean lightly but make easy to clean".
If you'd prefer, I could make the option create a directory cargo-timing and create all the reports under that, which would make them much easier to clean up with one rm -r cargo-timing.

* The canvas drawing mechanism has issues.  It breaks on large graphs (no graphs appear). Adjusting the sliders incorrectly can cause it to break. It is dependent on the browser and graphics card memory limits.  Perhaps it works well enough for most people?

I haven't seen the former, but I have seen the latter with settings of the sliders. I think it works well enough to be quite useful, and we can always fix further issues in the future.

* I've noticed some people being a little confused about how to read the information.  Ideally I think there would be more built-in help to understand the information.  (This isn't any sort of blocker, just a note about it being an overall poor user experience.)

  * Relatedly, some users have been confused due to it not being obvious that you need to scroll horizontally.

* The mouse hover drawing can be quite slow on large graphs.

I think all four of these points ware worth evaluating and working on further; it could be more self-explanatory and better presented. But I don't think these are blockers either. And once --timings becomes more widely available, I'm hoping that leads more people to contribute to its usability.
I think the information is more valuable than the exact format.

Also, just FYI, I've been wanting to support generating reports after the build. Maybe something like cargo report timings, which would generate a report for the most recent build. This would require saving the raw information every time, assuming it isn't too slow. I think it is still useful to have something like --timings to generate reports on every build, though.

I love that idea, if it doesn't add significantly to build time. I did notice that the Timings class went out of its way to turn itself into a stub if disabled, so I assumed that it had a non-trivial performance impact. (Not least of which, it appears that having it enabled at all will make rmeta_required return true when it otherwise wouldn't have.)

Overall I'm excited to move forward! However, I think there might be a few small things to address first (especially the first two points).

Happy to address any of these that you consider blockers, and happy to document the rest in issues with an appropriate label.

Personally I agree that it's probably time to stabilize this given how useful it's been throughout its unstable lifecycle (yay nightly sorta?). I agree with all of @ehuss's points but I think that for stabilization, as @joshtriplett mentions as well, we don't necessarily need to solve all of them. I think it'd be good to think about what stability of this option means and only focus on that part. For example I wouldn't consider the literal HTML output stable at all, only that we do output HTML with something human-readable. In that sense any UI issues or performance issues of the HTML itself I think are fine to say "good enough for now and can be improved later".
On the other hand though the fact that we output HTML I think is something worth scrutinizing, in addition to the interface requesting it (e.g. --timings or not). In that sense I don't think that we ever really seriously considered what a stable interface for this would be, so I think it might be worthwhile to consider that before stabilizing.
Personally I would love it if we could enable this by default and always emit target/debug/timings/cargo-timings.html or something like that (along with a duplicate timestamped-file). If we did that then we wouldn't need a CLI flag at all and we could have a section of documentation explaining it and how to read the output as well (and the generated HTML itself could perhaps link to our documentation as well). If we could go this route it's also the smallest to stabilize I think where it's simply an extra file being produced on the build, and deciding the path for that shouldn't be too too bad.

:umbrella: The latest upstream changes (presumably #10254) made this pull request unmergeable. Please resolve the merge conflicts.

I've rewritten this PR to remove -Ztimings=info, and to move the timings reports to target/cargo-timings.

Rebased on current HEAD to pick up recent CI fixes.

OK, I believe I've now addressed all the feedback, and I think this is ready.

This all seems reasonable to me, thanks!
I think this is a pretty reasonable place to propose:
@rfcbot fcp merge

Since we have unanimous signoff:
@bors r+

:pushpin: Commit 85589e1a8c121ef4febfdc5713484ba6516a16d4 has been approved by joshtriplett

:hourglass: Testing commit 85589e1a8c121ef4febfdc5713484ba6516a16d4 with merge 3bc0e6d83f7f5da0161ce445f8864b0b639776a9...

:sunny: Test successful - checks-actions
Approved by: joshtriplett
Pushing 3bc0e6d83f7f5da0161ce445f8864b0b639776a9 to master...