Getting TypeError: 'NoneType' object is not iterable on training model

Rasa version: Rasa 1.7.2
Python version: Python 3.6.8
Operating system:  OSX
Issue: Type error on the training rasa model
Error (including full traceback):
2020-04-01 17:13:56 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu.md' is 'md'.
2020-04-01 17:13:56 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/responses.md' is 'nlg.md'.
2020-04-01 17:13:56 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/stories.md' is 'unk'.
2020-04-01 17:13:56 DEBUG    pykwalify.compat  - Using yaml library: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ruamel/yaml/__init__.py
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/core/domain.py:148: FutureWarning: No tracker session configuration was found in the loaded domain. Domains without a session config will automatically receive a session expiration time of 60 minutes in Rasa version 2.0 if not configured otherwise.
  session_config = cls._get_session_config(data.get(SESSION_CONFIG_KEY, {}))
2020-04-01 17:13:56 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/responses.md' is 'nlg.md'.
2020-04-01 17:13:56 DEBUG    rasa.nlu.training_data.loading  - Training data format of 'data/nlu.md' is 'md'.
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/bin/rasa", line 11, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/__main__.py", line 76, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/cli/train.py", line 76, in train
    additional_arguments=extract_additional_arguments(args),
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/train.py", line 50, in train
    additional_arguments=additional_arguments,
  File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/train.py", line 101, in train_async
    additional_arguments,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/train.py", line 144, in _train_async_internal
    file_importer.get_stories(), file_importer.get_nlu_data()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/importers/importer.py", line 269, in get_nlu_data
    nlu_data = await asyncio.gather(*nlu_data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/importers/rasa.py", line 61, in get_nlu_data
    return utils.training_data_from_paths(self._nlu_files, language)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/importers/utils.py", line 9, in training_data_from_paths
    training_datas = [loading.load_data(nlu_file, language) for nlu_file in paths]
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/importers/utils.py", line 9, in <listcomp>
    training_datas = [loading.load_data(nlu_file, language) for nlu_file in paths]
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/nlu/training_data/loading.py", line 67, in load_data
    data_sets = [_load(f, language) for f in files]
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/nlu/training_data/loading.py", line 67, in <listcomp>
    data_sets = [_load(f, language) for f in files]
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/nlu/training_data/loading.py", line 138, in _load
    return reader.read(filename, language=language, fformat=fformat)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/nlu/training_data/formats/readerwriter.py", line 15, in read
    return self.reads(rasa.utils.io.read_file(filename), **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/nlu/training_data/formats/markdown.py", line 75, in reads
    self._parse_item(line)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/nlu/training_data/formats/markdown.py", line 116, in _parse_item
    parsed = self.parse_training_example(item)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/nlu/training_data/formats/markdown.py", line 180, in parse_training_example
    message = Message.build(plain_text, self.current_title)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/rasa/nlu/training_data/message.py", line 75, in build
    split_intent, response_key = cls.separate_intent_response_key(intent)
TypeError: 'NoneType' object is not iterable

Command or request that led to error:
rasa train -c ./config.yml --debug
Content of configuration file (config.yml) :
language: en
pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100
policies:
  - name: MemoizationPolicy
    max_history: 1
  - name: MappingPolicy

Content of domain file (domain.yml) :
intents:
- greet
- bye
- thank
- faq
responses:
  utter_noworries:
  - text: No worries!
  utter_greet:
  - text: Hi
  utter_bye:
  - text: Bye!
actions:
- respond_faq

Thanks for raising this issue, @alwx will get back to you about it soon✨
Please also check out the docs and the forum in case your issue was raised there too 🤗


KMeans: Multi-GPU performance saturated even with init='random'

10-15s with 1 or 2 GPUs on 1080ti for homesite kmeans notebook.  As if (compared to previously) improvements allowed for across the board for all GPU counts, but saturated in performance for a different reason that was possibly related to why 4 and 8 GPUs had similar performance before.

One issue I noticed is even if we set random_state=1 in both cases (1 and 2 gpu), the total number of iteration differs. 2 GPU version is running for over 200iter, while 1 GPU version finishes around 150. Both cases are reproducible (i.e. 1GPU always runs for ~150, 2GPU always ~200) but they should be both similar. The only reason why this would happen I can come up with on the spot is because in 1GPU case we do scaling on the GPU while in the 2GPU version on the CPU.

On volta I get very poor performance with the kmeans image notebook.
GPU#   Time for fit
1            44s
2            29s
4            24s
8            30s # after warmup
8            45s  # first time run on gpus
The first time ran, it seems to be sequentially allocating on the GPU, as if you have some cuda operation (e.g. sync or info getter) that is sequential and getting the cuda context setup sequentially).  Need to avoid that.
For post-warmup, see low GPU usage for 8 GPUs:

Even larger works, although slow.

spends most of the time on the CPU doing single-threaded work.  Pokes on the GPU (1 at a time) sometimes.
Only after a while does it do use all GPUs in parallel:

Then it goes back to single-threaded stuff on CPU with no GPU usage.

import h2o4gpu
import numpy as np
>>> X = np.random.rand(80636906,34)
>>> model = h2o4gpu.KMeans(n_clusters=5,random_state=1234).fit(X)
Segmentation fault (core dumped)

ython 3.6.4 (default, Mar 23 2018, 19:10:33)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import h2o4gpu
>>> import numpy as np
>>> X = np.random.rand(80636906*3,50)
model = h2o4gpu.KMeans(n_clusters=5,random_state=1234).fit(X)
>>> model = h2o4gpu.KMeans(n_clusters=5,random_state=1234).fit(X)
>>> y = np.random.rand(80636906, 15)
>>> model = h2o4gpu.KMeans(n_clusters=5,random_state=1234).fit(y)
>>> %model = h2o4gpu.KMeans(n_clusters=5,random_state=1234).fit(y)
  File "<stdin>", line 1
    %model = h2o4gpu.KMeans(n_clusters=5,random_state=1234).fit(y)
    ^
SyntaxError: invalid syntax
>>> import time
>>> starttime=time.time() ; model = h2o4gpu.KMeans(n_clusters=5,random_state=1234).fit(y) ; diff=time.time()-starttime
>>> starttime=time.time() ; model = h2o4gpu.KMeans(n_clusters=5,random_state=1234, n_gpus=4).fit(y) ; diff4=time.time()-starttime
>>> diff
16.521016120910645
>>> diff4
18.117528438568115
>>> starttime=time.time() ; model = h2o4gpu.KMeans(n_clusters=5,random_state=1234, n_gpus=2).fit(y) ; diff2=time.time()-starttime
>>> diff2
26.564540147781372
>>>

Couldn't do 1 GPU, wouldn't fit. Could redo, but too late.
So at least not slower with 8 GPUs, but not much faster.  Mostly limited by transfer and kmeans|| I think, but probably other improvements can be made.
Also can do big problem not possible on less GPUs:


Add support for Delete

Would be useful to be able to delete keys in place as a transform, does not currently exist.

@JoshKCarroll you want to perform the delete operation ?


Bluetooth: Controller: No CIS established event if CIS failed to be established on peripheral

Describe the bug
If the peripheral host accepts a CIS request that then fails to be established, the host seemingly never receives a Disconnect event.
To Reproduce
N/A
Expected behavior
The spec states:

When the Controller receives the HCI_LE_Accept_CIS_Request command, the Controller sends the HCI_Command_Status event to the Host. An HCI_LE_CIS_Established event will be generated when the CIS is established or is considered lost before being established.

Impact
Can put the host into a non-working state, which requires a complete reset.
Logs and console outputN/A
Environment (please complete the following information):

Commit SHA or Version used: 1b7ef97c31f4b734cae9cfb2b251c1b86b40d109

Additional context
N/A

@Thalley Does this commit influence this GH issue? Was the host receiving the Failed to be Established event from the controller? https://github.com/zephyrproject-rtos/zephyr/commit/db588104921ac8bfd3f4c2b1b41c84cf5e710ec2

@erbr-ot Could you check if Failed to be Established reason will be returned to Peripheral Host by this code:
https://github.com/zephyrproject-rtos/zephyr/blob/114548ef607aefd712dda7e5ea954981e778185b/subsys/bluetooth/controller/ll_sw/ull_llcp_cc.c#L254-L287
Controller is doing this:
https://github.com/zephyrproject-rtos/zephyr/blob/114548ef607aefd712dda7e5ea954981e778185b/subsys/bluetooth/controller/ll_sw/ull_llcp.c#L1347-L1366

@Thalley Does this commit influence this GH issue? Was the host receiving the Failed to be Established event from the controller? db58810

No, the commit from Rubin only affects the host, and this issue is for the controller


Chinese translation

Hey. Can you provide a list of the words or text that needs to be translated?

Hey. Can you provide a list of the words or text that needs to be translated?

Hey, you can see an example here (Spanish) -> en. You should basically create another directory for Chinese and keep the same file structure.
You should also add zk in this list -> https://github.com/semaphore-protocol/website/blob/cfab949e2ab476b23663464ba68043fbf73d9d17/docusaurus.config.js#L24
If you want to test it locally you can create a command like this -> https://github.com/semaphore-protocol/website/blob/cfab949e2ab476b23663464ba68043fbf73d9d17/package.json#L7

Hey @SofiiaVarich! Great! If you have more questions, please let us know.


Simplify and Agree on .nena formatting guidelines

The .nena formatting guidelines are now a bit old and have moved past the draft stage. As @jamespstrachan builds the text input tool, we should think more carefully about what should absolutely go in to the .nena format and what we should leave out as an unnecessary complication.
An example of a feature in the draft documentation that is probably superfluous is:
line breaks – marked with / and //. These were intended to preserve poetic indentations from .docx sources. But those features seem less relevant in relation to the new audio files.
A questionable example is comments – which are surrounded in brackets and marked with a speaker: (GK: text of interjection?). Do we want to keep this kind of data in the .nena format? @GeoffreyKhan is this kind of thing something you need to be able to do?
Some things that should absolutely be kept include language markers. The suggested markup currently is, e.g., <E>Hello<E>. So maybe this should currently be done in the same way while inputting text? In .docx these values are normally indicated via superscript letters. @GeoffreyKhan would you be comfortable placing <> tags around such letters when you do your copying/pasting?

A questionable example is comments
https://github.com/CambridgeSemiticsLab/nena_corpus#comments – which
are surrounded in brackets and marked with a speaker: |(GK: text of
interjection?)|. Do we want to keep this kind of data in the |.nena|
format? @GeoffreyKhan https://github.com/GeoffreyKhan is this kind of
thing something you need to be able to do?
GK: These are not necessary for the database.

Some things that should absolutely be kept include language markers
https://github.com/CambridgeSemiticsLab/nena_corpus#text-markup. The
suggested markup currently is, e.g., |Hello|. So maybe this
should currently be done in the same way while inputting text? In
|.docx| these values are normally indicated via superscript letters.
@GeoffreyKhan https://github.com/GeoffreyKhan would you be
comfortable placing |<>| tags around such letters when you do your
copying/pasting?

GK: That would be fine.
thanks
Geoffrey

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/CambridgeSemiticsLab/nena_corpus/issues/7, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AMC4DG4323333GDLJQ66MV3RH6VHFANCNFSM4LNUWK4Q.

--
Geoffrey Khan
Regius Professor of Hebrew
University of Cambridge
Faculty of Asian and Middle Eastern Studies
Sidgwick Avenue
Cambridge CB3 9DA
UK

This has been implemented in https://github.com/CambridgeSemiticsLab/nena_corpus/blob/master/docs/nena_format.md#Foreign-Language-Tag
Also see /standards.
And it is now incorporated into a new parser.


TypeError: Cannot read property 'forEach' of undefined

When I follow the instruction to create the extension app, the below error occurred:
 WARN  There are uncommitted changes in the current repository, it's recommended to commit or stash them first.
? Still proceed? Yes

📦  Installing vue-cli-plugin-chrome-extension-cli...

yarn add v1.22.11
[1/4] Resolving packages...
[2/4] Fetching packages...
info<EMAIL_ADDRESS>The platform "win32" is incompatible with this module.
info<EMAIL_ADDRESS>is an optional dependency and failed compatibility check. Excluding it from installation.
info<EMAIL_ADDRESS>The platform "win32" is incompatible with this module.
info<EMAIL_ADDRESS>is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved 1 new dependency.
info Direct dependencies
└─<EMAIL_ADDRESS>info All dependencies
└─<EMAIL_ADDRESS>Done in 15.09s.
✔  Successfully installed plugin: vue-cli-plugin-chrome-extension-cli

🚀  Invoking generator for vue-cli-plugin-chrome-extension-cli...
 ERROR  TypeError: Cannot read property 'forEach' of undefined
TypeError: Cannot read property 'forEach' of undefined
    at module.exports (D:\myproj\autorobot\node_modules\vue-cli-plugin-chrome-extension-cli\generator\index.js:24:14)
    at Generator.initPlugins (C:\Users\tristany\AppData\Roaming\npm\node_modules\@vue\cli\lib\Generator.js:150:13)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Generator.generate (C:\Users\tristany\AppData\Roaming\npm\node_modules\@vue\cli\lib\Generator.js:168:5)
    at async runGenerator (C:\Users\tristany\AppData\Roaming\npm\node_modules\@vue\cli\lib\invoke.js:111:3)
    at async invoke (C:\Users\tristany\AppData\Roaming\npm\node_modules\@vue\cli\lib\invoke.js:92:3)

sorry, I can’t reproduce, please provide the os system, node.js version or yarn version, vue-cli version, and the version that created vue

I tried another laptop. The issue is gone. But on my current laptop, the issue stays.
Here is the information(all commands are executed in powershell 7.2)
PS D:\myproj\autorobot> npm version
{
  autorobot: '0.1.0',
  npm: '6.14.14',
  ares: '1.17.2',
  brotli: '1.0.9',
  cldr: '39.0',
  icu: '69.1',
  llhttp: '2.1.3',
  modules: '83',
  napi: '8',
  nghttp2: '1.42.0',
  node: '14.17.5',
  openssl: '1.1.1k',
  tz: '2021a',
  unicode: '13.0',
  uv: '1.41.0',
  v8: '8.4.371.23-node.76',
  zlib: '1.2.11'
}
PS D:\myproj\autorobot> yarn --version
1.22.17
PS D:\myproj\autorobot> node -v
v14.17.5
PS D:\myproj\autorobot> vue --version
@vue/cli 4.5.15

Add OS information:
Microsoft Windows [Version 10.0.19042.1288]

I clean my cache, remove all lock files. Then, the issue disappeared.
Strange for a while
Thanks @sanyu1225 for helping on this issue over night.


output of "state()" does not contain the "o" flag when the "f" command is waiting for input

Describe the bug
The output of state() does not contain the o flag when the f command is waiting for input.
To Reproduce
Run this shell command:
vim -Nu NONE -S <(cat <<'EOF'
    au VimEnter * call Func()
    fu Func()
        call feedkeys('f', 't')
        call timer_start(100, {-> writefile([state()], '/tmp/.log', 'a')}, #{repeat: 10})
        call timer_start(1000, {-> feedkeys("\e")})
    endfu
EOF
)

Wait one second, then run this Ex command:
:!cat /tmp/.log

The output is:
Sc
Sc
Sc
Sc
Sc
Sc
Sc
Sc
Sc
c

The o flag is absent from all the lines.
Expected behavior
The o flag is present in at least some of the lines.
Environment

Vim version: 8.2 Included patches: 1-1042
OS: Ubuntu 16.04.6 LTS
Terminal: XTerm(356)

Additional context
This seems to contradict :h state() which explains that the output of state() should contain the o flag when Vim is in operator-pending mode or when it's waiting for the argument of a command such as f:

o   operator pending or waiting for a command argument,
    e.g. after |f|

The documentation is wrong, "o" is only added when an operator is pending.  Not when waiting for a count or an argument for "f".

Updated docs:
These characters indicate the state, generally indicating that
something is busy:
m	halfway a mapping, :normal command, feedkeys() or
stuffed command
o	operator pending, e.g. after |d|
a	Insert mode autocomplete active
x	executing an autocommand
w	blocked on waiting, e.g. ch_evalexpr(), ch_read() and
ch_readraw() when reading json
S	not triggering SafeState or SafeStateAgain, e.g. after
|f| or a count
c	callback invoked, including timer (repeats for
recursiveness up to "ccc")
s	screen has scrolled for messages


Test RB analysis and generate test data

Summary
Added two files:

rb_generate_data.py - generate data and analysis for the test.
test_rb_analysis.py - check if the analysis return the same output

Details and comments
rb_generate_data.py:
For the generate data I have made the code support future data generator for RB experiment. Both the experiment data (meta data and count) in additional for analysis are saved to two separate json.
test_rb_analysis.py:
the test check that the metadata of the experiment matches the one given and it checks that the analysis will return the same results for identical distribution of "counts"

See: https://github.com/Qiskit/qiskit-experiments/issues/123

@ItamarGoldman - note that you need to update your code following #123

close #123


support refine by multiple sources using ES filter

1, Support refine by multiple sources using ES filter feature
2, Keeps backward compatible when only one source is selected
3, Keeps backward compatible when "All" is selected
4, Remove the old mustunittemplate
5, For multiple sources, need integration testing with frontend; expected sources are a list of strings separated by comma.Locally tested by hardcode

Coverage remained the same at 0.0% when pulling 62ba1f246a5195d49f4144a4f5fef0da0bc2bd98 on camelliazhang:linkedin2 into 397abb8c89c0ec56eddb85df9ece6aa4575a27f7 on linkedin:linkedin.


Finish app design

Create intuitive design, select colors for light and dark theme acc to link

private val DarkColorScheme = darkColorScheme(
primary = Purple80,
secondary = PurpleGrey80,
tertiary = Pink80
)
private val LightColorScheme = lightColorScheme(
primary = Purple40,
secondary = PurpleGrey40,
tertiary = Pink40
/* Other default colors to override
background = Color(0xFFFFFBFE),
surface = Color(0xFFFFFBFE),
onPrimary = Color.White,
onSecondary = Color.White,
onTertiary = Color.White,
onBackground = Color(0xFF1C1B1F),
onSurface = Color(0xFF1C1B1F),
*/

)
/* */
val Purple80 = Color(0xFFD0BCFF)
val PurpleGrey80 = Color(0xFFCCC2DC)
val Pink80 = Color(0xFFEFB8C8)
val Purple40 = Color(0xFF6650a4)
val PurpleGrey40 = Color(0xFF625b71)
val Pink40 = Color(0xFF7D5260)


Latest release fails to start

Describe the bug
Latest release with the new secrets stuff fails to start up
Log entries
This is what I get on my actual deployed instance:
2020-11-06T07:27:26-08:00 [Main] ****************************************

2020-11-06T07:27:26-08:00 [Main] Starting up version 5.5.0

2020-11-06T07:27:26-08:00 [Main] Timezone offset is 480

2020-11-06T07:27:26-08:00 [Main] Current time is Fri Nov 06 2020 07:27:26 GMT-0800 (Pacific Standard Time)

2020-11-06T07:27:26-08:00 [Settings] Unable to read the settings file: ENOENT: no such file or directory, open '/run/secrets/settings'.

2020-11-06T07:27:26-08:00 [Main] [Settings] Unable to load file /run/secrets/settings.

And this is what I get in the main branch trying to debug this:
2020-11-06T07:33:51-08:00 [Main] ****************************************
src/Log.ts:36
2020-11-06T07:33:51-08:00 [Main] Starting up version 5.5.0
src/Log.ts:36
2020-11-06T07:33:51-08:00 [Main] Timezone offset is 480
src/Log.ts:36
2020-11-06T07:33:51-08:00 [Main] Current time is Fri Nov 06 2020 07:33:51 GMT-0800 (Pacific Standard Time)
src/Log.ts:36
2020-11-06T07:33:51-08:00 [Settings] Unable to read the secrets file: ENOENT: no such file or directory, open '/run/secrets/secrets'.
src/Log.ts:45
2020-11-06T07:33:51-08:00 [Settings] Loaded settings from /run/secrets/settings
src/Log.ts:36
2020-11-06T07:33:51-08:00 [MQTT] Connected to MQTT server mqtt://mqtt:1883
src/Log.ts:36
2020-11-06T07:33:51-08:00 [Local storage] Creating local storage folders in /node-deepstackai-trigger.
src/Log.ts:36
2020-11-06T07:33:51-08:00 [Local storage] Enabling background purge every 60 minutes for files older than 30 minutes.
src/Log.ts:36
2020-11-06T07:33:51-08:00 [Local storage] Running purge
src/Log.ts:36
2020-11-06T07:33:51-08:00 [Main] Web server enabled.
src/Log.ts:36
2020-11-06T07:33:51-08:00 [Triggers] Unable to read the secrets file: ENOENT: no such file or directory, open '/run/secrets/secrets'.
src/Log.ts:45

@TonyBrobston I'm digging into this but if you have a few minutes and can look at it too it would be appreciated. Looks like we missed something during the code reviews and a straight upgrade to the new release breaks existing installs.

This looks like it's two separate problems. The one I'm seeing in local development is because replaceSecrets doesn't check for a null secrets before attempting to do a replacement.
Adding this to the function resolves that problem in the local dev environment:
  // If no secrets were provided don't attempt to do a replacement
  if (!secrets) return;

That gets me past the issue to the same one that I see in my deployed instance. Looking into that now.

Ok I think I figured this out. This happens when:

You don't have a secrets file
You use existing mustache templates in the settings or triggers file

The above code really needs to be:
  // If no secrets were provided don't attempt to do a replacement
  if (!secrets) settings;

Ok this still isn't fully fixed. Looks like there is an issue where if it fails to load from /run/secrets/settings it never attempts to load from /config/settings.json.

Looks like the remaining issue is in readSettings() which was throwing an exception instead of just logging and returning null, which prevented the .some() from proceeding to try and load from the alternate locations.

Confirmed this now works on my real deployment. Just waiting for the latest build to pass with updated unit tests then I'll release this.

@danecreekphotography I'm just now seeing this thread.
Sounds good.

Setting this up for the first time.
Configured my docker-compose file as follows, and I am seeing the error reported for this issue i.e. "[Settings] Unable to read the secrets file: ENOENT: no such file or directory, open '/run/secrets/secrets'."
`  deepstack-ai:
container_name: deepstack-ai
image: deepquestai/deepstack:latest
restart: always
volumes:
- /mnt/storage/deepstack:/datastore
environment:
- VISION-DETECTION=True
trigger:
container_name: trigger
volumes:
- //<IP_ADDRESS>//aiinput:/aiinput
environment:
- PUID=$PUID
- PGID=$PGID
- TZ=$TZ
ports:
- "4242:4242"
secrets:
- triggers
- settings
image: danecreekphotography/node-deepstackai-trigger:latest
restart: always
depends_on:
- deepstack-ai`


Items hierarchy support

Items hierarchy support:
[ ] Parent
  [ ] Child 1
  [ ] Child 2

If all children are marked as done, parent also should be marked as done.
If parent is marked as done, children also should be marked as done.

Need to add a proper description for this feature.


time: add Time.Compare

There are currently 3 methods to compare time.Time objects: Before(), Equal() and After(). These are analogous to <, == and >. But the equivalents of <= and >= are missing. I suggest adding BeforeEqual() and AfterEqual() to complete this.
I am fully aware that these new methods are not strictly needed: t.BeforeEqual(u) could be written as either t.Before(u) || t.Equal(u) or !t.After(u) or t.Sub(u) <= 0. However, those three are less readable than a simple and obvious t.BeforeEqual(u). I have written code like
for day := begin; /* day <= end */ !day.After(end); day = day.Add(24 * time.Hour) {

where I felt that the comment was needed to clarify the intent.
Implementation of the proposed methods is trivial.

At least to me, !t.After( ) looks significantly more readable than BeforeEqual. "not after t" is a very natural way to express the concept of "at the moment t or before".

At least to me, !t.After( ) looks significantly more readable than BeforeEqual.

Really? If you wanted to write a for loop with x going from a to b, including both endpoints, would you write it like
for x := a; x <= b; x++ {

or
for x := a; !(x>b); x++ {

My guess would be that most people would write it the first way. And I don't really see why it should suddenly be different if x is a time.Time instead of a number.

If I wanted to write a for loop with x going from a to b in most languages, I would write it as a range.
In most languages, the range would be over the interval [a, b+1), because most languages use half-open intervals for ranges.
(To my dismay, Go does not yet support ranges for user-defined types.)
If I wanted to write a for loop over a sequence of days in Go, I would not use the 3-part loop form. (For me, the 3-part loop form is almost always less clear than some alternative.) Instead, I would write:
	x := a
	for !x.After(b) {
		…
		x = x.AddDate(0, 0, 1)
	}

I think that form is clearer because it emphasizes the semantics: x is increased until it is after b, regardless of whether b happens to be an integer number of days away from a. (I think the use of time.Date to represent entire days — rather than nanoseconds — is awkward even at best; compare #19700.)

My guess would be that most people would write it the first way. And I don't really see why it should suddenly be different if x is a time.Time instead of a number. At least this was the rationale for my proposal.

In English, anyway, dates are usually compared using "before" and "after" while numbers are compared using "greater" or "less." It's why < is the less-than operator, not the before-than operator.

I think that form is clearer because it emphasizes the semantics: x is increased until it is after b

Right, but x.BeforeEqual(b) also emphasizes semantics the same way: the loop continues as long as x is before or equal to b. I just find the negation in the condition !x.After(b) slightly harder to understand: loops are often more naturally expressed as a "while" condition instead of an "until" condition.

In English, anyway, dates are usually compared using "before" and "after" while numbers are usually compared using "greater" or "less." It's why < is the less-than operator, not the before-than operator.

Right, but that's just a matter of wording: "before" has the same meaning as "less". In languages with operator overloading, you would likely use < for both.
My point is just that it's often natural to write X <= Y for numbers (I checked the Go sources and there are many examples of for loops with a <= condition). And it should be equally natural to want to write "timestamp X is before or equal to timestamp Y".

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

I like this proposal. Getting ordering sense right is often tricky, and the lack of these methods means that I've often needed to write out an expression in terms of how I'm thinking about it, and then change the method and add the negation. It might just be me, but I find that extra negations make it harder to think about correctness, particularly when is trying to concentrate on the details of an algorithm.
However, another possibility might be to add a Compare method:
// Compare returns -1 if t1 is before t2, 0 if t1 equals t2 or 1 if t1 is after t2.
func (t1 Time) Compare(t2 Time)

This is not only fairly easy to read (x.BeforeEqual(y) becomes x.Compare(y) <= 0
but could have additional utility with the advent of generics (see https://github.com/golang/go/issues/50340#issuecomment-1023524918 for an example), and save a comparison when using a time as one component of a larger comparison.

Considering that time.Time values have nanosecond precision, I'm curious how often the difference between time.Before and time.BeforeEqual matters in practice.

@ianlancetaylor It can matter surprisingly often in my experience - for example, a time might well be exactly equal before you've slept at all and the value is what you start with, or when you're not using time.Now at all but taking items from a heap or using a faked-out time source.

However, another possibility might be to add a Compare method:

Sure, if that's the direction that Go is moving to. It's very explicit and it would also solve my use case.

Considering that time.Time values have nanosecond precision

As @rogpeppe mentioned, not all time.Time values are "random". They might be timestamps entered in some user interface or they might be timestamps coming from a lower-precision source.

I wasn't very enthusiastic about adding BeforeEqual and AfterEqual but adding Compare seems reasonable and fits in with the other places in the library that define a Compare. Retitling.
Does anyone object to adding Time.Compare?

Given that it's so simple to do, I just added https://go-review.googlesource.com/c/go/+/382734 to implement this.

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

Hi, @rogpeppe
I'm interested in this proposal and I noticed that this CL stuck for a while.
Thanks for your great work and hope you keep pushing it.

@hopehook thanks for the prompt. I'd forgotten about the CL entirely! I've made the appropriate changes now.

Wouldn't it be better to add constants like time.Past for -1, time.Future for +1 and time.Same for 0? I believe they are much better than having hardcore values.

@Delta456 We didn't do that for bytes.Compare or strings.Compare.  I don't think we have to do it here.


Add promise interface.

Why do we need promise interface? Why can't you use a promise wrapper?

@NodeGuy I appreciate the effort, but I'm not interested in maintaining the promises API, I suggest you publish a separate module on NPM with your preferred API.

@stevemao There isn't a common promise wrapper for this situation.
I created a separate module.


Update shapeless to 2.3.5

Updates com.chuusai:shapeless from 2.3.3 to 2.3.5.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (2.3.3).
You might want to review and update them manually.
project/Dependencies.scala

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.chuusai", artifactId = "shapeless" } ]

labels: library-update, semver-patch, old-version-remains

Superseded by #138.


BUG: Multiple Connect Buttons spawn different modals?

Hi! I'd like to share some unexpected behaviour we got while integrating web3connect to our dapp using Web3Connect.Button component. We were planning to have two buttons: one on welcome page and one in the header. But after testing we had a bug that sometimes after clicking the provider inside the modal, the onConnect would fire, but modal would be still shown. After looking at the DOM, we had roughly 10 divs with WEB3_CONNECT_MODAL_ID id. It looks like clicking a button opened two modals at the same time.
I think the reason for this that every Web3Connect.Button initializes its own core inside the constructor:
https://github.com/web3connect/web3connect/blob/master/src/components/ConnectButton.tsx#L10
We solved the bug by initializing the core outside the component and using our own button component:
// @flow
import * as React from 'react'
import Web3Connect from 'web3connect'
import Button from '~/components/layout/Button'
import Web3Integration from '~/logic/wallets/web3Integration'

const web3Connect = new Web3Connect.Core({
  providerOptions: {
    portis: {
      id: 'PORTIS_ID', // required
      network: 'mainnet', // optional
    },
    fortmatic: {
      key: 'FORTMATIC_KEY', // required
      network: 'mainnet', // optional
    },
  },
})

web3Connect.on('connect', (provider: any) => {
  if (provider) {
    Web3Integration.setWeb3(provider)
  }
})

const ConnectButton = () => (
  <Button
    color="primary"
    variant="contained"
    minWidth={140}
    onClick={() => {
      web3Connect.toggleModal()
    }}
  >
    Connect
  </Button>
)

export default ConnectButton

I don't know if this is the expected behaviour, but I think it should be fixed/mentioned in the docs

ConnectButton is being deprecated


[Event Request] Codeunit 353 Item Availability Forms Mgt – ShowItemAvailFromPurchLine

Dear ALAppExtensions Team,
we would like to request again the following code extensions to be added in codeunit 353 Item Availability Forms Mgt – ShowItemAvailFromPurchLine.
https://github.com/microsoft/ALAppExtensions/issues/4643
Event has been shipped but without Handled Parameter.
Code Change as followed
WITH PurchLine DO BEGIN
  TESTFIELD(Type,Type::Item);
  TESTFIELD("No.");
  Item.RESET;
  Item.GET("No.");
  FilterItem(Item,"Location Code","Variant Code","Expected Receipt Date");

// >> New Code
  //OnBeforeShowItemAvailFromPurchLine(Item,PurchLine);
  OnBeforeShowItemAvailFromPurchLine(Item,PurchLine,IsHandled);
  IF IsHandled THEN
    EXIT;
// >> New Code

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


[Files] Rename create_directory to create

Per Johan, OK to leave this named create_directory() for GA. Closing the issue.


Generate a person model

✍ Description

A person represents a flesh-and-blood human in the system, and should be
the core data model representing most actors in the system.
Eventually, we will want to attach the filled form results to a person
record, and Accounts should belong to a person to grant the ability to
log into the system.
Note that this table explicitly does not include any authentication
related material.  tokens generated for things like magic login, or
authorization credentials for regular users, should exist in their own
dedicated classes.  I'll follow up with an email+token based auth flow
implementation later.

@all-contributors
Please add @cflipse for code.

@all-contributors
Please add @cflipse for code.


Add a new persona: My Persona

Provide a description of your role and responsibilities and what your observability goals are

Hi @Lall-091!
Not sure if I understand. How do you envision this persona in the docs?


error reporting

We run the ion_blacklist script once a minute via cron, but it doesn't report if it fails. Same with the ion_bot script.
There should be some mechanism for reporting breakdowns to the developers.

we should talk about what that mechanism is. if it fails repeatedly and we try to send the message somewhere, we could get a lot of spam before it gets fixed. it's easy enough to send errors out to the network somewhere, be it /r/imagesofnetwork, discord, or something else.
once we decide how we want to manage this, i can get something set up.
relevant: maybe #59 could have a facility to check for crashes or something. not really sure yet. looking for input.

Let's double down on Supervisord and Discord.

Discord Links to documentation, development servers, and libraries can be found on the Discort API server.
Supervisord documention and plugin examples
Chron StackOverflow on Implementing chron scheduling in supervisor


List of pending adoptions

Include on the landing page a link to the regularly updated list of adoptions and pending adoptions.
To begin with:

DMP Online by Digital Curation Centre (DCC) in the UK
DMP Tool by California Digital Library (CDL) in the US
DMP OPIDoR by Centre national de la recherche scientifique (CNRS) in France
RDMO by Leibniz-Institut für Astrophysik Potsdam in Germany
Data Stewardship Wizzard by Elixir research infrastructure in the EU
Argos - OpenDMP by OpenAIRE and EUDAT research infrastructures in the EU
F1000Research open research publisher in the UK
Norwegian Open Research Data Infrastructure in Norway
Haplo repository in the UK
TU Wien, TU Graz, Uni Wien via FAIR Data Austria project

Stockholm University SU-maDMP project (using DMP Online API v0 for output source)


Unsafe usage of PyYAML yaml.load()

The default behavior of PyYAML's yaml.load() method is unsafe and can execute code in yaml files.
yaml.safe_load() should be used instead.

fix


Create a subpage with basic info about Packit usage

Part of https://github.com/packit/dashboard/issues/192

[ ] Create a new subpage to show a basic about Packit usage/adoption
[ ] Use data from the newly created API endpoint implemented in packit/packit-service#1724


maybe improve types documentation a bit

Deployed here: https://kit-svelte-dev-git-better-docs-svelte.vercel.app/docs/types-presentation
Some of the documentation around types is a bit ropey. For example I don't love how this looks:

Also, the configuration documentation is kind of all over the place — this stuff really belongs in types/index.d.ts, so that it appears inline when you're actually writing the config, rather than being manually written in a separate documentation file where it's less useful (and manually maintained as we make changes).
In this PR I'm trying some ideas for how we might improve it a bit. Am open to all suggestions. Some of this might need custom CSS to make it look halfway decent.

Just learning TypeDoc while learning TS.
My main motivation to learn TS is in fact having docs for free*. This combo is making me fall in love with TS. I heard JsDocs was gaining some traction in the svelte community as the first step to TS without having to get married to it.
Should svelte/kit have both? It seems JsDoc and TsDoc is mostly interoperable (don't have to add the type in the params in TsDoc as one diference), don't know enough to comment on it, but I've seen some TypeDoc magic that I can't wait to see around here. I might be talking nonsense and all of this is already implemented or something else, I just wanted to bump this, I'd loved to see better documentation not only for final users but also for devs of svelte/kit.

During the work on #7087 I felt hindered by having to write more detailed docs inside JSDoc. You can't use JSDoc within JSDoc for example (/** @type {..} */ will close the outer JSDoc). Moreover, it could get overwhelming seing an ocean of docs when hovering over a function. In also complicates future enhancements like translation or JS+TS in the docs with a switch.
I therefore think we need a different approach - some kind of mix between manual docs and "include X from the types". Made-up example-MD file that would need to be preprocessed:
## cookies

### set

--definition-- // <- this would extract the type definition from types.d.ts into here
--summary-- // <- this would extract the JSDoc from types.d.ts into here in case you don't want to duplicate things

Lorem ipsum stuff

Reverting part of my previous comment - the API docs should be very concise and more on the short side. The cookie example is the biggest it should probably get. If there's more it should go into a different part of the docs, explaining things in more detail and in the context of usage. What remains is the question about code samples, which I don't have a good answer to.

I pushed a POC of a generator enhancement that transforms interfaces into better looking docs similar to what you proposed. I added a border to each of the snippets that present sub headline to better distinguish them from regular code snippets. I also kept a trimmed down version of the whole interface at the start of each section so people can get a quick overview of that type.
I think the result is much better-looking, but only if we have actually some docs in there - so I think if we change this, we should also get down to writing a lot more quick API docs here at the same time.
The final part of the picture (for me at least) would be to restructure the types and modules sections into one "API Reference" section, and we group the modules / interfaces logically into sub pages. Something like this (picture incoming)

Pushed up some design tweaks that make type blocks visually distinct from code blocks, and make things more grouped visually.
This is definitely an improvement over the status quo, but there's a bunch of issues (inherited or otherwise) that we should try and address if we can:

[ ] code block references value imports, not types

[ ] this still looks overwhelming, not sure if there's a better way to present it

[ ] unrelated to this work, but super confusing wording

[ ] properties without descriptions look weird, especially at the end

We need to add loads more docs

[ ] do I as a user need to see this? seems complicated and irrelevant

[ ] repetition of cancel() looks kinda awkward in situations like this

What if the top was just interface BeforeNavigate extends Navigation {...} or something?

[ ] this is duplicative

Everything in this block is described immediately afterward — it's not clear it serves any real value

[ ] some types don't create links, like ValidatedConfig here

Also, a sequence of properties without descriptions looks super weird

[ ] indentation is wacky. also, what does "dest null" mean?

[ ] bullet points and text are way out of alignment

[ ] these should all link back to the relevant docs sections

[ ] why do some things get property descriptions and others don't?

Also, the configuration docs should be driven from the types. we need to be able to show default values somehow

[ ] references to generics show up as 'any'

[ ] wonky margins around code blocks

[ ] double line breaks are unnecessary

(assuming we don't get rid of the contents of these interfaces altogether)

[ ] nested properties don't look great

This is a very thorough analysis! I agree with most of that. I think the two biggest things we can do are

doc this shit up: everything should have some jsdoc so it doesn't look that empty
rethink modules and types sections: I think we should merge them to an API section. If we think it gets too big that way, we could split each into a separate page, which would also help with "what ends/starts where?"

Another issue — probably shouldn't block merging this PR, but worth opening as follow-up — type variables are ignored when converting JSDoc to TypeScript:

Don't know if you already saw this, so I'm posting it again: IMO it makes sense to rethink modules and types sections in general: I think we should merge them to an API section. Right now things are split across these two sections (for example $app/forms appears in both), which is weird. If we think it gets too big that way, we could split each into a separate page, which would also help with "what ends/starts where?". I quite like Angular's approach here, with an overview page from which you can jump to the details of what you need https://angular.io/api

Yeah, I had that in the back of my mind as I was fixing up the smaller things. It's definitely an option, though for the SubmitFunction in particular I found it made more sense just to move it into @sveltejs/kit (notwithstanding the fact that it makes this a breaking change).
It is a big page. I definitely wouldn't rule out us subdividing it further later like Angular does. But I think it's worth avoiding if we can — Angular's API is just monstrously large, and if SvelteKit ends up as unwieldy as that then I plan to retire.
Until then I think the modules/types distinction makes sense, given that all the types are exported from @sveltejs/kit or ./$types. I think combining them at this stage would reduce clarity

There's definitely room to add more descriptions, but that doesn't need to hold up this PR any further. I think the big thing remaining is to sort out the configuration docs:

[ ] they're still written as though we can't document nested properties — needs tweaking
[ ] we have both https://kit-svelte-8bwewn5rw-svelte.vercel.app/docs/configuration and https://kit-svelte-8bwewn5rw-svelte.vercel.app/docs/types#sveltejs-kit-kitconfig. There Can Be Only One. I'm not sure how best to shuffle things around though
[ ] defaults aren't present for all values yet. (Also, it might be nice if we had some iconography for params/defaults/returns or at least a slightly improved design, rather than bullet points)
[ ] if we retire /docs/configuration then it would be good to generate hash links for properties, at least at the top level

I vote for keeping the configuration section. It can be autogenerated, but I think it's better to have it separate, since it's rather big and semantically doesn't fit into types/modules for me.

Another thing to fix in a follow-up PR

The create-svelte tests are failing — something to do with moving SubmitFunction form $app/forms to @sveltejs/kit I assume. @dummdidumm any idea why this is happening?

No idea why this fails ... I need to investigate more thoroughly tomorrow. So far it looks like this only fails for .svelte files, so this might be some bug inside language tools.

No idea why this fails ... I need to investigate more thoroughly tomorrow. So far it looks like this only fails for .svelte files, so this might be some bug inside language tools.

I lied, this also happens inside TS files. Still no idea why, it seems related to the specific interface, because if I for example use the BeforeNavigate interface it works.

I was trying to configure the trailingSlash option, as introduced in #733, but found it's missing in the current config and found this to be the PR where it got removed. I couldn't really figure out why, so was wondering if it was intentional? If so, is the feature no longer supported?

Never mind, saw that it's now part of the page options.


Adressing Issue #46443 Users blindly copying example

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Closes #46443

Hey @mallsjr
This PR seems to make similar changes as the existing PR <#46472>. As such, we are going to close this as duplicate.
If you feel you have additional changes to expand upon this PR, please feel free to push your commits and request this PR be reopened.
Thanks again! 😊

If you have any questions, feel free to ask questions on the 'Contributors' category on our forum or the contributors chat room.


runtime/cgo: the handle pointer is freed during C asynchronous callback

What version of Go are you using (go version)?

$ go version
go version go1.19 linux/amd64

Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1056322479=/tmp/go-build -gno-record-gcc-switches"

What did you do?

https://go.dev/play/p/k68Hr3YCWdP
when the C asynchronous callback, the runtime/cgo handle pointer has been freed by the gc, causing the wrong memory to be used during the callback
What did you expect to see?
keep the handle pointer until handle.Delete() is called
What did you see instead?
handle pointer has been freed by the gc

I think it's just a wrong usage.
handle could be GC freed, after C.callback_wrapper returning, it's by design.
according to your code:
func DoAsync() {
	handle := cgo.NewHandle("test")
	pointer := unsafe.Pointer(&handle)
	fmt.Printf("handle:%d,pointer: %v\n", handle, pointer)
	C.callback_wrapper(pointer)
}

@doujiang24 I temporarily use sync.map to save the pointer, I'm not sure if cgo handle should provide this feature.This example is just to show the problem I'm having

I believe this violates cgo pointer rules https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers
specifically:

C code may not keep a copy of a Go pointer after the call returns.


[webapp] ssh skip ssl check

customer was using a self-signed cert, so ssl check fails
add option to skip ssl check

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

[x] I adhere to the Command Guidelines.

@patricklee2 I believe for CLI to skip self-signed cert check the recommended way is to use the env variable AZURE_CLI_DISABLE_CONNECTION_VERIFICATION=1
if the API's are from the management libraries Microsoft.web, CLI handles this by default, but if not your code should handle this by reading the environment variable  value and pass in a verify argument.
we recently this with zip deployment - please take a look & verify if this fixes the issue here instead having to pass skip check as a parameter.

closes in favour of https://github.com/Azure/azure-cli/pull/9206


xworker hooks cannot be used

Users may need to be able to hook into XWorker runtime.
I've tried to pass in functions to onAfterRun/onAfterRunAsync and got an error.
This may repro: https://bugzpodder.pyscriptapps.com/polyscript-worker-errors-copy/latest/
However the domain is blocked by my computer so I don't know exactly.
Not sure if its webpack specific or not but it seems that the error is just serializing a function to postMessage which doesnt work, makes sense.

I've tried to pass in functions to onAfterRun/onAfterRunAsync and got an error.

As the TypeScript hints, anything Worker related has to be a string. You can't pass functions because you can't pass the main scope down to workers ... this is a Web limitation, not a polyscript one, I am afraid.

P.S. please feel free to check PyScript next branch, pyscript.core folder, to see how we use worker hooks to bootstrap the module within Workers.


[nomnigraph] Expose transformations to python

Adding a python interface to the transformations

Thank you for your contribution! PyTorch and Caffe2 are now officially merged with the PyTorch 1.0 prerelease. Please submit on https://github.com/pytorch/pytorch if this is still an issue over there.


Use the correct name of the UI project.

Current coverage is 37.46%

Merging #37 into master will increase coverage by +0.06% as of e65cccb

@@            master     #37   diff @@
======================================
  Files          205     205
  Stmts        11056   11056
  Branches      1939    1939
  Methods          0       0
======================================
+ Hit           4136    4142     +6
  Partial        537     537
+ Missed        6383    6377     -6

Review entire Coverage Diff as of e65cccb

Powered by Codecov. Updated on successful CI builds.


As a user, I want an example of TypeScript compilation using bun

In a ./examples/app-typescript directory

This will start off of as a basic file server.

The app will be a simple index.html with a div id="app" element and a script element with src="index.ts". That index.ts script will set the inner HTML of the "id=app" element to "Hello, world!".
The examples will be served using serverz and have an index.html landing page with a list of all examples (like three.js does).


Add mock impl for Adobe Granite Asset API

Even though it's not the recommended API for assets, the Adobe Granite Asset API has some useful methods compared to the Day CQ Asset API (like removeAsset).
We use it in some projects (especially the removeAsset functionality).
This PR adds mocks for this.
I opted to use wrappers for the existing mocks (MockAsset & MockAssetManager) instead of re-implementing the same or similar code. MockRendition already had the required methods so that was solved by just adding the other interface.

thanks for the contribution! i applied some cosmetic changes - format code following our standes, add license headers, eliminate some code warnings.
sonarcloud analysis fails due to lack of coverage - can you check https://github.com/wcm-io/io.wcm.testing.aem-mock/pull/49/checks?check_run_id=28941901554 and add a few more coverage methods? thanks!

Sure, I'll have a look. Thanks for the fast reply.

thanks to @henrykuijpers  for the in-depth review! a few cosmetic points are already fixed by my reformatting, most are still relevant.

I tried to tests for all logic but it's missing some coverage on all the "unsupported oepration" lines ofcourse. My IDE says 78% total coverage, curious to see how much coverage on new lines now...

I tried to add tests for all logic but it's missing some coverage on all the "unsupported oepration" lines ofcourse. My IDE says 78% total coverage, curious to see how much coverage on new lines now...

it still reports 77%. we do not have to test unsupported operations, and also not error handling for exotic edge cases. but i will add a comment or two where i think a coverage is missing or i have an open question.


Segmentation Component - Issues when switching from custom to default segment

Steps to reproduce

Add segmentation component
Add 2 segments. Both custom.
Submit the dialog. As expected, we see 2 tabs corresponding to 2 custom segments.
Re-edit the component and switch the 2nd custom segment to Default.
As expected, the custom fields are hidden.
Submit the dialog. Notice that tab title does not show default. Instead, it still custom condition.
Re-edit the dialog, switch that default segment back to Custom. Update the segment name and condition.
Submit the dialog and notice that component output and the view as published output. In our test, it did not save the new segment information.

Please let me know if you would like me to demo this.
Thank you.

@adobe export issue to Jira project SITES as Bug


Updates #499 - Frontend for displaying star comments in user profile page

Included:
Stories for ProfileSubmissions and ProfileStarComments
added tests to [username].test.js
adds dummyStarData.js to dummy files
backend files from other PR

Many rules from profileStarComments.module.scss could be changed to bootsrap class names.
 display: flex;->d-flex
align-items: center;->align-items-center
justify-content: center;->justify-content-center
Also padding and margins.


User Profile

"You're not currently signed up for a challenge" should not show if user is ineligible for a challenge.

https://github.com/osumb/challenges/pull/40


[Feature Request] Use better default interface font (for Windows 10 in Chinese locale) or let users change it

Description

RenderDoc's default interface font (宋体, simsun.ttc) for Windows 10 in Chinese locale is really ugly and unreadable.
Setting Global Font Scale help a little bit but the main issue is the font.

(RenderDoc using 宋体)
Although ReaderDoc is in English, it will still use the English part of 宋体. The actual default interface font in Windows 10 is 微软雅黑 (Microsoft YaHei UI), which is a okay sans-serif font for English. I think this should also be the default in RenderDoc.

(Windows 10's File Explorer using 微软雅黑)

This is a common issue in Qt apps. Musescore and Krita solve this by letting users to change interface font. (they use 微软雅黑 as default in recent versions)

(Interface font selection UI in Krita and Musescore, also comparsion between 宋体, 微软雅黑 and Roboto)
I would like RenderDoc to use similar solutions, or at least use 微软雅黑 as the default.
Environment

RenderDoc version: v1.17
Operating System: Windows 10 1703
Graphics API: OpenGL, Vulkan, DX12

Thanks for reporting this, it's certainly a blind-spot of mine so I welcome any feedback like this.
I'm a little uncomfortable changing the default font based only on one report, especially without being familiar with the options and renderdoc not really being localised in any way (aside from the code being somewhat prepared for translation). Being able to change the default font though is an entirely reasonable request and so that commit should let you customise it to any on your system that Qt reports.
Please let me know if this works for you and addresses the issue. I'm certainly willing to make further changes to improve the experience with RenderDoc on other languages and locales, but I want to be sure I'm doing the right thing since it's not something I can verify myself.

Please let me know if this works for you and addresses the issue. I'm certainly willing to make further changes to improve the experience with RenderDoc on other languages and locales, but I want to be sure I'm doing the right thing since it's not something I can verify myself.

Thanks, but I don't have the build environment and am not willing to set it up, do I need to wait for nightly builds on https://renderdoc.org/builds#nightly or are there other places for binary releases?

I'm a little uncomfortable changing the default font based only on one report, especially without being familiar with the options and renderdoc not really being localised in any way (aside from the code being somewhat prepared for translation).

I agreed. For now allowing customization is enough.
When Chinese user base become large that localization is needed, it's probably the time for setting the default. (by the way, Chinese part of that font is even more horrible, sigh...)

Working as expected, thanks!

There are however a minor issue, and a (good) side effect I find during trying to solve it:
The font in the red rect looks like Courier New (which is not really bad so I'm fine with that), and I can't make RenderDoc choose the custom font for this (by trying all combinations of "prefer mono fonts" and choose custom propotional/mono fonts)

The side effect I find is that now I can choose mono font as the main interface font, and I see some people wanting mono fonts for interface in previous issues. I think ultimately the solution to this is maybe allowing two custom fonts, one for main interface (propotional is common, but mono can be choosed) and one for tables, logs, console and script editor, etc. (mono is common, but propotional can be choosed). This also get rid of the "prefer mono" boolean since we can just choose the same font.

Yes builds are created nightly (on western europe time) at that build. For reference there's no complex build environment needed, you only need a somewhat recent visual studio installed (2015 is the minimum) which if you're developing on windows you presumably already have.
I'm not sure I understand what the problem you're describing is. You say the default monospace font is fine as-is and that's what I'd have said as well, and being able to choose a monospace font for the default is also not a problem and I'd agree there.
The 'prefer monospaced' option is there specifically for elements which typically use the default font for things like resource and event names which are considered 'text' so use the default font that is likely proportional, but some people prefer that to be considered 'data' and use a monospaced font.

The 'prefer monospaced' option is there specifically for elements which typically use the default font for things like resource and event names which are considered 'text'

I guess I interpreted this option wrong, I thought that it's for forcing tables, consoles, script editor, etc. to use mono font.
What I'm suggesting is to allow seperate custom font for those elements too. Since main UI font is now customizable, it feels a bit weird that those don't mirror that. This is not totally unconventional in complex programs, for example Blender allows you to customize both. The current default is indeed okay and usable (not great though), so I'd say this is just a minor issue, and I'm fine if you decide not to implement this.

you only need a somewhat recent visual studio installed (2015 is the minimum)

Yeah... that's the part I hate, I use gcc and do stuff manually, but it's not worth going into since testing without building from source is viable.

I think that makes sense. I'm still not sure there's much need for customising the monospace font but it's straightforward to add and I can certainly appreciate that it makes for a more consistent customisation.
That commit adds the ability to change it, and also tries to clarify what the 'prefer monospace' toggle is for - as above it's more about switching some of the UI elements which typically use a default font for places like the event browser and pipeline state listings where mostly you will see resource names and a fixed width font has limited utility.


Update umbrella README, add rudimentary Windows section

Description

Update umbrella README.md for readability and structure. Add rudimentary differentiation between Linux, Windows and Mac section.

As per discussion with @stephanbcbauer

Pre-review checks
Please ensure to do as many of the following checks as possible, before asking for committer review:

[ ] DEPENDENCIES are up-to-date. Dash license tool. Committers can open IP issues for restricted libs.
[ ] Copyright and license header are present on all affected files

There seems to be an issue with the recent version bump to 0.11.4

✖ umbrella => (version: "0.11.4", path: "charts/umbrella") > chart version not ok. Needs a version bump!

As noted in the Matrix chat:
When running Minikube through Docker for Desktop on Windows, there is an issue with forwarding any requests through the minikube ip.
Alternatively I tried the native Kubernetes Cluster that is included in Docker for Desktop on Windows. Anyone trying to run Minikube on Windows through Docker for Desktop would have access to it as natively as well.
The steps described in the README also natively work with that Cluster. The only addition is installing the nginx-controller:
helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace

helm install cert-manager jetstack/cert-manager --namespace umbrella --create-namespace --version v1.14.4 --set installCRDs=true

cd charts/umbrella

kubectl apply -f cluster-issuer.yaml

helm install -f your-values.yaml umbrella . --namespace umbrella

The only other change is that Docker for Desktops Kubernetes Cluster uses <IP_ADDRESS> instead of an independent IP. With this configuration I was able to run the umbrella charts unter Windows.
I'm not sure if using the native Docker for Desktop Cluster an option for you but I thought I'd document it at least.


Pub.Dev - at_contacts_flutter

Lead: @Gayathri-GA
Inconsistency with overview section from other landing pages.  See screenshots below.

[x] Should we use at_contacts_flutter title (inconsistent across other packages too)
[x] No overview (see 2nd screen shot)

Ideal version from at_common_flutter

This is done and merged to trunk branch.
Needs to be published.

The publishing was held off as there were two issues for contacts and contacts_groups.

Adding a contact with image was failing
Creating a new contacts' group was failing

This has been fixed now and would be tested and published.

Published: at_contacts_flutter: ^4.0.1


Provide default configuration file showing all default keybindings

I can't find a list of default keybindings so I can't see the names of actions in order to rebind them. I strongly believe that the functionality of jk and JK should be swapped around and I thought this would be a quick change, but I can't seem to find anywhere what their actions are called. Other people seem to be figuring out rebinding things, so I must just be missing it somewhere.

Check out the Newsboat and Podboat Operations sections in the documentation. For a more compact view have a look at the raw DSV file. In particular, you're interested in:

next (Shift+J)
prev (Shift+K)
next-feed (J)
prev-feed (K)

Thank you very much! Hopefully the next person who gets stuck can find this issue now.

Another place where you can find the available operations is in the help dialog.
It can be opened in Newsboat itself with the ? key and shows the operations which are available/bound in the opened view.


feat: explicitly close messages channel on exit notification

With this change the client can be explicitly closed. This allows for
the receiver of the messages channel to observe the end of the stream
and terminate.

For some context see this change for how it simplifies our usage of lspower. https://github.com/influxdata/flux-lsp/pull/353/files#diff-48d27698cc708c7efe770fd897e4885efaa0a15cae30d54936068603ba03bbd9R352
TL;DR We use lspower to run a language server in the browser via WASM. As such we do not use the Server type but have an API exposed to Javascript that directly calls the Service.call method. As a result we need to manage the reading of messages to the client and therefore need to observe when the message channel is complete.

OK some github weirdness happened and I need to close this PR. I have opened a new one here #24. Our  fork was somehow off this repo https://github.com/attila-lin/lspower  which appears to be a direct copy of your repo? Were you aware of this? Also not sure how I managed to submit a PR to this repo in the first place but I guess it doesn't matter too much.


optimize skiplist

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

4 of 4 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.0004%) to 99.747%

Totals

Change from base Build<PHONE_NUMBER>:
0.0004%

Covered Lines:
1182

Relevant Lines:
1185

💛  - Coveralls


VSCode syntax highlighting not working on block comment after object key

Bicep version 0.3.255
In this example, /*comment*/ is not being colorized:

The code:
resource test 'Microsoft.AAD/domainServices@2021-03-01' = {
  name/*comment*/:'test'
}

This is fixed in Bicep v0.4.1008.


Major issue/PR housecleaning

[x] Draft template text for closing issues
[ ] Close all non-bug issues for v1
[ ] Analyse feature requests and decide whether to move them to v2
[x] Close all open v1 PRs against master
[x] Redirect open v1 PRs against mdl-1.x

Sample text for closing open PRs against mdl-1.1 or mdl-1.2:
Hello there! We're currently doing some housecleaning on the repository to prepare for the big changes to v2, and part of this is to currently move all PRs against 1.x to the mdl-1.x branch. Since it's not possible to target PRs at a new branch, we're closing this PR and asking you to reopen against mdl-1.x if you think it's still valid.
Thank you!

@sgomes making a v1-bug label for all of the issues filed against the v1 codebase that we're keeping open / carrying over.

Thanks, @traviskaufman!

Resolving as this has been open for a while and we've begun active work on v2.


Add optional parameter 'file_name_filter' to gradle task :blackbox:itest

Task itest of module blackbox can be invoked with -Pfile_name_filter=.
Only RST files whose name contains  are accounted for when running the
doctests. For example:
gradle :blackbox:itest -Pfile_name_filter=sharding

results in only the doctests within file 'blackbox/docs/general/ddl/sharding.rst'
being run.
Summary of the changes / Why this improves CrateDB
Checklist

[ ] User relevant changes are recorded in CHANGES.txt
[ ] Touched code is covered by tests
[ ] Documentation has been updated if necessary
[ ] CLA is signed
[ ] This does not contain breaking changes, or if it does:

It is released within a major release
It is recorded in CHANGES.txt
It was marked as deprecated in an earlier release if possible
You've thought about the consequences and other components are adapted
(E.g. AdminUI)

retest this please

retest this please


<EMAIL_ADDRESS>decompress error

Installing 'mc' (4.8.24) [64bit]
mcwin32-build215-setup.exe (3.7 MB) [=============================================================================================================] 100%
Checking hash of mcwin32-build215-setup.exe ... ok.
Extracting mcwin32-build215-setup.exe ... Invoke-ExternalCommand: C:\Users\D\Apps\Scoop\apps\scoop\current\lib\decompress.ps1:192
Line |
 192 |  … tus = Invoke-ExternalCommand (Get-HelperPath -Helper Innounp) $ArgLis …
     |                                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Cannot validate argument on parameter 'FilePath'. The argument is null or empty. Provide an argument that is not null or empty, and then
     | try the command again.

Failed to extract files from C:\Users\D\Apps\Scoop\apps\mc\4.8.24\mcwin32-build215-setup.exe.
Log file:
  ~\Apps\Scoop\apps\mc\4.8.24\innounp.log

Please try again or create a new issue by using the following link and paste your console output:
https://github.com/lukesampson/scoop-extras/issues/new?title=mc%404.8.24%3a+decompress+error

There's no file innounp.log at the specified path.
OS: Windows 10
Arch: x86-64
Anything else to specify?

scoop install innounp and reinstall


chart and legend colors don't match after clicking on label

Sorry if this is stupid, I'm new to this.
My issue is that after clicking on a label in legend to hide that entry from the chart the graph colors all update while the colors on legend don't.

after click:

I've also found that on https://codepen.io/chimmer/pen/aJvBgv same happens.
This is the css I use:
https://pastebin.com/xSF9EJCb
it's pure css, but that shouldn't matter since codepen has the same issue while using less, right?
I would want the chart to not update colors. Do I have to use onClick and manualy update the colors in my css?

That is pretty much just a CSS issue, you can't use the n-th child selector like I did in my pen you linked, but use selectors for the actual series' class names e.g. .ct-series-b. I simply copied your CSS and added line styles, you should be able to figure what's happening by that i hope: https://codepen.io/chimmer/pen/QVvMgz.


AppleNoteStore: NoteStore tried to rip Note50 but had to rescue error: can't modify frozen String

Hi there, I've stumbled on your tool by accident, it's fantastic except I'm not sure why plain text of every note doesn't get exported with mac application version 17 of Db.  It works and finishes the process with Successfully finished  although the log has errors while trying exporting every single note.

Describe the bug

Fails to export plaintext field using Mac backup group.com.apple.notes/NoteStore.sqlite
Every note rip returns an error: AppleNoteStore: NoteStore tried to rip Note50 but had to rescue error: can't modify frozen String:

From debug_log.txt
Rip Note: Ripping note from Note ID 50
D, [2024-11-07T21:07:00.058326 #93879] DEBUG -- : Rip Note: Looking up account for 289
D, [2024-11-07T21:07:00.058347 #93879] DEBUG -- : Rip Note: Looking up folder for 295
E, [2024-11-07T21:07:00.058992 #93879] ERROR -- : AppleNoteStore: NoteStore tried to rip Note50 but had to rescue error: can't modify frozen String: "\x1F\x8B\b\x00\x00\x00\x00\x00\x00\x03\xE3`\x10z\xC3\xC4\xC1 \xC0 \xF5\x94IH>)3\xBD$3/91/\xA34\xC9J_?\xBF<EMAIL_ADDRESS>\xC0\x06\x16\xE1\x02\xAAb\x85\x8A\xF1Cur\x02\xC5\xD8\xA0:\x05\x04\x98\xC1bL\x1A\xECP\x11\x11\xA8\b\xAF\x06\aT\x9F\xB8\x00\x13XD\x1C\xA8\x8F\x13\xAAJR\x80\x13,\xC6\xAF\xC1\x05\x15Q\x82\xEA\x93\xD0\xE0\x86\x8A\xA8B\xED\x93\xD4\xE0\x81\x9A\xA4\x06\x15\x91\x02\x9A\xC4\vU\xA5\x0E\xD5'\xAD\xC1'%\xC6\xC5\x01\xF4\xFD\x7F \xE0\a\x86\x04\x9C\xAD$\xC3%\xC5%p\xA6\xFB\xE2\x89\x10.\xFF\x90M\x93\"\xBD}\xCE\x9B\n\t1qh\x01\xB1\xB4\x16\e\x87\xBC\x10\x93\x04#\x00\xC7\xD1\x82\x0Fq\x01\x00\x00"
D, [2024-11-07T21:07:00.059030 #93879] DEBUG -- : Rip Note: Ripping note from Note ID 36
D, [2024-11-07T21:07:00.059115 #93879] DEBUG -- : Rip Note: Looking up account for 289
D, [2024-11-07T21:07:00.059134 #93879] DEBUG -- : Rip Note: Looking up folder for 295
E, [2024-11-07T21:07:00.060042 #93879] ERROR -- : AppleNoteStore: NoteStore tried to rip Note36 but had to rescue error: can't modify frozen String: "\x1F\x8B\b\x00\x00\x00\x00\x00\x00\x03\xE3`\x10\xBA\xCD\xC4\xC1 \xC0 u\x85Ih\"cxf^\xAA\x82[~NN%\x97\xA3BXfqib\x8E\x82{ifJ\xAABI\xBEBIF\xAABx~QN\x8AB~\x9A\x02H%\x97obJj\x0EHK@irvjII\xAA\x82\x9A\x82WiqIf\x9E\x82Gbnnbr6\x17WFIIA\xB1\x95\xBE~fIi^j\xB1^bAAN\xAA^r~\xAE~b\xA9~R~~\xB6~9\xD0\x00\xDD4\x90\x9D\xFA\x99)\x86\x06f\xC6&\x96\xC6\x16\x96f\xF6\xB9%\xB6\x86\x86R\x02\\, \xE7\x01\x1D\b\xA65\x18\xA5\x04\x81\"\x8C@\x91\x89\x8CR`\x86\x06\x93\x94\x18\x17\aP\xF2?\x10\xF0\x03\x15\xC2\xD9J\xB2\\\xD2\\\x02'\x12\xF6\n\x9B\xB3\xB9\x1D\xD8\xA1\xFDe\x82\xA0\xD2\xAC\xADB\xCC\x1C\x13\x19\x85\x988\x18\xB5\xB88\xB8\x85\x80FH0j\x80\xD8\xEEB,\x02\f`6H\xCE\x8D\xC3N\x88I\x82\xD1\xCB\x8E2\xF7\x03\x00\n\xFA\x98q`\x01\x00\x00"

Desktop (please complete the following information):

OS: Mac 14.6.1
ruby 3.3.6

Command used
ruby notes_cloud_ripper.rb -f ../NotesBackup/group.com.apple.notes/NoteStore.sqlite
Please confirm the following

Error occurs on the latest version of this program on GitHub v0.18
You have run bundle install Yes

For what it's worth, I'm experiencing this issue as well.
E, [2024-11-07T13:16:01.147963 #47789] ERROR -- : AppleNoteStore: NoteStore tried to rip Note1 but had to rescue error: can't modify frozen String
Mac 14.6.1 / Ruby 3.1.2
Command used:
ruby notes_cloud_ripper.rb -m ../group.com.apple.notes

Thanks for reporting this! I'll take a look as soon as I can. Appreciate your detailed report.

Sorry for the delay, was tied up all weekend. Could you try running rake test and see if the tests that don't require a full backup pass? Additionally, could you please provide the output of cat Gemfile.lock?
Appreciate the report!

I would note that #112 also had the issue of everything being marked as a frozen string, but I still have not been able to recreate this on my Mac using the same version of Ruby. I'm curious to know if your Gemfile.lock file shows something radically different and how you installed ruby (brew, rvm, ... etc?). I'm using rvm.
My guess is, if you can, moving to ruby-3.2 will solve the issue.

I'm struggling to reproduce this. I'm using three different systems: Mac OS 14.6.1, Alpine Linux (via Docker), and my normal development Linux machine. On each I'm trying Ruby 3.0 - 3.3, including 3.3.6.
I went so far as to take the protobuf you in-lined in your original report and grafted it into one of my test databases, as well as exported it as a separate binary and ran tests with RSpec. I can't get this error to occur.
I should've mentioned that I had an Intel based mac 2019 MBP and I've migrated to ARM mac mini and now it's working with
ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [arm64-darwin24]
All notes are exported perfectly.
Please let me know if I should close this issue or you'd like to look further?

No need to apologize, it is good to get away!

Please let me know if I should close this issue or you'd like to look further?

It is fascinating to know that something in the change from the intel MBP to the arm mac mini fixed this. I'm torn as I don't necessarily want to keep issues open that I can't make headway on, but this also is the third time the issue has cropped up. I'll probably leave it open for a bit and close it if I simply can't make any progress at recreating the issue.
Are you able to get the Gemfile.lock from the MBP, or is it gone at this point? I'd be keen to see the differences between one of yours that worked and one that didn't.
Appreciate the help!

It is fascinating to know that something in the change from the intel MBP to the arm mac mini fixed this

To me surprise both files Gemfile.lock are identical on both machines Intel and ARM. I put them to Beyond compare, not a single different line.

Though I have 2 new issues now, I will just leave them here but I don't expect you to respond in this thread as it could be something unrelated and I don't want to blow up this issue anymore. If you recommend I'll just create 2 new separate issues.

When calling the script with a Library path I get the error and wrong version of DB Guessed Notes Version: 8 on Mac

ruby notes_cloud_ripper.rb -m /Users/user/Library/Group\ Containers/group.com.apple.notes/

Starting Apple Notes Parser at Mon Nov 18 22:20:51 2024
Storing the results in ./output/2024_11_18-22_20_51

Created a new AppleBackup from Mac backup: /Users/user/Library/Group Containers/group.com.apple.notes/
Guessed Notes Version: 8 on Mac
/opt/homebrew/lib/ruby/gems/3.3.0/gems/sqlite3-1.7.3-arm64-darwin/lib/sqlite3/database.rb:177:in `initialize': no such table: ZACCOUNT (SQLite3::SQLException)
	from /opt/homebrew/lib/ruby/gems/3.3.0/gems/sqlite3-1.7.3-arm64-darwin/lib/sqlite3/database.rb:177:in `new'
	from /opt/homebrew/lib/ruby/gems/3.3.0/gems/sqlite3-1.7.3-arm64-darwin/lib/sqlite3/database.rb:177:in `prepare'
	from /opt/homebrew/lib/ruby/gems/3.3.0/gems/sqlite3-1.7.3-arm64-darwin/lib/sqlite3/database.rb:221:in `execute'
	from /Users/user/Documents/NotesBackups/apple_cloud_notes_parser/lib/AppleNoteStore.rb:393:in `rip_accounts'
	from /Users/user/Documents/NotesBackups/apple_cloud_notes_parser/lib/AppleNoteStore.rb:225:in `rip_all_objects'
	from /Users/user/Documents/NotesBackups/apple_cloud_notes_parser/lib/AppleBackup.rb:243:in `block in rip_notes'
	from /Users/user/Documents/NotesBackups/apple_cloud_notes_parser/lib/AppleBackup.rb:240:in `each'
	from /Users/user/Documents/NotesBackups/apple_cloud_notes_parser/lib/AppleBackup.rb:240:in `rip_notes'
	from notes_cloud_ripper.rb:194:in `<main>'

When I use the script with a copy of the folder group.com.apple.notes it gives the DB version correctly Guessed Notes Version: 17 on Mac

ruby notes_cloud_ripper.rb -m ../NotesBackup_7.11.2024/group.com.apple.notes

Starting Apple Notes Parser at Tue Nov 19 21:41:12 2024
Storing the results in ./output/2024_11_19-21_41_12

Created a new AppleBackup from Mac backup: ../NotesBackup_7.11.2024/group.com.apple.notes
Guessed Notes Version: 17 on Mac
Apple Decrypter: Attempting to decrypt objects without a password list set, check the -w option for more success
Updated AppleNoteStore object with 955 AppleNotes in 41 folders belonging to 1 accounts.
Adding the ZICNOTEDATA.ZPLAINTEXT and ZICNOTEDATA.ZDECOMPRESSEDDATA columns, this takes a few seconds

Successfully finished at Tue Nov 19 21:41:14 2024

I get all plain text of all notes correctly this time although I'm seeing errors with handling all attachments.
ERROR -- : Can't call back_up_file with filepath_on_disk that does not exist: ../NotesBackup_7.11.2024/group.com.apple.notes/Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/Previews/7B77107A-E03B-4E74-B042-A49A4BDB72C3-1-768x768.png
more from debug_log.txt
[2024-11-19T21:41:12.660784 #82489] DEBUG -- : Embedded Thumbnail 7B77107A-E03B-4E74-B042-A49A4BDB72C3-1-768x768:
	Expected Filepath: 'Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/Previews/7B77107A-E03B-4E74-B042-A49A4BDB72C3-1-768x768.png' (length: 105),
	Expected location: '../NotesBackup_7.11.2024/group.com.apple.notes/Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/Previews/7B77107A-E03B-4E74-B042-A49A4BDB72C3-1-768x768.png',
	Orphaned Parent: Embedded Object com.apple.drawing.2: 7B77107A-E03B-4E74-B042-A49A4BDB72C3 with drawing in
E, [2024-11-19T21:41:12.660800 #82489] ERROR -- : Can't call back_up_file with filepath_on_disk that does not exist: ../NotesBackup_7.11.2024/group.com.apple.notes/Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/Previews/7B77107A-E03B-4E74-B042-A49A4BDB72C3-1-768x768.png
D, [2024-11-19T21:41:12.660829 #82489] DEBUG -- : Note 66: Created a new Embedded Object of type thumbnail
D, [2024-11-19T21:41:12.660847 #82489] DEBUG -- : Embedded Thumbnail 7B77107A-E03B-4E74-B042-A49A4BDB72C3-1-768x768-1:
	Expected Filepath: 'Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/Previews/7B77107A-E03B-4E74-B042-A49A4BDB72C3-1-768x768-1.png' (length: 107),
	Expected location: '../NotesBackup_7.11.2024/group.com.apple.notes/Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/Previews/7B77107A-E03B-4E74-B042-A49A4BDB72C3-1-768x768-1.png',
	Orphaned Parent: Embedded Object com.apple.drawing.2: 7B77107A-E03B-4E74-B042-A49A4BDB72C3 with drawing in
E, [2024-11-19T21:41:12.660853 #82489] ERROR -- : Can't call back_up_file with filepath_on_disk that does not exist: ../NotesBackup_7.11.2024/group.com.apple.notes/Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/Previews/7B77107A-E03B-4E74-B042-A49A4BDB72C3-1-768x768-1.png
D, [2024-11-19T21:41:12.660863 #82489] DEBUG -- : Note 66: Created a new Embedded Object of type com.apple.drawing.2
E, [2024-11-19T21:41:12.660887 #82489] ERROR -- : Can't call back_up_file with filepath_on_disk that does not exist: ../NotesBackup_7.11.2024/group.com.apple.notes/Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/FallbackImages/7B77107A-E03B-4E74-B042-A49A4BDB72C3/1_10631477-7452-4585-8D58-3ED7FDE79C6E/FallbackImage.png
D, [2024-11-19T21:41:12.660914 #82489] DEBUG -- : AppleNote: Note 66 replacing attachment 5E029B37-712F-4F82-BC06-35422207AEE6
D, [2024-11-19T21:41:12.661093 #82489] DEBUG -- : Note 66: Created a new Embedded Object of type thumbnail
D, [2024-11-19T21:41:12.661111 #82489] DEBUG -- : Embedded Thumbnail 5E029B37-712F-4F82-BC06-35422207AEE6-1-768x768-0:
	Expected Filepath: 'Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/Previews/5E029B37-712F-4F82-BC06-35422207AEE6-1-768x768-0.png' (length: 107),
	Expected location: '../NotesBackup_7.11.2024/group.com.apple.notes/Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/Previews/5E029B37-712F-4F82-BC06-35422207AEE6-1-768x768-0.png',
	Orphaned Parent: Embedded Object com.apple.drawing.2: 5E029B37-712F-4F82-BC06-35422207AEE6 with drawing in

It looks like this constructed path is expected to exist /group.com.apple.notes/Accounts/... though Accounts folder is definitely not present.

Thanks for the additional information! The first one strikes me as potentially a permissions issue, I believe I was getting similar results when testing the Docker images, hence now making a copy of the data before it runs, as well.
For the second issue, that is a bug that is somewhat known to me that I, embarrassingly, just haven't fixed yet. Most iOS upgrades and Mac upgrades have had roughly the same changes in them. iOS 16 changed how thumbnails are generated on iOS, but apparently didn't change it on Mac. I never made platform-specific code paths, just version-specific. This is one of the reasons I spent a lot of time earlier this year adding a test suite, I needed it to make sure updates to platform-specific stuff didn't break other versions.
My apologies for the delay on fixing it. If you have an iPhone backup on the same mac and sync your notes, pointing it at the backup should work.

Here's my structure, no Accounts folder
ls -l ~/Library/Group\ Containers/group.com.apple.notes
total 73200
drwxr-xr-x@  37 user  staff      1184  9 Nov 14:31 FallbackImages
drwx------    7 user  staff       224  4 Jan  2022 Library
drwxr-xr-x  391 user  staff     12512 17 Nov 15:27 Media
-rw-r--r--@   1 user  staff  37298176 21 Nov 05:38 NoteStore.sqlite
-rw-r--r--@   1 user  staff     32768 20 Nov 09:39 NoteStore.sqlite-shm
-rw-r--r--@   1 user  staff    123632 21 Nov 05:38 NoteStore.sqlite-wal
-rw-r--r--@   1 user  staff       261 10 Nov 00:03 NotesIndexerState-HTML
-rw-r--r--@   1 user  staff       261 21 Nov 05:38 NotesIndexerState-Modern
drwxr-xr-x@   3 user  staff        96 12 Feb  2023 Paper
drwxr-xr-x  868 user  staff     27776 17 Nov 15:27 Previews
-rwx------@   1 user  staff         0 26 Sep  2017 SharingExtensionDeviceUUID
drwxr-xr-x@   3 user  staff        96  4 Jan  2022 Thumbnails
-rwx------    1 user  staff         0 28 Apr  2021 com.apple.notes.databaseopen.lock
drwxr-xr-x@   4 user  staff       128 10 Nov 00:02 tmp

Thanks for sharing that. May I ask if you could look at the older computer to see if it is the same? I'm wondering if the folder difference has anything to do with the longevity of the installation (i.e. a new install might lay things out differently from an install that has carried over from a few versions).
I have had in mind a change for how files are found. Currently I produce one canonical file path and look for it. Between the thumbnail issues and now this, I think I need to produce a few possibilities and check for which of them exists. I'll add that to the list.

I just added an additional check to the Mac backup validation that will ensure the database file opens and has the right columns in it. Would you mind pulling down the latest version and seeing if that behaves differently when you try to use the non-backup folder? It hopefully won't identify the database as version 8 anymore if it is actually a permissions issue.

Ok, I think I have a workaround for you, try my latest commit. I'm working on refactoring the code to be prettier, but now this should find files whether or not an Accounts folder exists.
https://github.com/threeplanetssoftware/apple_cloud_notes_parser/blob/ff95b2bb288a2fd21991589f7e80a651f868113c/lib/AppleBackupMac.rb#L46-L56
I had forgotten we ran into the same issue with iTunes backups (#24), I just never implemented the same fix for Mac backups.

I've just pulled the latest master, commit https://github.com/threeplanetssoftware/apple_cloud_notes_parser/commit/ff95b2bb288a2fd21991589f7e80a651f868113c

The first one strikes me as potentially a permissions issue

You are right about permissions, I just had to give Terminal full File system access.
And everything works perfectly with ruby notes_cloud_ripper.rb -m /Users/user/Library/Group\ Containers/group.com.apple.notes
I can see the new folder /output/2024_11_22-13_34_14/files/Accounts/3840DFFC-95CD-4C2C-A016-FCF1D7667202/ where all attachments are now.
Notes text body are there, attachments are there. Fantastic!

May I ask if you could look at the older computer to see if it is the same?

I also did check the old MBP , it's the same FS structure with no Accounts folder through. Maybe it's because I used mac migration tool from old mac to the new mac mini.
I'm speechless, thank you so much for your help and this amazing tool!

I must add this: I did try this latest commit https://github.com/threeplanetssoftware/apple_cloud_notes_parser/commit/ff95b2bb288a2fd21991589f7e80a651f868113c on my old Intel MBP, I'm still having the same error for every note can't modify frozen String, though on ARM mac mini everything works great

I'm speechless, thank you so much for your help and this is amazing tool!

My pleasure! Thanks for the kind words! Thanks also for the confirmation that the intel-based Mac still is unhappy. That might help with troubleshooting. Have a great day!

Hi @threeplanetssoftware, I believe my issue is also related to this. But, each of the notes is truncated.
For all notes I see:
AppleNoteStore: NoteStore tried to rip Note7 but had to rescue error: can't modify frozen String:
Btw, I connected directly to the sqlite file and browsed the ZICCLOUDSYNCINGOBJECT table. I see more content than what is exported here, but even then it's truncated. Do you know where the rest of the data is stored? I see ZICNOTEDATA and assume it's in the blob data and I was thinking this project was decoding that blob and extracting. I haven't looked through this file thoroughly, but it looks like from the log at least that only ZICCLOUDSYNCINGOBJECT is selected -- is that right?
Missing from here which has the larger selection is ZICCLOUDSYNCINGOBJECT.ZWIDGETSNIPPET that I am referring to:
Query is SELECT ZICCLOUDSYNCINGOBJECT.ZNAME, ZICCLOUDSYNCINGOBJECT.Z_PK, ZICCLOUDSYNCINGOBJECT.ZSERVERRECORDDATA, ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT, ZICCLOUDSYNCINGOBJECT.ZCRYPTOVERIFIER, ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT, ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER, ZICCLOUDSYNCINGOBJECT.ZSERVERSHAREDATA, ZICCLOUDSYNCINGOBJECT.ZUSERRECORDNAME, ZICCLOUDSYNCINGOBJECT.ZACCOUNTDATA, ZICCLOUDSYNCINGOBJECT.ZACCOUNTNAMEFORACCOUNTLISTSORTING FROM ZICCLOUDSYNCINGOBJECT WHERE ZICCLOUDSYNCINGOBJECT.Z_PK=?

Hello! Can you please tell me what version of Ruby you are running and what version of Mac OS? Are you on ARM or Intel? The version will be kicked out at the start of the script, if you can copy and paste that whole line, I'd appreciate it.
At this point I ignore the widget snippet, because that data is captured elsewhere in the file.

@nikolsky2 If you still have the Intel Mac, would you mind trying the version I just pushed? I think I finally figured out the Frozen String error, thanks to  @Banaanhangwagen 's debug in #116. Would really enjoy seeing if it works on that machine.

@threeplanetssoftware I've just pulled the latest 0.20 from the master on my intel MBP. All notes are exported successfully now!
Idk if this would actually help you to pinpoint the fix, I did try 0.19.1 on Intel as well, and it still has the same issue with Frozen Strings so it's been definitely fixed in 0.20 now

Thank you for the confirmation!


feat: creating a GitHub action for installing karmor

Create a GitHub action for installing karmor in the CI.
In a big workflow file, this will be prerequisite to https://github.com/kubearmor/KubeArmor/issues/1128
From the installation perspective, one have to use the following in the workflow.
- name: Install karmor
  uses: kubearmor/setup-karmor@main

This can be done in the individual repo as well, but it will be good if we can have this as part of kubearmor.
cc @Ankurk99 @nyrahul

https://github.com/xiao-jay/kubearmor-client/actions/runs/4425881156/jobs/7761479618
This is the ci you want to?

cc @kranurag7 @Ankurk99  @nyrahul
I will write action file in the future.

is this issue still relevant?

I think it's not required anymore.
We had a project which required this but I think it's already handled.
Please check: https://github.com/kubearmor/kubearmor-action?tab=readme-ov-file#action-install-kubearmor


Roll Flutter Engine from f320b8c36fee to 77a27b466452 (3 revisions)

https://github.com/flutter/engine/compare/f320b8c36fee...77a27b466452
2023-06-27<EMAIL_ADDRESS>[Impeller] Fixes for GLES color mask setup (flutter/engine#43225)
2023-06-27<EMAIL_ADDRESS>Roll ANGLE from cba77bceb26c to 02292814a9d3 (1 revision) (flutter/engine#43224)
2023-06-27<EMAIL_ADDRESS>Roll Skia from 370132bcadb1 to 5209dc7702d0 (1 revision) (flutter/engine#43223)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC jimgraham@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


When a migration containing multiple tables fails all tables...

..created up to the point of failure are orphaned
This is because the migration entry does not exist.
Since the down() function may use dropIfExists, I would suggest storing an entry for the migration with a completion boolean which marks it as incomplete, in order for the down() function to be able to clean up failed migrations in development.  Upon migration completion the boolean flag is set to true.

Is this MySQL? If so, the issue is that DDL statements are not transactionable. There’s not really a good workaround for this. Either we create the migration entry and it is incomplete or we don’t create it and we have orphan tables. Running down doesn’t help if the migration is using any methods that can fail (like alter). I personally use the migrate fresh command in this case to reset the database in development.
I should not that other grammars may support transactions with DDL statements.

The only thing I can think of with your suggestion is that we’d be able to print out some message about being in an inconsistent state when running the migrations up again.

In development with a new database migrate fresh might be the way to go, but if you've got a migration on a existing database that you're writing and it fails, then you're in trouble.
The down() command might not cover all cases, but having the ability to run migrate down --once, which would trigger the most recent migration in the database, could be really invaluable.  This might be even more so the case, when looking at a production migration that fails, for whatever reason.
The only way I can see to make that work is pre-inserting the migration record with an incomplete flag ( which you could also use to denote an inconsistent state ).

migrate fresh is only to be used in development against a local database.  You wouldn't actually run it against a production database.
I'm not sure I follow what running migrate down --once would do if we had an incomplete record?  The state still could be 1/2 migrated or 3/4 migrated or whatever.

migrate fresh is only to be used in development against a local database

Right

I'm not sure I follow what running migrate down --once would do if we had an incomplete record?

If the down function was written write, and an entry was in the database, then the migration attempt could be reverted.
public function down()
{
    if (schema.hasColumn('users', 'FK_avatar'))
    {
        schema.table('users', function( table )
        {
             table.dropForeign(['FK_avatar']);
             table.dropColumn('FK_avatar');
        });
    }

}

Coming back to this one.
migrate down --once exists.
migrate fresh will work for all grammars as of qb 7.0.0
Mistakes in production migrations should be fixed with a new migration.


Bypass Pattern Files

Hi;
First of all congratulations on the project, it is a great job.
I am currently testing  pheditor to edit .txt files only and allowing only the following permissions: newfile, editfile, deletefile, renamefile, uploadfile
define ('PATTERN_FILES', '/^^ A-Za-z0-9-_.\/> *) .(txt)$/i');
define('PERMISSIONS', 'newfile,editfile,deletefile,renamefile,uploadfile');

I have seen that the "renamefile" permission can be "abused" in order to execute php code on the server since it seems to be missing a check of the pattern files. A use case:

Create a file from the editor (new-file.txt)
The new file is displayed in the "file manager"
Edit the file "new-file.txt" and add, for example:  <? Php phpinfo.php?>
Double click on the file "new-file.txt"; opens in a new window and displays " <? php phpinfo.php?> as if it were a simple text file
Rename the file "new-file.txt" to "new-file.php"
The renamed file disappears from the file manager
In the tab that was opened in point 4, replace the extension .txt to .php
Refresh the tab and the result of executing the function phpinfo () is displayed;

This can be solved in two ways:
A. Remove 'renamefile' permissions
B. Add a check before executing the rename operation.
In my case, I reuse an existing check in the pheditor code and add it between lines 241 and 242 ( if we see original code ):
pheditor.php
[ ...]
case 'rename':
  if (isset($_POST['path']) && file_exists(MAIN_DIR . $_POST['path']) && isset($_POST['name']) && empty($_POST['name']) === false) {

    $path = MAIN_DIR . $_POST['path'];

    //   NEW CODE  HERE
    // Check valid patter files before rename
    if (empty(PATTERN_FILES) === false && !preg_match(PATTERN_FILES, str_replace(basename($path), '', dirname($path)) . DS . $_POST['name']))  {
      die(json_error('Invalid file pattern: ' )); }**

    $new_path = str_replace(basename($path), '', dirname($path)) . DS . $_POST['name'];

    if ($_POST['path'] == '/') {
      echo json_error('Unable to rename main directory');
[...]

Well, I know that access to the editor is quite restricted, it has its security measures, etc. but in my case, the editor will be used by several people. I also don't know if there are more pheditor users who are interested to know about this and how to fix it. I would appreciate if you think it is a good solution or if you propose something better (sorry, I'm not a developer)
Thank you very much for your work.
Regards,

Problem fixes.
Thanks for your report.


This is the sentences program

hi

My second pull request for the multiples program

Have now submitted the summing program..