aiohttp leaks connections and logs to stdout

Got this output:
Unclosed connection
client_connection: Connection<ConnectionKey(host='s3.amazonaws.com', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>

led me to this issue:
https://github.com/aio-libs/aiohttp/issues/5277
unfortunately, there really isn't much development happening on the client side of the aiohttp house, afaict.
I'm only using get/list_objects_v2, so for my usecase, I may just try to turn aiohttp-s3-client into a sans-io library.  have to give this some thought.  maybe I'll try shoving a httpx.session (my goto async client these days)  into aiohttp-s3-client and see where it breaks, looks like things like resp.status vs resp.status_code.
I'm thinking about just monkey patching this logging away from aiohttp, but digging into the aiohttp code makes me feel unsettled.

okay, I've spent some time thinking about this, and the name typing RequestContextManager gave me a clue.
your examples in the README.md are not desirable.
it should be either:
    # Get object by bucket+key
    async with client.get("bucket/key") as resp:
        data = await resp.read()

or
    # Get object by bucket+key
    resp = await client.get("bucket/key")
    data = await resp.read()
    resp.release()  # must release

This covers all of your examples.
aiohttp documentation:
https://docs.aiohttp.org/en/stable/http_request_lifecycle.html
context manager implementation:
https://github.com/aio-libs/aiohttp/blob/v3.8.4/aiohttp/client.py#L1159
i'm a lot happier about aiohttp now :)

@toppk do you want to fix examples or add mention about it?

yeah, I'll submit a PR later (maybe tomorrow).  I will convert all the examples for async with style, as that matches the recommendations in aiohttp  and the least brittle.

This can be closed now.


Add command to convert Markdown selection to PreTeXt

This PR adds a new command that converts a Markdown selection to PreTeXt.
I thought the markdown-to-pretext conversion code was useful independently and deserving of its own repository so I created one and linked to it. It is currently using a direct git dependency link but I want to move it to a standard npm package soon.
I would be happy with any other arrangement (can move the code into this repo instead, add you as collaborator on my repo, or relocate my repo). But I thought it best for it to be a separate project, hopefully with its own cli soon.
I have not run eslint, as my version of eslint was unhappy with the commands.

@oscarlevin I think I resolved the prettier issues.

Very nice.  I tried this out on a few markdown snippets and it's looking like it works for a good number of elements.  Shall I make some issues on the md2ptx repo for what still needs to be done?  I've forked that so if I find some time I can try to contribute there too.
I like the idea of making this it's own module, and probably should do that with any future latex2ptx scripts as well.

Very nice. I tried this out on a few markdown snippets and it's looking like it works for a good number of elements. Shall I make some issues on the md2ptx repo for what still needs to be done? I've forked that so if I find some time I can try to contribute there too.
I like the idea of making this it's own module, and probably should do that with any future latex2ptx scripts as well.

Absolutely, issues would be great!


An unexpected error has occurred with osprey (version 0.1a)

(dl3) dendisuhubdy@grok-machine:~/data$ molencoder train --dataset molecules/chembl22.h5 --cuda
Epoch 0:
t = 100, loss = 0.0532
t = 200, loss = 0.0467
t = 300, loss = 0.0473
t = 400, loss = 0.0473
t = 500, loss = 0.0468
t = 600, loss = 0.0469
t = 700, loss = 0.0467
t = 800, loss = 0.0474
t = 900, loss = 0.0470
t = 1000, loss = 0.0456
t = 1100, loss = 0.0481
t = 1200, loss = 0.0443
t = 1300, loss = 0.0485
t = 1400, loss = 0.0474
t = 1500, loss = 0.0465
t = 1600, loss = 0.0469
t = 1700, loss = 0.0460
t = 1800, loss = 0.0453
t = 1900, loss = 0.0476
t = 2000, loss = 0.0466
t = 2100, loss = 0.0454
t = 2200, loss = 0.0471
t = 2300, loss = 0.0462
t = 2400, loss = 0.0458
t = 2500, loss = 0.0464
t = 2600, loss = 0.0475
t = 2700, loss = 0.0474
t = 2800, loss = 0.0442
t = 2900, loss = 0.0463
t = 3000, loss = 0.0455
t = 3100, loss = 0.0458
t = 3200, loss = 0.0457
t = 3300, loss = 0.0470
t = 3400, loss = 0.0467
t = 3500, loss = 0.0450
t = 3600, loss = 0.0473
t = 3700, loss = 0.0471
t = 3800, loss = 0.0471
t = 3900, loss = 0.0461
t = 4000, loss = 0.0497
An unexpected error has occurred with osprey (version 0.1a), please
consider sending the following traceback to the osprey GitHub issue tracker at:
        https://github.com/cxhernandez/molencoder/issues

Traceback (most recent call last):
  File "/u/suhubdyd/.conda/envs/dl3/bin/molencoder", line 11, in <module>
    load_entry_point('molencoder==0.1a0', 'console_scripts', 'molencoder')()
  File "/u/suhubdyd/.conda/envs/dl3/lib/python3.6/site-packages/molencoder-0.1a0-py3.6.egg/molencoder/cli/main.py", line 33, in main
  File "/u/suhubdyd/.conda/envs/dl3/lib/python3.6/site-packages/molencoder-0.1a0-py3.6.egg/molencoder/cli/main.py", line 38, in args_func
  File "/u/suhubdyd/.conda/envs/dl3/lib/python3.6/site-packages/molencoder-0.1a0-py3.6.egg/molencoder/cli/parser_train.py", line 55, in func
  File "/u/suhubdyd/.conda/envs/dl3/lib/python3.6/site-packages/molencoder-0.1a0-py3.6.egg/molencoder/utils.py", line 89, in validate_model
ValueError: too many values to unpack (expected 2)

This was a bug from a few commits ago. Should be fixed now in master! Sorry about that!

@cxhernandez no problem! Thanks!


Task specific signature mode.

Right now, the tasks inherit signature mode from the sos instance. Even for PBS task engine, the signature mode is passed to the template and to the sos execute command.
I am wondering if there are cases that we want to keep signatures for main sos workflow, but use different signature mode for the tasks (something like task: sig_mode='ignore'. For example, the cluster might not like the multiple small files created by signatures (#991) and there is no need for signatures for large number of small tasks, or for production runs.

:1st_place_medal: for the 1000^{th} issue ...

For example, the cluster might not like the multiple small files created by signatures (#991) and there is no need for signatures for large number of small tasks, or for production runs.

Sure! But how to prevent rerun in this case? Use some local .sos signatures?

Do not know. Need to think harder.

With the new .task file format, tasks store signatures within the completed .task file. There is therefore no longer a concern on small signature files. The only use of
task: sig_mode='ignore'

would be skipping the calculation of md5 of input and output files, at the cost of not able to skip a task, which is not important for production runs. However, I would argue in the later case a workflow level -s ignore would suffice so there is no need for task-specific signature mode.
So I think this will be  WONTFIX unless there is another strong argument for it.


not getting pop-up every 4 hour

Hey Mario,
I try your script today, I only got 1 pop which is the beginning when I push out the policy from jamf. am I doing something wrong? it supposed to do a pop-up every 4 hours if user press defer?

Yep, the LaunchDaemon reloads regularly in the background and displays the prompt about 4 hours after the last alert, unless you've customized the script or LaunchDaemon payloads and changed the timing. Can you give the latest release a try without any modifications and see if you run into the same issue?

The .pkg that are on the site?
What did you used to wrap it up?
On Mon, Jan 31, 2022 at 6:01 PM Mario Panighetti @.***>
wrote:

Yep, the LaunchDaemon reloads regularly in the background and displays the
prompt about 4 hours after the last alert, unless you've customized the
script or LaunchDaemon payloads and changed the timing. Can you give the
latest release a try without any modifications and see if you run into the
same issue?
—
Reply to this email directly, view it on GitHub
https://github.com/mpanighetti/install-or-defer/issues/71#issuecomment-1026406202,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWSIZAM5OMAPPJSLIN7VHI3UY45HZANCNFSM5NBA62LQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

Yeah, the .pkg file included in the release is an unmodified default payload you can install directly on your systems. If you're making any changes to the payload (e.g. customizing alert text and branding image), you'll need to assemble the new package. README includes guidance on using munkipkg to do this, but other tools should work fine if preferred.

Thank Mario. I will give this a go again tomorrow as I would like to have
custom logo.
If I need to repackage the installer again I would need to Included the
launchdaimon payload and the post install and pre install script?
On Mon, Jan 31, 2022 at 7:26 PM Mario Panighetti @.***>
wrote:

Yeah, the .pkg file included in the release is an unmodified default
payload you can install directly on your systems. If you're making any
changes to the payload (e.g. customizing alert text and branding image),
you'll need to assemble the new package. README includes guidance on
using munkipkg to do this
https://github.com/mpanighetti/install-or-defer#installer-creation, but
other tools should work fine if preferred.
—
Reply to this email directly, view it on GitHub
https://github.com/mpanighetti/install-or-defer/issues/71#issuecomment-1026444105,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWSIZANITVKFK3IJNFASIVDUY5HFVANCNFSM5NBA62LQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

You'll need all of it to ensure a clean installation (preinstall removes existing resources if you've run it before) and to automate reloading the script each time it prompts the user (post install loads the LaunchDaemon the first time, LaunchDaemon handles the recurring checks).

That exactly what I did the other day using munkipkg and wrapping it up
with .plist for the launchdaimon with post and Pre.
I also trying wrapping it up with composer but I never get the initials
pop-up.
Be great if you would post a quick YouTube tutorial.
On Mon, Jan 31, 2022 at 7:33 PM Mario Panighetti @.***>
wrote:

You'll need all of it to ensure a clean installation (preinstall removes
existing resources if you've run it before) and to automate reloading the
script each time it prompts the user (post install loads the LaunchDaemon
the first time, LaunchDaemon handles the recurring checks).
—
Reply to this email directly, view it on GitHub
https://github.com/mpanighetti/install-or-defer/issues/71#issuecomment-1026446554,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWSIZALMOVAWFIACVNZSDYDUY5H67ANCNFSM5NBA62LQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

For what it's worth, the latest release moved custom messaging logos from the script to a configuration profile key, meaning you can make that customization without making any changes to the script or repackaging. Give it a try and see if the alert starts showing up on the unmodified payload.

Going to close this out since it seems like you just need to load the LaunchDaemon as part of your deployment.


Custom width

Would be nice if we can set the width of the sheet.

sent out a PR to that effect so maybe he reviews and adds that as soon as possible.

news on this?

Hi everyone, there is an open pr about this and I will review it as soon as possible. Any news I communicate in this issue.

Hello Devs! version 1.0.2 has been published on pub.dev with this improvement developed by @RGPaul. Thank you for everything!


Instruction to use ansible is not working

Hi,
I tried running the azure resource group creation playbook by following these steps given in the MS site:https://docs.microsoft.com/en-us/azure/developer/ansible/install-on-linux-vm?tabs=ansible#file-credentials
but it is throwing this error
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'urllib3.packages.six'
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (msrestazure) on ansible-controller's Python /usr/bin/python3.6. Please read module documentation and install in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}
This issue is logged here:
https://github.com/ansible/ansible/issues/74758
Regards
Rajaniesh

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 88ac6b7d-76dc-189b-4ce7-d5f2b4043162
Version Independent ID: 5982bd9f-1339-3a62-75a4-35161e88368e
Content: Quickstart - Configure Ansible using Azure CLI
Content Source: articles/ansible/install-on-linux-vm.md
Service: ansible
GitHub Login: @TomArcherMsft
Microsoft Alias: tarcher

Hey @rajaniesh,
Are you running the playbook from within Azure Cloud Shell? If not, which version of Ansible are you using?
Thank you,

I could make it work on CentOS 8 because on CentOS 7 it has a conflict with Python version because Python version required by Ansible is 3.5 or higher but by default the version on CentOS 7 is not more then 2.75 and if we upgrade it it has undesirable effects like yum is not working and this procedure is not written any where in your documentation. I have wasted so much time and energy on this finally I have taken fresh CentoS 8 machine and installed python 3.5 then it started working

@rajaniesh I appreciate the reply. To confirm the issue is between the Python version and Ansible, not the az Ansible collection correct?
Do you have a requirement to use Ansible 2.10+? Another option might be to use Ansible 2.9 which doesn't have the same Python version requirement. It's not an ideal long-term solution, but might work in the short-term if that's what will work in your environment.

You raise an excellent point @rajaniesh, I will update the doc noting the Python version requirements for ansible-core.
Sidenote:
Do you have a requirement to use Ansible 2.11+? Another option might be to use a lower ansible-core version which can use Python 2.75.

I've added a note detailing the Python version requirements for the different Ansible versions. A PR has been submitted and the changed should be public within a few days.
Thank you for opening the issue @rajaniesh!


Update or replace django-environ

We use the package django-environ in edx/blockstore. This package was last released in 2018 and it hasn't yet added support for Django3.2. Please follow the guidance in https://openedx.atlassian.net/wiki/spaces/AC/pages/3036972032/Handling+Outdated+Dependencies to resolve the problem this poses for the Open edX Django 3.2 upgrade.

Hey, I'm happy to take this one.

@BbrSofiane Thank you for reaching out, I highly appreciate your interest and help with this. I have assigned this task to you. Please feel free to open a PR once you are ready, I will take care of it from there on.

The original django-environ repo doesn't seem to be maintained and despite other prompts (@joke2k/django-environ#292 and @joke2k/django-environ#282) it doesn't look like the maintainer is interested in bringing it back to life.
A fork was created, django-environ-2, and that one is actively maintained and currently supports Django 3.2.

Opened edx/blockstore#121.

@BbrSofiane Looks like https://github.com/edx/blockstore/pull/121 was merged.

@natabene turns out I have to make another change.
The library I used, django-environ-2, is archived and its maintainer is now looking after django-environ which has now been updated to support Django 3.2.
I'll create a new PR.

@BbrSofiane Thanks so much for your update!

@BbrSofiane Thank you for an update, we will review.

@BbrSofiane thanks for your help on this issue. After reviewing the changes, I'll now close this issue.


Custom logging (duplicate logs on Linux with debug builds)

Is your feature request related to a problem? Please describe.
Yes. Currently on Linux the call set_enable_write_console_messages_to_stdout() hinders a custom logging solution during development by duplicating log messages.
Describe the solution you'd like
Remove the call to set_enable_write_console_messages_to_stdout(); from what I can tell there is no equivalent for Windows/macOS so I think it is consistent if we remove this.
Describe alternatives you've considered
Not sure there is any alternative to prevent duplicate messages in debug builds on Linux as I don't think it's practical nor desirable to disable custom logging in that situation.
Would you assign yourself to implement this feature?

[x] Yes

Additional context
I want console log messages to always be routed via my own logging facility regardless of build type and whether console is actually available. Here is an example of how I achieve this:
const embedded = typeof rpc !== 'undefined';

if (embedded) {
  const tee = false;

  const console_methods = {
    log: console.log,
    info: console.info,
    warn: console.warn,
    error: console.error,
  }

  console.log = function() {
    const args = Array.prototype.slice.call(arguments, 0);
    if (tee && typeof console_methods.log === 'function') {
      console_methods.log.apply(null, args);
    }
    return window.rpc.notify('console.log', ...args)
  }
  console.info = function() {
    const args = Array.prototype.slice.call(arguments, 0);
    if (tee && typeof console_methods.info === 'function') {
      console_methods.info.apply(null, args);
    }
    return window.rpc.notify('console.info', ...args);
  }
  console.warn = function() {
    const args = Array.prototype.slice.call(arguments, 0);
    if (tee && typeof console_methods.warn === 'function') {
      console_methods.warn.apply(null, args);
    }
    return window.rpc.notify('console.warn', ...args);
  }
  console.error = function() {
    const args = Array.prototype.slice.call(arguments, 0);
    if (tee && typeof console_methods.error === 'function') {
      console_methods.error.apply(null, args);
    }
    return window.rpc.notify('console.error', ...args);
  }
}

Currently if tee is true then it routes via the existing console functions and the duplicates will appear on Linux for debug builds. I would like to be enable tee and have no duplicates.

Thanks @wusyong :clap:


ValueError: 'order_id' is both an index level and a column label, which is ambiguous.

Depth Feature Synthesis Error
I am trying to apply DFS to my EntitySet. It is simple and has the following structure:
Image of Structure
However, ft.dfs is not appliable nor to target_entity='orders', neither to target_entity='customers'. I found an open issue in your repository: issue
But this is still open and I can't figure out how to fix the errors which appear with both of target entities.
Bug/Feature Request Description
There are warnings of the following kind:
WARNING    Attempting to add feature <Feature: customer_zip_code_prefix / 1> which is already present. This is likely a bug.
Error's message:
ValueError: 'order_id' is both an index level and a column label, which is ambiguous.
Expectations
How to get here a feature_matrix_spec without any errors and warnings?
Thanks for your time!
Output of featuretools.show_info()

[paste the output of featuretools.show_info() here below this line]
Featuretools version: 0.15.0
Featuretools installation directory: /usr/local/lib/python3.6/dist-packages/featuretools
SYSTEM INFO
python: 3.6.9.final.0
python-bits: 64
OS: Linux
OS-release: 4.19.104+
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
INSTALLED VERSIONS
numpy: 1.18.4
pandas: 1.0.4
tqdm: 4.41.1
PyYAML: 3.13
cloudpickle: 1.3.0
dask: 2.12.0
distributed: 1.25.3
psutil: 5.4.8
pip: 19.3.1
setuptools: 47.1.1

Hi @MaratSaidov , thanks for the bug report.  Happy to help you with these issues.  Could you share the code you used to create the entityset, and attach the structure image to your comment directly?

Hi @rwedge , thank you for your help. Here is the structure of an EntitySet:

Here is the code which creates this EntitySet:
es = ft.EntitySet(id = 'automl')

es = es.entity_from_dataframe(entity_id='orders', dataframe=olist_orders, index='order_id')

es = es.entity_from_dataframe(entity_id='customers', dataframe=olist_customers, index='customer_id')

r_customer_order = ft.Relationship(es['customers']['customer_id'], es['orders']['customer_id'])

es = es.add_relationships([r_customer_order])

Then I call a depth wise synthesis function:
feature_matrix_spec, feature_names_spec = ft.dfs(entityset=es, target_entity='customers',
                                                 agg_primitives=agg_primitives,
                                                 trans_primitives=trans_primitives,
                                                 max_depth=2, features_only=False)

Here agg_primitives and trans_primitives are just lists of randomly taken primitives from your library.
The described dataframes ( olist_orders, olist_customers) actually have the unique identifiers order_id and customer_id respectively.

Hi @MaratSaidov ,
I made a mock dataset and was able to replicate the warnings you were getting, we can investigate those.  I didn't encounter the ValueError you reported.  Are you able to share the data / code you used to create the EntitySet.  A reproducible example would help a lot with diagnosing this.
Other information that would help:

The full stack trace of the ValueError
If you could find the smallest group of primitives necessary create this error

Hi @rwedge,
I allocated a problem a bit. Consider two structures:

This input is suitable to a bunch of DFS runs. However, if I try to add an edgeless table to this structure:

In this case I get the described above ValueError.
Here you could find a bug example notebook: Bug Example
Here the data tables are placed: data
The original dataset you could find here: dataset
Notes about other information:

A full stack trace is quite large, so you might find it in the notebook I attached to.
Actually the problem appears in lots of randomly generated primitives sets. Consider the following for example:

agg_primitives = ['last']
trans_primitives = ['age']

Thank you!

Hi @MaratSaidov,
Thanks for the example notebook!  I was able to reproduce the error and figure out what was causing it.
In both of the graphs you included in your last post, the index column for the order_items entity, order_id, is listed as id variable type instead of index type.  This happened when adding the relationship between orders and order_items.  Featuretools does not support having having a column be both the index column for an entity and a foreign key column.  There should be an error when trying to add a relationship that would cause this scenario.  Since there was no error when trying to add the relationship, there ends up being an error later during the calculation that is harder to diagnose.
However, we can fix this relationship problem.  Instead of using order_id as the index for the order_items entity, we should use a different index column.   Featuretools can create a new index column automatically when creating an entity from dataframe.
We can remove one preprocessing step:
# choose only unique indices
print("olist_order_items.shape:", olist_order_items.shape)
olist_order_items = olist_order_items.iloc[olist_order_items.drop_duplicates(['order_id']).index, :]
print("olist_order_items.shape:", olist_order_items.shape)

I removed the above code dropping rows since with a different index than order_id for the order_items entity,  it is ok to have multiple rows with the same order id.
Then we need to add a new index.  entity_from_dataframe has a make_index option, so we just need to update the line where we add the order_items entity.
es = es.entity_from_dataframe(entity_id='order_items', dataframe=olist_order_items, index='order_item_unique_id', make_index=True)

After making those changes I was able to run dfs using Last and Age without errors.
I'm changing the name of the issue to reflect what we should do to fix it in the future, which is to prevent adding a relationship where the child variable is also the index of the child entity.


"order" in Content Categories not sorting right

One more item isn't working like it should (or not working like it does whit Direct Publisher processing things anyway).
The "order" field has 4 options, 3 of which are automated.  I noticed that some Categories on another channel were not ordered right and went to look at the Content Categories I had in the JSON feed.  The first problem was that I had all of them marked as "most_recent" when about 4 of them should have been "chronological" - so I changed the option to chronological and re-created the channel and side loaded the new version to a Roku device.
BUT . . . nothing changed in the channel Categories created.  Everything was still displayed as "most_recent".
In the Direct Publisher processing of feeds it bases "chronological" on either the "releaseDate" or "dateAdded" (I really don't know which because I always set them to the same date).  But in the wizard it doesn't seem to matter what it is set to ("most_recent" and "chronological" seem to produce the same results.
Here is the feed I am working with.  The first Category (most recently produced things) is supposed to be showing the most recent items first (which it does - and it also happens to be the order they appear in the feed, I always add the new stuff to the top).  But the next 4 Categories SHOULD be sorted and listed based on the "chronological" option.  So the episode numbers should count up rather than backwards (like the first category is).
184-161
121-160
81-120
41-80
1-40
http://sabbathschooltv.com/roku/whatsupprof.json
At some point I'm going to have to rework the whole thing and list everything by years or something.
But that reminds me of another thing I have been wondering.  Is there any limit on how many items can be in a Category in a row now?  Roku always said there was a limit of 40 for some reason.
Also, can there be more than 15 Categories/rows now?  They also imposed that limit for some reason.  (???)

I also just noticed that another Category ("Francois du Plessis Series") is supposed to be listed by "most_recent" but I think that everything is just listed in the order the Series appear in the feed.  So really no ordering of things at all.  That would explain why the first 5 Categories are all listed as "most_recent" too (since they is the order each appears in the feed (latest always added to the top).

This will be looked into as an enhancement.  It will not make the next version.

Looking more "exactly" at the JSON feed specification and on how things work with DP now  I would guess that the "dateAdded" is used for sorting by DP, because when I've added new things to a series it always bumps it up to the front of the Latest Category list again.
Episodes in a Series have both "releaseDate" and "dateAdded" while the Series itself only has a "releaseDate".  It would be nice if the JSON feed specification was more EXACT in its description.
https://developer.roku.com/docs/specs/direct-publisher-feed-specs/json-dp-spec.md#category
The order of the category. Must be one of the following:
manual: For playlists only
most_recent: reverse chronological order
chronological: the order in which the content was published (e.g., Episode 1, Episode 2, etc.)
most_popular: sort by popularity (based on Roku usage data).

@jsonnentag I think I have a fix for this.  Would you be willing to test it before I release it?

@jsonnentag I think I have a fix for this. Would you be willing to test it before I release it?

OK.
I was about to start testing out the latest version you created, so might as well test out this fix too.  :)

Thanks. I will put together a zip package for you tomorrow. You'll need to add your manifest to it. This test version sorts items in the grid row based on the  'releaseDate' field.

@jsonnentag Please add your manifest to this zip file and test it.  Each grid row is sorted on the 'releaseDate' field in the JSON feed file.  Rows that specify 'manual' are not affected.  Let me know if this sorts correctly.  If so, I'll make it generally available.
DPtoSceneGraph.zip

OK, I tested it out and I see why it was giving some strange results.  I'll attach the manifest file so you can see the same thing.  But I think it is working according to how you set it up, just not the same are the original DP setup.  I think DP from Roku used the full date and time of "dateAdded" rather than "releaseDate".
I was looking at the first 3 rows of content and thinking everything is looking good, then I got to the rows for "Episodes: 41-80" and "Episodes: 1-40" and things looked scrambled.  But that is because I added those to the JSON file all at one time after they had a lot of episodes and since I didn't go and figure out what date they really released them I just went with a date and then added 1 hour to the "dateAdded" value so that they would be listed in order.
*** The first row is sorted the opposite of the others, because it has the latest episodes.  I just wanted the latest to be first and most easily accessible.  Sometime I need to rework the whole arrangement of things and make it a series with years as seasons.
So I don't know if you would want to change that sorting to be based on "dateAdded" and also based on the time along with the date or just tell people what to expect to be used for the sorting and that it isn't the same as the original DP.  I could easily go through things in the JSON feed and change the "releaseDate" values to sequential dates.
(Apparently you can't select a file to attach if it doesn't have an extension, so I just added .txt and you can delete it after DL.)  :)
manifest.txt

I'll take a look at what, if anything, I can use from the 'dateAdded' field.
I'm not sure using 'dateAdded' is appropriate, in my opinion, since it simply means the date the content was added to the feed file.  That's why it is part of the Content object.  'releaseDate', however, is on the base level of the object and provides the date the content was first made available to the public.
You can, however, achieve some of you goal by defining a 'Newly Added' (playlist) category to control the order of new content.
Lastly, your manifest file indicates you're using a old version.  As such, you're missing out on some new features.

I had to view the JSON Feed Specification to see what they "really" SAY (not that they construct things based on what they say . . .) and they describe things to function the way that you set things up, even though they used the "dateAdded" for their sorting when they claim that the sorting was done based on "releaseDate".  Just in case they say things differently in other places, this is where I found the following:
https://developer.roku.com/docs/specs/direct-publisher-feed-specs/json-dp-spec.md#shortformvideo
releaseDate 	string 	Required 	The date the video first became available. This field is used to sort programs chronologically and group related content in Roku Search. Conforms to ISO 8601 format: {YYYY}-{MM}-{DD}. For example, 2020-11-11
https://developer.roku.com/docs/specs/direct-publisher-feed-specs/json-dp-spec.md#content
dateAdded 	string 	Required 	The date the video was added to the library in the ISO 8601 format: {YYYY}-{MM}-{DD}T{hh}:{mm}:{ss}+{TZ}. For example, 2020-11-11T22:21:37+00:00.
This information is used to generate the “Recently Added” category.
I still haven't looked at the latest "beta" you created.  Did you ever create a "Recently Added" category option?  I think that is about one of the last things I could see that was in the original DP but I hadn't seen in the files you created.  I wasn't like a super critical thing, but it is kind of nice for people to be able to see what is new when there are a pile of things to sift through.  :)

When I was maintaining a DP channel, I would define a playlist row that contained all newly added content. Each month or so, I would update the list by adding new items and removing "old" ones.

Resolved in beta.5 release.


linker: Add ROM_SECTIONS location

Allow putting sections after common-rom.

I actually no longer have a dependency on this, but I guess it doesn't harm either?
In-tree there are many uses of zephyr_linker_sources(SECTIONS ...), which I thought had the behavior I was expecting like the ROM_SECTIONS in this PR.

Ping @ithinuel is comment addressed?

Allow putting sections after common-rom.

More details in the commit message following https://docs.zephyrproject.org/latest/contribute/guidelines.html#commit-message-body are appreciated, This is very short and missing the why and any further details.

@nashif I've updated the commit message
@tejlmand does the SECTIONS put after bss/noinit make sense to begin with? Shouldn't it be always in ROM? Ref #72307

Assigned to @tejlmand to get feedback


More slots in processed datasets?

At the moment a 'common' dataset has the following format (src/datasets/api/anndata_dataset.yaml):
AnnData object with n_obs × n_vars = 16382 × 18771
    obs: 'celltype', 'batch', tissue'
    uns: 'dataset_id'
    layers: 'counts', 'log_cpm', 'log_scran_pooling', 'sqrt_cpm'

The log_cpm, log_scran_pooling and sqrt_cpm layers were added because a lot of methods use one of these three normalisation techniques as the default.
However, I'm noticing a lot also use a PCA dimensionality reduction or the set of HVG genes. We could add those slots to be part of the spec as well, so that a common dataset looks as follows:
AnnData object with n_obs × n_vars = 16382 × 18771
    obs: 'celltype', 'batch', tissue'
    var: 'hvg_log_cpm', 'hvg_log_scran_pooling', 'hvg_sqrt_cpm'
    uns: 'dataset_id'
    obsm: 'log_cpm_pca', 'log_scran_pooling_pca', 'sqrt_cpm_pca'
    layers: 'counts', 'log_cpm', 'log_scran_pooling', 'sqrt_cpm'

Thoughts?

Is it wise to store all of this in the same object? For large datasets,
we're effectively making the dataset resource usage (both memory and
network) ~5x bigger (4 layers, 3 hvg layers) when most methods will only
use one of these. Could we instead have the component compute all of these
and save them as different outputs which methods can then choose from?
On Wed, 16 Nov 2022 at 10:37, Robrecht Cannoodt @.***>
wrote:

At the moment a 'common' dataset has the following format (
src/datasets/api/anndata_dataset.yaml
https://github.com/openproblems-bio/openproblems-v2/blob/183eeb8c4d893fd6a14b24bda3f9e70ea2681687/src/datasets/api/anndata_dataset.yaml
):
AnnData object with n_obs × n_vars = 16382 × 18771
obs: 'celltype', 'batch', tissue'

uns: 'dataset_id'

layers: 'counts', 'log_cpm', 'log_scran_pooling', 'sqrt_cpm'

The log_cpm, log_scran_pooling and sqrt_cpm layers were added because a
lot of methods use one of these three normalisation techniques as the
default.
However, I'm noticing a lot also use a PCA dimensionality reduction or the
set of HVG genes. We could add those slots to be part of the spec as well,
so that a common dataset looks as follows:
AnnData object with n_obs × n_vars = 16382 × 18771
obs: 'celltype', 'batch', tissue'

var: 'hvg_log_cpm', 'hvg_log_scran_pooling', 'hvg_sqrt_cpm'

uns: 'dataset_id'

obsm: 'log_cpm_pca', 'log_scran_pooling_pca', 'sqrt_cpm_pca'

layers: 'counts', 'log_cpm', 'log_scran_pooling', 'sqrt_cpm'

Thoughts?
—
Reply to this email directly, view it on GitHub
https://github.com/openproblems-bio/openproblems-v2/issues/17, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACA3DXZ45RHVGXGLXQAI5XLWIT5UFANCNFSM6AAAAAASCLYY4M
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Hmm.. I made the switch from layers: {'counts': ..., 'normalized': ...} to this variant because it's more practical in that a method can at runtime decide which normalisation method to use or choose its own; my thinking was that if it becomes too big to load into memory we can always use the on-disk format.
But you're absolutely right, transfer costs also play an important role in this equation, and there's no way around it with AnnData as a data format.
I'll make the necessary changes tomorrow to drop the various layers in favour for one normalized layer per h5ad file. Doing so resolves the original problem in this issue ^^

3 hvg layers
I wasn't intending on making a separate hvg matrix but rather store a boolean in .var so the AnnData can be subset when needed.


Exceptions handling: a proper global approach

The approach in this commit is only partial
Same for most the addSyncTaskListVer8() parsed options + all the catch(Exception e){} empty entries
exp+proof of concept: we manually edit settings file and change entries for dst_offset and time_diff_allowable_time to 600. Open SMBSync2, import the hacked task list. Do not enter settings, exit SMBSync2. Open the auto-saved task list created on exit: it will contain the 600 value for both the hacked fields.
If we start the sync: setSyncOptionDifferentFileAllowableTime(600) is triggered and getSyncOptionDifferentFileAllowableTime() will always return 600 in the active session unless we go to task settings. The parser Exception won't trigger since 600 is an integer. The sync task will effectively apply 600 seconds time offset. Same for the dst offset. Only if we edit the task in advanced settings, problem is fixed
Whenever we read a settings file, we must account for corruption, user modifications and exceptions. I think all the entries in addSyncTaskListVer8() should be checked at read, and proper default values set if invalid entries detected. This is a rule when a program accepts values from a user input or external file.
fix approach 1 (faster to code, but complicated to maintain):
exp (code not exact):
private static void addSyncTaskListVer8(..) {
                ...
                int set = Integer.parseInt(parm[34];
                // add code to handle exception if it is an invalid string like "zzz"
                if (set != 1 or or 3 or 10) // valid range or values
                    set = 3; // default value
                stli.setSyncOptionDifferentFileAllowableTime(set);
                ...
}

A better approach 2 example: (simpler for management and code update)
(SyncTaskUtil.java)
// we only check if it is valid type, integer vs string for example
private static void addSyncTaskListVer8(..) {
                ...
                int set;
                try {
                       set = Integer.valueOf(parm[34]);
                     } catch(Exception e) {
                       set = syncOptionDeterminChangedFileByTimeValue_default;
                     }
                stli.setSyncOptionDifferentFileAllowableTime(set);
                ...
}

(SyncTaskItem.java)
// here we define the defaults
private int syncOptionDeterminChangedFileByTimeValue_default = 3;
public void setSyncOptionDifferentFileAllowableTime(int p) {
   if (p != // code to check valid range / values) {
       p = syncOptionDeterminChangedFileByTimeValue_default;
       // code to warn in messages that an invalid entry was read and resetted to default
   }
   syncOptionDeterminChangedFileByTimeValue = p;
}

Approach 3,: Even more simpler management of the code
Just read all user settings in addSyncTaskListVer8() and pass them as strings to
ValidateInput_SyncOptionDifferentFileAllowableTime(string value) functions
(SyncTaskUtil.java)
// read all user settings as strings
private static void addSyncTaskListVer8(..) {
                ...
                stli.ValidateInput_SyncOptionDifferentFileAllowableTime(parm[34]);
                ...
}

(SyncTaskItem.java)
// validate entries and set defaults

private int syncOptionDeterminChangedFileByTimeValue_default = 3;
public void ValidateInput_SyncOptionDifferentFileAllowableTime(string value) {
   int set;
   if (value != // code to check valid type, range and values) {
       set = syncOptionDeterminChangedFileByTimeValue_default;
       // code to warn in messages that an invalid entry was read and resetted to default
   } else set = Integer.valueOf(value);

   setSyncOptionDifferentFileAllowableTime(set);
}

This way all defaults and type/range/values are checked in same code area
shouldn't be complicated to code at all. Once done, maintenance is easier.
Hope you can fix it as you are used to the code. It would take me weeks to look through all your code to get the proper defaults :-)
Best regards

hope you got my point of view and sorry for long post :-)

Thank you for your suggestion.

You're welcome and thank you for the great work !
I will commit a few enhancements / fixes, but it looks really great now 👍


slow startup time

b554_main.cpuprofile.txt
b554_renderer.cpuprofile.txt

VSCode Version: Code - Insiders 1.13.0-insider (eaa887c507e1614f90c9f04a30ceda775759e814, 2017-06-06T10:23:09.381Z)
OS Version: Darwin x64 16.7.0
CPUs: Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz (8 x 2800)
Memory (System): 16.00GB (0.46GB free)
Memory (Process): 245.88MB working set (0.00MB peak, 176.03MB private, 141.80MB shared)
Load (avg): 2, 2, 2
VM: 0%
Initial Startup: yes
Screen Reader: no
Empty Workspace: no
Timings:

Component
Task
Time (ms)

main
start => app.isReady
263

main
app.isReady => window.loadUrl()
81

renderer
window.loadUrl() => begin to require(workbench.main.js)
452

renderer
require(workbench.main.js)
453

renderer
-> of which require() node_modules
101

renderer
create extension host => extensions onReady()
368

renderer
restore viewlet
483

renderer
restore editor view state
375

renderer
overall workbench load
599

main + renderer
start => extensions ready
1639

main + renderer
start => workbench ready
1861

:warning: Make sure to attach these files: :warning:
-/Users/sundarms/b554_main.cpuprofile.txt
-/Users/sundarms/b554_renderer.cpuprofile.txt

Thanks for this issue. I seems that the profiles are a little short due to https://github.com/Microsoft/vscode/issues/27852. From the trace things seem not too bad you might have been alerted without a valid reason. Starting under 2seconds is below our averages


Revert "Introduce timestamps and timeouts (#1760)"

Should fix https://github.com/AdoptOpenJDK/openjdk-build/issues/1769
This reverts commit 5aa149aefe096724be75f93bce0c2894cc311ff2.

Approved via slack by @karianna


IBM Certificate Manager is deprecated -> Replace it by Red Hat Cert Manager

Is your feature request related to a problem? Please describe.
At the time of writing, CP4BA is running foundational servie v4.6.5
I just noticed that IBM Certificate manager (cert-manager) Operator is deprecated since Foundational Service v4.6  (cf. this link)
Foundational Service documentation recommends to Migrate from IBM Certificate Manager to Red Hat Cert Manager.
Describe the solution you'd like
Swapping the IBM Certificate manager for the Red Hat one would align with futur usage, and prevent having to do a hotfix the day ibm-cert-manager causes a problem with furtur CloudPak installations.
Describe alternatives you've considered
None, as this is the recommended way going forward by the public documentation of IBM Foundational Services.
Additional context
N/A

We will make this change as part of the CP4D 5.1 release #836

Resolved


feat: add vertical concatinator to common

It seems I don't get an error here for missing config as well when running test_registered_functions. This is kind of weird.


#bug; Withdrawal and Deposit mean opposite?

Describe the bug
There is a problem in the meaning of the actions: the withdraw and the deposit actions.
Reproduction
When I search for some withdrawal actions, I take some hash and look it up in Scan Polygon, all of them are withdrawal actions.
Link to subgraph on uniswap-v3-polygon
Screenshots

Addressing this bug in this PR. Will begin indexing. Should take ~2 weeks to index for most Uniswap V3 subgraphs.


[sig-devex][Feature:ImageEcosystem][perl][Slow] hot deploy for openshift perl image  hot deploy should work

This image-ecosystem test fails on ARM because it depends on behaviour specific to mod_perl, which is only included in the RHSCL (RHEL 7) perl images which are not available on ARM.  The UBI8 perl images which are available for ARM use mod_fcgid and behave differently.  Therefore, the test needs to be adapted to different behaviour before switching it to the UBI8 image.
https://github.com/sclorg/s2i-perl-container/issues/206

/lifecycle frozen

This test is now skipped on ARM in 4.10, but will continue to fail in 4.9.

4.10 is now generally available.


language service response documentation file path (QuickInfo、CompletionEntryDetails、SignatureHelp)

Suggestion
in vscode IntelliSense will rendered markdown documents, but now the base URI is file being editing.
but should be relative to defining file.
So we need to add a file field to any response with documentation to render the markdown link / image correctly
vscode issues https://github.com/microsoft/vscode/issues/160018
🔍 Search Terms
QuickInfo、CompletionEntryDetails、SignatureHelp

[x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
[x] This feature would agree with the rest of TypeScript's Design Goals.

⭐ Suggestion
response documentation file path. when commands QuickInfo、CompletionEntryDetails、SignatureHelp.
📃 Motivating Example
💻 Use Cases
Improve vscode typescript documentation rendering correctness

Duplicate of #47718


Send alerts to Stashboard

Stashboard is a python status page with a rest api.
http://www.stashboard.org/
It would be great if cabot could update the status page automatically.

Seems it have been care of by @packetcollision see issue #197, though i haven't been able to make it work locally


Is there a way to support Java and JVM metrics?

Is there a way to support Java and JVM metrics?
So we can see inside the JVM and get recommendations from inside, as it is shown outside the JVM as fully used.
Thanks!

Hey @ErezWeiss
Can you explain what are you trying to achieve?
You'd like to see cpu and memory usage of the java process over time?
How are you going to use that information?

Hi @arikalon1
KRR's memory suggestions rely on the general consumption of the container, but Java always allocates it all. The real info is inside the JVM, and I want to get suggestions based on it.


Identify rivers that ends on a stream

The correct way is that waterway=stream ends on waterway=river but not the opposite.
It will be interesting to have a section for them.

The rule I always heard is “if you can jump over it, then it's a =stream, otherwise it's =river”. It's possible for a river to get narrower until it's a stream. So I don't think this is always a mapping mistake.
However I think it would be useful to highlight this cases, to allow mappers to find mistakes. I have an idea how to do it.

Here's a first attempt: http://data.waterwaymap.org/planet-waterway-stream-ends.geojson (data will not update)
(As of now) It's a 22 MiB GeoJOSN file, with 110k Features. It's all the ways from the flowing_water filter, excluding anything with waterway=stream. The normal “flowing water” filter is in ``flowing_water.tagfilterfunc](https://github.com/amandasaurus/waterwaymap.org/blob/main/flowing_water.tagfilterfunc), I did a quick hack to exclude =stream: [flowing_water_wo_streams.tagfilterfunc`
Each point contains upstream_m, which is the totaly length of all upstream waterways which end here. This
You can download load this file into JOSM and work on the data.
Remember not all points are mapping mistakes. 🙂
Is this sort of data OK for you? If so, I can add it to the regular generation, and make a map based on that.

@angoca @mxdanger  What do you think of that data file? Is that OK? Is that the sort of thing you want?
If so, I can generate that regularly and make some sort of web page.

Hi @amandasaurus, the file is great. It has helped me to correct these invalid transitions.
https://www.instagram.com/p/C_a_JBmupEC/
https://x.com/OSM_Colombia/status/1830645693784170518
If you can generate it in a regularly basis it will be great.

Excellent! 🙂.
You can download the file from data.waterwaymap.org/planet-waterway-stream-ends.geojson.gz. It's a zipped GeoJSON file that you open in JOSM (or any GIS software). _(NB: it's a different URL from before.
more docs
Have fun mapping. 🙂 If you have any more interesting ideas, please let me know. 🙂


Provide usage instructions in the readMe.

I'm rather a novice at HTML and JS, so even some basic instructions stating how to utilize this in a manner which allows it to automatically receive updates would be useful.

There's not really a way to use it that automatically receives updates, because... the code is not receiving updates!
You can use it in your project by just downloading prefixer.js and putting it in your project's JavaScript directory. Then embed it in your code with a normal

You can use it in your project by just downloading prefixer.js and putting it in your project's JavaScript directory. Then embed it in your code with a normal

No problem, happy to help! Well, the options behave similarly as stuff you would specify on the CLI for other programs (after all prefixer.js is pretty much a self-contained creature), but you specify them in your HTML file as part of the

with

(The value for query is a "CSS selector", the same syntax you use to tell the browser how to decide which elements to style when you write a CSS file. That means if you only want to fix-up one specific element, you could do prefix({query: '#that-element-id'}), for example.)

@towerofnix, would you be able to perhaps diagnose why it doesn't appear to function for me? I have the undermentioned HTML file:

					<time datetime="+2024-10-14T00:22:10+01:00">
						+2024-10-14T00:22:10+01:00
					</time>

					I was recently notified of the undermentioned:

							```eml
							Return-Path<EMAIL_ADDRESS>							X-Original-To<EMAIL_ADDRESS>							Delivered-To<EMAIL_ADDRESS>							Authentication-Results: mail.protonmail.ch; dkim=pass (Good 2048 bit
							    rsa-sha256 signature) header.d=addy.io header.a=rsa-sha256
							Authentication-Results: mail.protonmail.ch; dmarc=pass (p=reject dis=none)
							 header.from=rokejulianlockhart.addy.io
							Authentication-Results: mail.protonmail.ch; spf=pass
							 smtp.mailfrom=rokejulianlockhart.addy.io
							Authentication-Results: mail.protonmail.ch; arc=none smtp.remote-ip=<IP_ADDRESS>
							Authentication-Results: mail.protonmail.ch; dkim=pass (2048-bit key) header.d=addy.io
<EMAIL_ADDRESS>header.b="innjsXKc"
							Received: from mail2.anonaddy.me (mail2.anonaddy.me [<IP_ADDRESS>]) (using TLSv1.3 with
							 cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
							  key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No
							 client certificate requested) by mailin014.protonmail.ch (Postfix) with ESMTPS id
							 4XRfXh2FQHz7QQ5p for<EMAIL_ADDRESS>Mon, 14 Oct 2024
							 01:19:48 +0000 (UTC)
							Received: from mail2.anonaddy.me (mail2.anonaddy.me [<IP_ADDRESS>]) by mail2.anonaddy.me
							 (Postfix) with ESMTPS id 75171FAC83 for<EMAIL_ADDRESS>							 Mon, 14 Oct 2024 02:19:47 +0100 (BST)
							Received: from mx-out2001.wikimedia.org (mx-out2001.wikimedia.org [<IP_ADDRESS>]) by
							 mail2.anonaddy.me (Postfix) with ESMTPS id 607ECFAC24 for
<EMAIL_ADDRESS>Mon, 14 Oct 2024 02:19:44 +0100 (BST)
							Received: from localhost (unknown [IPv6:2620:0:860:cabe:d93e:8745:4d3c:7214]) by
							 mx-out2001.wikimedia.org (Postfix) with ESMTP id 47B7121455 for
<EMAIL_ADDRESS>Mon, 14 Oct 2024 01:19:42 +0000 (UTC)
							Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=addy.io; s=default2;
							 t=1728868787; h=from:from:reply-to:reply-to:subject:subject:date:date:
							  message-id:message-id:to:to:cc:mime-version:mime-version:
							  content-type:content-type:
							  content-transfer-encoding:content-transfer-encoding:list-unsubscribe;
							 bh=z1r8VAJVVEzEcsYJD1ZSc3EAYJLOm5hJ3S43Y0UAj14=;
							 b=innjsXKc3P0BVFcS8zJDSUWguTKEsBL+KNsTIVd1o+EKjbSUmoRtLSnx37bdOVPtSKd9uK
							 Zkz3AX7mM1dXyFMHBrQpMJR3+jNq/kFdI7LlL6TlCT8q9f0shTgWF5bw4CIgNq5AcUzhBc
							 yG+rjK15WpCWOf5XImhYctRKWxgi9L78FzdhfJ2lvDnsYftkOvlAoNZBFsbJR7ip5rr9ae
							 PkEA0YojxfDBsWkOBulBgBkKf5PoaWDxoEURNQQduG2/DJypRvH26oWcfVzNN0GRlFRHyc
							 lvq8ihSd0CUIf0D5ttNdqMCAbjIo608+X15a++/+hJm6++0iQ64lmOMa1NczRA==
							From: "Wikipedia 'wiki at wikimedia.org'"<EMAIL_ADDRESS>							To: Rokejulianlockhart<EMAIL_ADDRESS>							Reply-To<EMAIL_ADDRESS>							Subject: Wikipedia page User talk:Tobias Conradi has been changed by Wbm1058
							Feedback-Id: F:da7fadb2-20ff-45eb-aaf3-c6b0bb22b54b:anonaddy
							Message-Id<EMAIL_ADDRESS>							List-Unsubscribe: <https://en.wikipedia.org/wiki/Special:Preferences>
							X-Anonaddy-Authentication-Results: mail2.anonaddy.me; dkim=pass header.d=wikimedia.org
							 header.s=wiki-mail header.b=M2Je2WyL; dmarc=pass (policy=none)
							 header.from=wikimedia.org; spf=pass (mail2.anonaddy.me: domain of
<EMAIL_ADDRESS>designates <IP_ADDRESS> as
							 permitted sender<EMAIL_ADDRESS>							X-Anonaddy-Original-Sender<EMAIL_ADDRESS>							X-Anonaddy-Original-Envelope-From<EMAIL_ADDRESS>							X-Anonaddy-Original-From-Header: Wikipedia<EMAIL_ADDRESS>							X-Anonaddy-Original-Reply-To-Header<EMAIL_ADDRESS>							X-Anonaddy-Original-To: "Rokejulianlockhart"<EMAIL_ADDRESS>							Mime-Version: 1.0
							Date: Mon, 14 Oct 2024 01:19:47 +0000
							Content-Type: multipart/mixed;boundary=---------------------51e31e9d86a7e637dd4ec912ccd031be
							Content-Transfer-Encoding: quoted-printable
							X-Pm-Spam: 0yezJI6YSpyJec91ztFGcjIwoJyLCvXBZcQniisnOERJt9TTVOUdSRQUiwslOjLdFJEL
							 CN0lSXZ1BElETjIbpAuLTsV0MI1kTEl0R0XSZ1fT0BVRTXh0FFRUQiU6IBdWzSCJLQZ1EOl0X0XT1l
							 LUEIV9RMIi6x0yWjLsMIwIj4C4OM4CxuMTNTMgMlzbGkGVdIlGusdHImLh1xzaWrGlcZ5SuiQXZSXi
							 wNWUkJF9RT9lWJxUQERUl9DWVUVJRSZUJfRURkQPxtFQ06CIRWAzswIjICO444xMCuTMNMMTgzlGbG
							 dkVluIGhHNILRXy0NXdWZuQ9uYmkGVZcV2uyVGZmLy9JdZySCJLQZ1EOl0X1XBZlETEZVRSXJ1Qfxk
							 QkQPxtFQ06CIRWAzswIjICO444xMCuTMNMMTgzlGbGdkVluIGsGJILNnjlJ3bnLlNRlbmjnNcbJ3lv
							 NmLSbdJJTLCfEZUSVEMO91T0TF5pbIjsF0MINlQQ9lRVQTNpbIjdTBLfwSivN2cmciU0xOizCJLY92
							 yiQWZiOx0N9LjzCJLcIi6hJye3YpR4ib2piJObJmvsICemIhNVndG53JbIojiEBVVVQFRIsUysm1IX
							 N3jlJ3bjI7pBSIli0QTO0iyiwSNFUPJ8zRFiFIUOAju3ADMTNxAAwMDyDcMOAD30cDMCLQJ9EUkS1B
							 XRRUJJR1Q0Ti4AsOjTlJIIojt24yMiMiwEiU0xi0OLMjsk9lIWZ1J9pZ1vmZbIpj7SBlI0TiQsiOnh
							 W1aZV2ftFmbSZ6IhzImjGFda91ml5Wa3X1RVkbm13BXXIzwwQjMTNzEwiIifXNaa1Ww0J3bWY05owI
							 jtCJLYlWswNzXlct9Rlb26CIbI1mh0xWaWYuc9tY2umlYZRWfulmZXZ1RVkbmwjILMQjwxIjMmLrNQ
							 icHtCJLYlWswNzXlcw99icmwjoILAjwwUzNTMwAA3MDwjgMNAz3iwCNWbpF9jbFlXRYZ92yt9Ve2bl
							 RI6bChm1IaxWjuQXY2Yt9luYmfWRZZlmu1RXZmbkVIwLjwjQMMIjxrNmLHciQJtLCsWlYXN2hnVGd3
							 b5J5hX2iWUbOJiVBREUEVTVwiIipWFbb9FjlRXY2Zy99weVim9cIojw0UjLTO1QAxNzwzYMNAzysET
							 NnIfBBhc36SIbM4Cw3ADMjN3cQ0MTzjANOADz0UzNCLyJ5fdWtmFbZIS6hdnIGdoNlzbGzC1dZ5Wki
							 IXZXf91JzLCkmlcIojiwcjNzYxcY4YjwjcZYQTwzIDZDZzIExOW9n1I
							X-Pm-Origin: external
							X-Pm-Transfer-Encryption: TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
							X-Pm-Content-Encryption: on-delivery
							X-Pm-Spamscore: 0
							X-Pm-Spam-Action: inbox

							-----------------------51e31e9d86a7e637dd4ec912ccd031be
							Content-Transfer-Encoding: quoted-printable
							Content-Type: text/plain;charset=utf-8

							Dear Rokejulianlockhart,

							The Wikipedia page User talk:Tobias Conradi has been changed on
							2024-10-14 by Wbm1058, see
							<https://en.wikipedia.org/wiki/User_talk:Tobias_Conradi> for the current
							revision.

							To view this change, see
							https://en.wikipedia.org/w/index.php?title=3DUser_talk:Tobias_Conradi&diff=
							=3Dnext&oldid=3D1014530696

							For all changes since your last visit, see
							https://en.wikipedia.org/w/index.php?title=3DUser_talk:Tobias_Conradi&diff=
							=3D0&oldid=3D1014530696

							Editor's summary: Undid revision [[Special:Diff/1014530696|1014530696]]
							by [[Special:Contributions/Paine Ellsworth|Paine Ellsworth]] ([[User
							talk:Paine Ellsworth|talk]]) =E2=80=93 removing {{r to userspace}} which i=
							s
							intended for placement outside of user-space. see [[Template talk:R to
							user namespace#Redirects from user talk to user namespace]]

							Contact the editor:
							mail: https://en.wikipedia.org/wiki/Special:EmailUser/Wbm1058
							wiki: https://en.wikipedia.org/wiki/User_talk:Wbm1058

							There will be no other notifications in case of further activity unless
							you visit this page while logged in. You could also reset the
							notification flags for all your watched pages on your watchlist.

							Your friendly Wikipedia notification system

							--
							To change your email notification settings, visit
							https://en.wikipedia.org/wiki/Special:Preferences

							To change your watchlist settings, visit
							https://en.wikipedia.org/wiki/Special:EditWatchlist

							To delete the page from your watchlist, visit
							https://en.wikipedia.org/w/index.php?title=3DUser_talk:Tobias_Conradi&acti=
							on=3Dunwatch

							Feedback and further assistance:
							https://en.wikipedia.org/wiki/Help:Contents

							    <!--banner-info-->
							This email was sent to<EMAIL_ADDRESS>(https://en.wik=
							ipedia.org/w/index.php?title=3DSpecial:ChangeEmail&returnto=3DSpecial%=
							3APreferences) from<EMAIL_ADDRESS>							To deactivate this alias copy and paste the url below into your web browse=
							r.

							https://app.addy.io/deactivate/da7fadb2-20ff-45eb-aaf3-c6b0bb22b54b?signat=
							ure=3Da3d2cc0098e7d613ddd500305caf5c8edcfd200c2e53eae7c2c46227b2492892
							<!--banner-info-->

							-----------------------51e31e9d86a7e637dd4ec912ccd031be--

							```

					It describes a modification to en.wikipedia.org/w/index.php?title=User_talk:Tobias_Conradi. I asked Poe what the significance of this man was (for I did not recall subscribing), but it didn't know anything about it. It should suffice to say that when I located en.wikipedia.org/w/index.php?title=Wikipedia:Administrators%27_noticeboard/Community_sanction/Archive10&oldid=1090648075#Tobias_Conradi, I realized.

					Additionally, do you recall the undermentioned?:

						I had neglected to do so previously! I hope this remediates gitlab.freedesktop.org/drm/amd/-/issues/3462!

					Unfortunately, that's not true - Prism still crashes the OS.

It's a little basic, because it's just a test file - the content is taken from a larger file that I can't share.
I can't see what should prevent it functioning:

tree "$PWD"

/home/RokeJulianLockhart/Documents/prefixer
├── prefixer.js
└── test.html

1 directory, 2 files

You have it almost spot on!!  The problem is that — the default query is actually pre > code. That means it only affects the text contents of  nested directly under
 elements. A
 on its own won't do.
You can get around this by either adding in the  element yourself (for example
 and
) or by changing the  line to be


✨TinaCMS - Create a Gatsby Starter

CC: @JackDevAU
Description
Create a Gatsby Starter that integrates TinaCMS based on the provided proof of concept (POC) repository at https://github.com/JackDevAU/tinatsby_doc. The starter needs to be cleaned up, a custom build script must be added (since Vercel's automagic build script cannot be used), and documentation should be written and published on tina.io.
Acceptance Criteria

[x] Clean up the POC at https://github.com/JackDevAU/tinatsby_doc
[x] ~Add a custom build script (avoid using Vercel's automagic build script)~
[ ] Write and publish documentation for the Gatsby Starter on tina.io (https://tina.io/docs/frameworks/gatsby)
[ ] Upload code from the PoC to the starter template: https://github.com/tinacms/tina-gatsby-starter.git

Copied from a chat with Hugo

I made a quick update video for you regarding the Gatsby x Tina starter: https://youtu.be/PCOTqD6VJhc
You can see that it works without skipping the client so that's pretty nice! I'd love to get your opinion about this, especially the CORS issue I mention at the end.
Here is the link to the repo, I made it public so you should have access: https://github.com/Freego1783/tina-gatsby-test

Note – if the Gatsby starter we build is actually a viable solution... should we try and get on here (Gatsby plugin library)?

Hey @JackDevAU,

I forked the starter repo and created a PR if you wanna have a look: https://github.com/tinacms/tina-gatsby-starter/pull/1
I also forked the Gatsby page on tina.io and created a PR: https://github.com/tinacms/tina.io/pull/2412

The key point for making Gatsby play nicely with Tina was to force the version of graphql to be the one expected by Tina. To achieve this we used a resolutions property in the package.json file.
After that everything went relatively well, except when we deployed on Vercel we had some problems with the visual editing in admin mode because Tina uses an iframe to display the content and we were experiencing CORS issues. To fix that we had to add a  headers config in gatsby-config.ts, but I think it's a bit too permissive and we should probably make it better.
What do you think?

Hey @JackDevAU
Looks like both PRs are ready to be merged but I cannot do it myself. Can you do it 🙂?


[NPM] Print running NPM version in stdout.

Reason for Change:

This PR is to print running NPM version in stdout which can help diagnose errors or know tested version (e.g., which version is used for testing).
In addition, this PR replace log with klog for better stdout format in npm.go and main.go
Issue Fixed:

Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests

Notes:

maybe with the cli tools we should add a version command

/azp run

maybe with the cli tools we should add a version command

That's a good idea.


Terms Question: Should we be translating React terms such as "props", "state", and "lifecycle"?

I am asking this because, currently I have translated these values and as I read through them, it makes it harder to catch the actual name of it. My suggestion it to keep singular values and add azerbaijani suffixes:

prop and proplar
state and state-li komponent (stateful component)
lifecycle metodları (lifecycle methods)

What do you guys think? In documentation and tutorial, we are already explaining what they are in Azerbaijani using simpler terms. So, I think explaining them in Azerbaijani while making people remember the naming this way would make future conversations easier in React community.

I absolutely agree, such use of these keywords I think will be easier to understand

I have updated the glossary for state and props as I have been experimenting with these words locally while translating and props and state are definitely the right way to go about this. If these words are being used in React's context, we should use the ones shown above. If these words are used in general context, you should use the actual translation, which I have also added to the glossary.


YouTube video description breaks JSON parsing

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2019.02.08. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2019.02.08

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
(bionic)ealgase@localhost:~/USB/YouTubearchives$ youtube-dl -v Od39xwKZ1d0
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'Od39xwKZ1d0']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2<IP_ADDRESS>
[debug] Python version 3.6.7 (CPython) - Linux-3.14.0-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.4, ffprobe 3.4.4, phantomjs 2.1.1
[debug] Proxy map: {}
[youtube] Od39xwKZ1d0: Downloading webpage
WARNING: [youtube] Od39xwKZ1d0: Failed to parse JSON Invalid \escape: line 1 column 98897 (char 98896)
[youtube] Od39xwKZ1d0: Downloading video info webpage
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://r6---sn-vgqsknel.googlevideo.com/videoplayback?ei=wwdmXILjFNDFwQHbnb3gDQ&clen=31890124&source=youtube&fvip=6&lmt=1536792839139822&expire=1550212131&pcm2=no&mime=video%2Fwebm&c=WEB&itag=303&key=yt6&ipbits=0&dur=290.683&sparams=aitags%2Cclen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2%2Cpl%2Crequiressl%2Csource%2Cexpire&requiressl=yes&id=o-AB-PnNf2R0KuhCyoy4ch-flN-NO-kue7Ks5xxMDwFmDW&ms=au%2Crdu&mt=1550190404&mv=m&gir=yes&pl=26&initcwndbps=1913750&signature=5B29175C0D3764D174A456AC0F2C2C678B6BE413.2C6BC4AD0D521C9ECC1D50C2B668F6466A5B6E5E&keepalive=yes&ip=2601%3A400%3Ac200%3Ac6ef%3Af18d%3Aa14f%3Ae313%3Abf36&mn=sn-vgqsknel%2Csn-vgqs7nlr&txp=5432332&mm=31%2C29&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278%2C298%2C299%2C302%2C303&ratebypass=yes'
[download] Destination: Emojis in Linux Terminal-Od39xwKZ1d0.f303.webm
[download] 100% of 30.41MiB in 00:04
[debug] Invoking downloader on 'https://r6---sn-vgqsknel.googlevideo.com/videoplayback?ei=wwdmXILjFNDFwQHbnb3gDQ&clen=4540185&source=youtube&fvip=6&lmt=1536795394737305&expire=1550212131&pcm2=no&mime=audio%2Fwebm&c=WEB&itag=251&ipbits=0&dur=290.701&sparams=clen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2%2Cpl%2Crequiressl%2Csource%2Cexpire&requiressl=yes&id=o-AB-PnNf2R0KuhCyoy4ch-flN-NO-kue7Ks5xxMDwFmDW&ms=au%2Crdu&mt=1550190404&mv=m&gir=yes&pl=26&initcwndbps=1913750&signature=8DF19FD6D4F0D10E05526B21D1DA3FC9C6950379.DA4177550CDF240A3C2A40AD0303C557548F063C&keepalive=yes&ip=2601%3A400%3Ac200%3Ac6ef%3Af18d%3Aa14f%3Ae313%3Abf36&key=yt6&txp=5411222&mm=31%2C29&mn=sn-vgqsknel%2Csn-vgqs7nlr&ratebypass=yes'
[download] Destination: Emojis in Linux Terminal-Od39xwKZ1d0.f251.webm
[download] 100% of 4.33MiB in 00:00
[ffmpeg] Merging formats into "Emojis in Linux Terminal-Od39xwKZ1d0.webm"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:Emojis in Linux Terminal-Od39xwKZ1d0.f303.webm' -i 'file:Emojis in Linux Terminal-Od39xwKZ1d0.f251.webm' -c copy -map 0:v:0 -map 1:a:0 'file:Emojis in Linux Terminal-Od39xwKZ1d0.temp.webm'
Deleting original file Emojis in Linux Terminal-Od39xwKZ1d0.f303.webm (pass -k to keep)
Deleting original file Emojis in Linux Terminal-Od39xwKZ1d0.f251.webm (pass -k to keep)

Description of your issue, suggested solution and other information
I found out the issue that was causing #17940.  The description of the video contains escape characters and breaks the JSON parsing.  While it doesn't cause a failure in simply downloading the video, if it's deep in watch history, it will (as shown in #17940).

I performed some review of this issue. I verified the given case and am able to make a stronger conclusion about the source.
The JSON parsing at question is in extractor/Common.py _parse_json()
After analyzing the json string I can say the following.
-The issue is due to the plaintext unicode escape strings in the video description
-When passed to the parsing function there is an extra escape character in the sequence
-I believe this may be due to an encoding error which is not the fault of youtube-dl
The only resolution to this in my mind would be to manually verify all incoming json for erroneous escape sequences which has performance implications and is not necessary in the majority of cases.
I will continue to check out the issue and see if I can confirm the original source of the encoding error.

Thanks for your help!


Add more wildcard modifiers to filter on characteristics other than the name

Issue #298 was resolved by augmenting the wildcard modifiers as documented here. I recently opened #1013 proposing a path: namespace that would provide commands such as is-dir and is-file. It would also be extremely useful if wildcards could be modified to match specific file types; e.g., **[type:dir] or **[type:file]. If #1013 is implemented the proposed modifiers aren't strictly speaking necessary since they can be written like this instead: each [p]{ if (path:is-file $p) { put $p } } [**]. Nonetheless, I think there is utility is being able to express a few of the more common cases directly in the wildcard. If for no other reason than it is far more efficient to do the filtering during wildcard expansion than after the fact.
Obviously we don't want this to become a mini-DSL that implements all the features of a tool like find. Timestamp comparison, for example, should probably not be implemented via wildcard modifiers. But the equivalent of find -maxdepth n would also be extremely useful and is otherwise expensive and hard to do after the wildcard has been expanded.
In addition to, or as an alternative to, the [type:...] modifier proposed above leverage the os.FileInfo proposal of #1013. Add a global modifier that takes a lambda that is called with a os.FileInfo like object and outputs $true or $false to determine if the path should be included or excluded. That would mostly resolve the performance implications of doing the filtering after the wildcard is expanded. While flexible and relatively efficient it's still very verbose -- especially for common cases such as enumerating just directories or files.

With the change in PR #1018 I can replace the find . -type f in my ff function with **[type:file].

Allowing a lambda as wildcard modifier would be incredibly useful and obviate the need for a complex wildcard DSL. With that feature you could, in theory, write a program to translate find commands into an equivalent Elvish script. At least once the path: module is implemented (see issue #1013). The lambda would take two arguments: the path and a os.File object. It would return

$true to keep the path, or
$false to ignore the path, or
$nil to ignore the path and prune the expansion tree at that node if the path is a directory.


--window-height&width don't work with the otg mode window

[ ] I have read the FAQ.
[ ] I have searched in existing issues.

Environment

OS: Windows
scrcpy version: 1.23
installation method: Windows release
device model: Huawei Matepad 10.4
Android version: 10

Describe the bug
As the window needs to be on top, I expect it to be small.
I used scrcpy --otg --always-on-top --window-width 40 --window-height 40, and tried different values, but the window size doesn't change at all.

The --window-width and --window-height options are currently specific to the mirroring window (i.e. not the OTG window, which is always 256×256).
You could submit a PR.

You could submit a PR.

I don't got the skill...

What should I do now?


Investigate inline styles not getting cleared out

Example: http://spin.atomicobject.com/2014/08/31/nature-matters-insights-on-biomimicry/
This post has inline styling on the image that cause it to overflow the container in Stringer. Loofah is supposed to handle this, so might need fixes upstream.

@swanson is this still an issue?

?Yes - still an issue. Probably not helpful without a reproducible test case - I don't think we can access the RSS feed on that post any more since so much time has passed.

From: Victor Koronen<EMAIL_ADDRESS>Sent: Monday, August 24, 2015 1:19 PM
To: swanson/stringer
Cc: Matthew D. Swanson
Subject: Re: [stringer] Investigate inline styles not getting cleared out (#337)
@swansonhttps://github.com/swanson is this still an issue?

Reply to this email directly or view it on GitHubhttps://github.com/swanson/stringer/issues/337#issuecomment-134306241.

Loofah is supposed to handle this, so might need fixes upstream.

In either case, it should be an issue on Loofah?

?I can get behind that - lets close the issue here.

From: Victor Koronen<EMAIL_ADDRESS>Sent: Monday, August 24, 2015 1:56 PM
To: swanson/stringer
Cc: Matthew D. Swanson
Subject: Re: [stringer] Investigate inline styles not getting cleared out (#337)
Loofah is supposed to handle this, so might need fixes upstream.
In either case, it should be an issue on Loofah?

Reply to this email directly or view it on GitHubhttps://github.com/swanson/stringer/issues/337#issuecomment-134317509.


df.to_csv drops index column from CSV entirely

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Modin version (modin.__version__):
Python version:
Code we can use to reproduce:

import modin.pandas as pd
cols = 100
arows = 20000
brows = 20000
keyrange = 100
a = pd.DataFrame(np.vstack([np.random.choice(keyrange,size=(arows)),np.random.normal(size=(cols,arows))]).transpose(),columns = ['key'] + ['avalue' + str(i) for i in range(1,1+cols)]).set_index('key')
b = pd.DataFrame(np.vstack([np.random.choice(keyrange,size=(brows)),np.random.normal(size=(cols,brows))]).transpose(),columns = ['key'] + ['bvalue' + str(i) for i in range(1,1+cols)]).set_index('key')
a.to_csv("a-big.csv")
b.to_csv("b-big.csv")
pd.read_csv("a-big.csv") # Has no key column
pd.read_csv("a-big.csv").set_index("key") # Errors

When using modin to write a Dataframe with a custom index to a csv file, the index is dropped entirely, and a range index is used when reading the Dataframe. Additionally, this may have been a transient error, but when I wrote the df using pandas, and then read it using modin, and called set_index('key') where 'key' was the original index, the output displayed correctly, but I got the following KeyError:
2022-01-20 13:23:49,267	ERROR worker.py:79 -- Unhandled error (suppress with RAY_IGNORE_UNHANDLED_ERRORS=1): ray::deploy_ray_func() (pid=78818, ip=<IP_ADDRESS>)
  File "/Users/rehandurrani/Documents/modin/modin/core/execution/ray/implementations/pandas_on_ray/partitioning/axis_partition.py", line 211, in deploy_ray_func
    result = func(*args)
  File "/Users/rehandurrani/Documents/modin/modin/core/dataframe/pandas/partitioning/axis_partition.py", line 160, in deploy_axis_func
    result = func(dataframe, **kwargs)
  File "/Users/rehandurrani/Documents/modin/modin/core/dataframe/pandas/dataframe/dataframe.py", line 1310, in _tree_reduce_func
    series_result = func(df, *args, **kwargs)
  File "/Users/rehandurrani/Documents/modin/modin/core/storage_formats/pandas/query_compiler.py", line 441, in map_func
    return pandas.merge(left, right, **kwargs)
  File "/Users/rehandurrani/.miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/reshape/merge.py", line 106, in merge
    op = _MergeOperation(
  File "/Users/rehandurrani/.miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/reshape/merge.py", line 699, in __init__
    ) = self._get_merge_keys()
  File "/Users/rehandurrani/.miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/reshape/merge.py", line 1096, in _get_merge_keys
    right_keys.append(right._get_label_or_level_values(rk))
  File "/Users/rehandurrani/.miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/generic.py", line 1779, in _get_label_or_level_values
    raise KeyError(key)
KeyError: 'key'

The code for the replication of this bug was borrowed from marsupialtail's question on the modin discourse.

Describe the problem

Source code / logs

Same problem reproduced for Dask engine, so the problem is not in the distributed version of to_csv that implemented for Ray.

The problem is not reproducible for 0.12.1 release, hence appeared later.

Found a commit that introduces the regression -https://github.com/modin-project/modin/commit/3c740dbfcdd69ddc3ab45a42be996e5c61104342.
@naren-ponder can you look at the problem?

I have put a fix in #4025 .

Reviewing #4025


Add the @ operator

This is related to the usage of Numpy and Pandas from Jinja, as stated in https://github.com/pallets/jinja/issues/1285.
Since Python 3.5 and the PEP 465, the @ operator has been introduced for matrix multiplication.
The documentation shows a use example:
>>> import numpy

>>> x = numpy.ones(3)
>>> x
array([ 1., 1., 1.])

>>> m = numpy.eye(3)
>>> m
array([[ 1., 0., 0.],
       [ 0., 1., 0.],
       [ 0., 0., 1.]])

>>> x @ m
array([ 1., 1., 1.])

Could it be added to the Jinja standard?

As in #788, you can inject operator.matmul into the globals to allow using this operator in Jinja without any syntax changes. I'm not comfortable changing the syntax of Jinja to support more operators at this time. See https://github.com/pallets/jinja/issues/1285#issuecomment-684900406 for a longer explanation.


APIPipeline's name now defaults to None

If no name is provided, name Pipeline {uid} will be used.


Revert to Go 1.18 in go.mod and make policy around this official

What does this PR implement/change/remove?
We don't seem to have an official strategy for the Go version in go.mod. I believe that, as a library, we should only bump this version if there are dependencies we use that would require us to move to this version (ref). I don't believe that we have any dependencies that warrant a bump. Bumping this version will require user to update to this version. While it is a good practice to build with the latest version of Go, this requirement will be making a decision for users that should be theirs and not ours. We are a library not a binary to be built. In this aspect, we are using the version in go.mod incorrectly, in my opinion, by doing this. This assumes we don't have a known use case where bumping this version if needed. I don't know of any at the moment. If there are any we should bring them up.
Apologies, i know i approved the PR to update this. The affects of this didn't dawn on me then.
The other side of the coin
There can be reasons to bump this version. I think before we bump the version we should have use cases the bump and more carefully consider the affects.
The go line must be greater than or equal to the go line of all dependencies.
The go command no longer attempts to maintain compatibility with the previous older version of Go.
The go command is more careful about keeping checksums of go.mod files in the go.sum file." - ref
Checklist

[ ] Tests added
[ ] Similar commits squashed

The HW vendor this change applies to (if applicable)
The HW model number, product name this change applies to (if applicable)
The BMC firmware and/or BIOS versions that this change applies to (if applicable)
What version of tooling - vendor specific or opensource does this change depend on (if applicable)
Description for changelog/release notes

@jacobweinstock based on the Go release policy, we'd want to be on a release that is not at its EOL?

Each major Go release is supported until there are two newer major releases
ref: https://go.dev/doc/devel/release#policy

Note that is the minimum version supported.
By setting it to 1.18, we don't prevent people from using latest version.
But by setting it to 1.22 (or oldest not EOL), we just force people to upgrade without technical justification other than "keep your go secure" (which is not this project's mission).
I see several options here: we could stay on the smallest go version of dependencies, or follow the base go version available for some platforms (e.g. min of go version available in oldest supported debian/redhat/ubuntu)

I agree with the OP: libraries should honor the definition of the go directive and specify what they need (apps probably should also as of 1.21 which introduces toolchain).

I see several options here: we could stay on the smallest go version of dependencies, or follow the base go version available for some platforms (e.g. min of go version available in oldest supported debian/redhat/ubuntu)

I think the only reason to care about supported platforms is in implementation detail because you wouldn't want to use a language feature > than the oldest supported platform.
Assuming there aren't any language feature issues I would stick with the definition of the go directive and select the oldest version that can be used for the language features leveraged.

Those are fair points, thanks for raising them. If we can get a rebase on main here then this can be merged.


Obtaining subscription-level deployment state is slow

Background
When azd attempts to obtain deployment state, it needs to find the ARM deployment that is linked to the environment. On our Playground subscription (~660 deployments), this is observed to take about 30 seconds.
The deployment state discovery involves:

List all deployments in the current subscription
Filter to the deployment based on environment tag or name

The listing is currently observed to be really slow.

I don't think so, that API needs to know the deployment name, which we don't know


Automate collection of various metrics

The MetPy team has scripts to collect metrics on GitHub usage that we can adapt and deploy across our various repos.
There are also BinderHub metrics that can be harvested from Jetstream.
We should automate the collection of whatever "useful" metrics we can.
cc:
@dopplershift
@ktyle

Suggested path forward:

Identify what kind of metrics we would like to collect.
Determine which of the above can be reasonably automated
Implement metrics collection

N.B. I supposed the order of (1) and (2) could be swapped :-)

It looks like Google removed its easy website tracking. Every tutorial I follow says to click options that no longer exist "tracking -> website tracking" in the settings. And links go to "Service Unavailable."
It looks like they now want us to use another service called Google tag manager https://marketingplatform.google.com/about/tag-manager/

Here are a some of the questions around metrics that I think would be useful to be able to answer:

What content is accessed most (least)?
How much binder activity do we have?
How much more (or less) is binder used than downloading the content to run locally?
How many users do we have?
What part of the world uses our content most?

For all of these it would be great to be able to plot trends over time.
Obviously, not all of these may be practical to capture. But it is a starting point :-)

Using the embedded API it looks like people would have to log in on the API to view the metrics, so this might not be the path we want to pursue.
I'm looking into making a GH action that downloads our Google analytics nightly. I'll have to pass in my log in information as secrets. Then once I have that figured out, we can make the action create a new plot nightly of the data?

Getting the raw data should be step one. If there is a way to embed a plot in our website  that might be really cool, but certainly isn't a must do. Simply displaying as text, for example, the total number of users would satisfy our NSF obligations. Having access to the raw data would allows us to analyze it anyway we want.

That may be a question for @dopplershift

Ooof, not sure there. When I did this for MetPy I had sufficient permissions I guess.

You may have to turn to the interweb for this one, @jukent . That's my favorite useless message: contact your administrator :-)

Maybe this resource would be helpful. Apparently it is being used for Intake

Thanks @clyne Looking into that now.

@jukent I got that email and I think I clicked the right buttons to grant you admin privileges.

Hmmm when I go to the relevant page I still see Julia has having just Viewer privs?

Try now! I made one more change.

The page looks different now! Yay!

![Screen Shot 2023-11-06 at 11 00 20 AM](https://github.com/ProjectPythia/projectpythia.github.io/ass

ets/46687291/bd1d11f9-1e40-456c-be10-4a22e4b076ca)
![Screen Shot 2023-11-06 at 11 00 46 AM](https://github.c

om/ProjectPythia/projectpythia.github.io/assets/46687291/ab8a05eb-5f85-4878-b18e-fc8f8178fc8d)
Every avenue I try I still encounter permissions errors. I'm not sure what I'm lacking with admin privileges on the analytics page.
I need to pass the baton on this.

An update on this:
Google permissions are a maze, so while I had admin permissions to the domain I did not have them to the metrics. Joel Daves helped me with this and it looks clear (if still sometimes confusing) the steps forward.
Made some progress in enabling the API and creating the service account to interact with the metrics, and then had to table this for AGU prep. I was hoping to focus on it this week but there was the hiccup of our Google domain expiring and having to switch to AWS. While I was not 100% confident that our metrics would be unaffected by this, I did not want to focus on this the last two days. The domain transfer will be complete in the next 6 hours, but it looks save to continue work.
Remaining steps according to this guide are to

Install the client library (this will want to be done on GitHub somewhere but I'm not 100% sure. For starters I will work on this locally to get a feel for it)
Set up and test a sample
Troubleshoot

So it looks like we're close to making progress and then having a more serious conversation about what code we want to write to grab what metrics.

After getting brew and the api installed, I set up the sample and edited the required fields except "VIEW_ID". Which I should be able to find at the Account Explorer. But for ProjectPythia, it says no UA views for this account. So I'll have to learn more about what UA views are as the next step before setting one up.

After getting brew and the api installed, I set up the sample and edited the required fields except "VIEW_ID". Which I should be able to find at the Account Explorer. But for ProjectPythia, it says no UA views for this account. So I'll have to learn more about what UA views are as the next step before setting one up.
Notes on how to make a new UA view

UA (Universal Analytics) is their older generation measurement solution that was replaced by the Google Analytics 4 [GA4]. That said, I am not sure if Account Explorer is the best solution to do this because Pythia's account already uses the newer generation [GA4] properties.

and it seems to be straightforward from this quick start using client libraries as long as you have permissions to enable an API with your Google account (I don't seem to have)

Interesting thanks for sharing that info before I went too far down this path! The API you linked don't seem to work for me, they say "Service Unavailable." Same with the quick start link.

I got the same errors when I clicked on them (or even your API links from a while ago), but they should work if you copy their link and paste in a new tab and go

Just an update here -- the analytics ap4 quick start guide I was following was deprecated at the end of January in favor of analytics v1. I thought AP4 would match up with our GA4 product, but that is just a naming coincidence and they no longer work together.
On the new service, I do not have permissions to enable it -- so I'll have to email some people again.

Update as of 3/11, opened a draft PR #407 that explains the steps I took there. The action is failing on my fork though because it says it can't find a file. I'm not sure if this is related to fork vs upstream issues. I can't look at this again until Friday because of a hackathon all week.
Essentially,

created necessary secrets for accessing API (didn't create secrets for creating a PR yet)
created a Python script that accesses API and calculates total users for Portal, Foundations, and Cookbooks, stores these in a json
created a Python script that displays metrics in a md file
added .md file to footer and index
created a GH action file to call these steps

I got the automate-metrics issue to pass when using workflow_dispatch call (you can see that here. But then I had to change it to on workflow_call to let the nightly_build action call it. Now the secrets don't seem to be passed in as expected and it is failing again. I think this is a lot of progress though.

Check it out!
https://projectpythia.org/metrics.html


BS Scraper does not interpret current_isolated correctly

BS publishes active cases including people in hospital, whereas current_isolated should not contain people in hospital. Thus the scraper needs to substract people in hospital from the number of isolated. How to proceed on days when current_hosp are not communicated? I have no idea yet.
See today's text from BS: https://www.gd.bs.ch//nm/2020-tagesbulletin-coronavirus-982-bestaetigte-faelle-im-kanton-basel-stadt-gd.html:
"Die Zahl der 982 Infektionen setzt sich zusammen aus 924 genesenen Personen (-), 50 Todesfällen (-) und acht aktiven Fällen (+ 3). Von den aktiven Fällen befinden sich drei Personen in Spitalpflege. "
I have manually fixed today's numbers here: https://github.com/openZH/covid_19/commit/e6c3275b205a9be3a132974d11e50224fc7baf66
...but the scraper has overwritten my correction again here: https://github.com/openZH/covid_19/commit/8d7f8d1629868ec12350c55766f0ca588274ca2a
So the scraper actively produces wrong numbers at the moment. Thanks for any help!

@jb3-2 quick fix: we'll disable the scraper for BS until we have a bugfix. Ok?

OK thank you!

#802

Thanks @dominikgehl! After merging #802, I re-enabled the BS scraper.

Thank you @dominikgehl @metaodi!


Pre processing policies

In this approach we prepare plan for given policy during cluster::refreshTopology, then Iter() is costless.

Without this patch I'm getting consistently in 10M mixed
2022/05/27 12:26:17 Finished
Benchmark time: 11771 ms

Closing in favor of #247


Roll 4.4.2 to 4.4.3.

Ongoing master (preview) versions should have a higher base version number than what has been released.
4.4.2 has been released.

Please update it thru npm version so it update both package.json and package-lock.json.
Done.


[SPARK-40601][PYTHON] Assert identical key size when cogrouping groups

Cogrouping two grouped DataFrames in PySpark that have different group key cardinalities raises an error that is not very descriptive:
left.groupby("id", "k")
    .cogroup(right.groupby("id"))

py4j.protocol.Py4JJavaError: An error occurred while calling o726.collectToPython.
: java.lang.IndexOutOfBoundsException: 1
	at scala.collection.mutable.ResizableArray.apply(ResizableArray.scala:46)
	at scala.collection.mutable.ResizableArray.apply$(ResizableArray.scala:45)
	at scala.collection.mutable.ArrayBuffer.apply(ArrayBuffer.scala:49)
	at org.apache.spark.sql.catalyst.plans.physical.HashShuffleSpec.$anonfun$createPartitioning$5(partitioning.scala:650)
...
org.apache.spark.sql.execution.exchange.EnsureRequirements.$anonfun$ensureDistributionAndOrdering$14(EnsureRequirements.scala:159)

What changes were proposed in this pull request?
Assert identical size of groupby keys and provide a meaningful error on cogroup.
Why are the changes needed?
The error does not provide information on how to solve the problem.
Does this PR introduce any user-facing change?
Yes, raises an AssertionError: group keys must have same size instead.
How was this patch tested?
Adds test test_different_group_key_cardinality to pyspark.sql.tests.test_pandas_cogrouped_map.

Ideally, EnsureRequirements should not call into HashShuffleSpec.createPartitioning(clustering) with a clustering that has an incompatible cardinality.

@HyukjinKwon @cloud-fan

Can one of the admins verify this patch?

@HyukjinKwon @cloud-fan

also cc @zhengruifeng

since the validation was moved into the SQL side, we may also move the UT into src/test/scala/org/apache/spark/sql/DataFrameSelfJoinSuite.scala

since the validation was moved into the SQL side, we may also move the UT into src/test/scala/org/apache/spark/sql/DataFrameSelfJoinSuite.scala

I am not sure about DataFrameSelfJoinSuite to be the best place for this. Though it is the only place where flatMapCoGroupsInPandas is currently tested, that test targets at self join ambiguity, hence the DataFrameSelfJoinSuite. This test is about keys. I'd rather put it into DatasetSuite, which has lots of groupBy tests.
I would still keep the one test in Python as Scala tests a private method (there is no other way in Scala to evoke the error), while Python tests a public API.

Merged to master


Create webapp dir, initialize pnpm workspaces, nuxt3 apps.

Will add cypress examples with ticket #17568

Will add cypress examples with ticket #17568

Can you add other tickets for header / footer as well?

Will add cypress examples with ticket #17568

Can you add other tickets for header / footer as well if we don't have those?

#17800
#17801
#17802


Fix ResizeBasedService trim warning

Description
@henon @mikes-gh
Fixes trim warring
'TValue' generic argument does not satisfy 'DynamicallyAccessedMemberTypes.PublicMethods' in 'Microsoft.JSInterop.DotNetObjectReference<TValue>'. The generic parameter 'TSelf' of 'MudBlazor.Services.ResizeBasedService<TSelf, TInfo, TAction, TaskOption>' does not have matching annotations. The source value must declare at least the same requirements as those declared on the target location it is assigned to.

Did a small refactoring:

renamed TaskOption -> TTaskOption because it was hiding enum that also has TaskOption name
removed unnecessary $
== replaced with is as it's more recommended
_subscriptions can be readonly, also for some reason it was initialized twice in the constructor

How Has This Been Tested?

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[x] The PR is submitted to the correct branch (dev).
[x] My code follows the code style of this project.
[ ] I've added relevant tests.

@ScarletKuro Great :-) Thanks a lot


feat: use GeoStylerContext composition for RuleCard and RuleOverview

Description
This makes use of the GeoStylerContext for <RuleCard> and <RuleOverview>.
BREAKING CHANGE: This removes the properties showAmount and showDuplicates from RuleCard. Please use
GeoStylerContext.composition.Rule.amount and
GeoStylerContext.composition.Rule.duplicate instead.

Do you introduce a breaking change?

[x] Yes
[ ] No
[ ] I am unsure (no worries, we'll find out)

:tada: This PR is included in version 13.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Help me to remove data-vue-tag="ssr"

Dear Gridsome Team,
I have used Gridsome and feel very happy.
I have a small question. My client for SEO look at html we generated use Gridsome and see that:
<meta data-vue-tag="ssr" name="og:url" content="https://*********/" />

They want to change into ↓
<meta name="og:url" content="https://*********/" />

(They do not want data-vue-tag="ssr"  appear in meta tag).
I look at document but could not found any information related how to customize this request.
Please help me to improve this issue.
Many thanks !!!

Hi, @thuyteq. The data-* attributes are necessary for vue-meta (which Gridsome uses to handle tags in the head section) to function correctly and cannot be removed, unfortunately. I don't think they should matter for SEO purposes, but if you have to remove them, then you must add the tags manually instead of using metaInfo.

Question answered by https://github.com/gridsome/gridsome/issues/813#issuecomment-555914305


License not on Marketplace

The license is not on the marketplace (see https://marketplace.visualstudio.com/items/dbaeumer.vscode-eslint/license).
To fix this, simply republish, @joaomoreno fixed this in vsce with https://github.com/Microsoft/vscode-vsce/issues/66.

Republished 0.10.13 yesterday.