[Bug] __tmp_not_partitioned CTAS inherited model config "external_location" and causing clashing with the model data table

Is this a new bug in dbt-athena?

[X] I believe this is a new bug in dbt-athena
[X] I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior
I have a model that writes out to more than 100 hive partitions.
I recently dropped the table and attempted to recreate via dbt build. I have an external_location in configuration as I was migrated onto dbt and want to avoid rebuild data / move s3 location.
The unexpected behavior is dbt build succeeded but my s3 path does not contain any data.
dbt / adaptor version:
"app": "dbt", "dbt_version": "1.7.17"
Registered adapter: athena=1.7.2

The CTAS generated:
On model.A.XX: -- /* {"app": "dbt", "dbt_version": "1.7.17", "profile_name": "A", "target_name": "prod", "node_id": "model.A.XX"} */

    create table "awsdatacatalog"."{schema}"."XX__tmp_not_partitioned"
    with (
      table_type='hive',
      is_external=true,external_location= {config.external_location}
      format='orc'
    )
    as
      SELECT

Expected Behavior
This XX__tmp_not_partitioned should not use the configured external path which is clashing with the actual model. Instead it should either use a unique tmp location or same as __dbt_tmp table which goes to s3_data_dir/schema/table__dbt_tmp
Steps To Reproduce
create a partitioned model with more than 100 hive partitions, with an external_location in config.
config:
    enabled: true
    external_location:some_p3_path

If this table exists in glue, drop it.
Run dbt run to create more than 100 partitions in one sql run.
Environment
- OS: ubuntu 22.04.4 jammy
- Python: 3.10
- dbt: 1.7.17
- dbt-athena-community: 1.7.2

Additional Context
I am happy to contribute to get this fixed if that helps speed things up.
https://github.com/dbt-athena/dbt-athena/blob/main/dbt/include/athena/macros/materializations/models/table/create_table_as.sql

@u-ra-ra-ra feel free to propose a pr and we will be happy to review it.

I will hopefully have some time next week to give it shot.

At my work, I'm experiencing the same issue, and I found that one solution is to set temporary to true on the following line in the file: dbt-athena/dbt/include/athena/macros/materializations/models/table/create_table_as.sql
line 165: change from temporary to true
{%- do run_query(create_table_as(true, tmp_relation, compiled_code, language, true)) -%}

I hope this helps!

@lucastrubiano thanks for spotting this issue. Do you mind to raise a PR with your fix? Thanks

@nicor88 Glad to push the changes, but I don’t have permissions to create a branch or PR. Any guidance on how to proceed?

@lucastrubiano
I recommend to have a look at the contributing section.
Long story short, if you are not a project maintainer you cannot push directly to dbt-athena/dbt-athena.
You have to fork the repository, and then propose a PR from your fork, pretty much here you can find a detailed guide on how to do it.


How to start KDE Authorization Service?

I need help starting a service. Applications such as Muon Package manager use the polkit-kde-1 service for authorization. I think I need to manually start this service using the rc.local script. But I don't know what command to put in the file for it to start the service.
Any help would be appreciated!

@Windoze98 Please see: http://packages.ubuntu.com/trusty/amd64/polkit-kde-1/filelist
The binary is /usr/lib/kde4/libexec/polkit-kde-authentication-agent-1.
Regards.

Unfortunatly, inputting that into rc.local still didn't fix my authorization errors. Whenever I use Muon Updater or Muon Package Manager, I keep getting the error:
The operation cannot continue since proper authorization was not provided.

@Windoze98 Although I really don't know K desktop, let us know the 2 outputs of:
ps aux | grep dbus and ps aux | grep polkit.
In Gnome, polkit-agent should be launched as "User's process" instead of "Root's process".
Regards.

For mere mortals using Crouton -- which is amazing -- but, if I wanted to start with a less striped down Kubuntu version - is that possible w/ another target that is out there?
So things like authorization and Java would be able to be installed easier (vs. turning on the level of software apt-get has access to...)
For hours I have been trying to get Muon (installed separately) to have proper authorizations, or just installing GEDIT or NANO, again hassle without authorizations...

My original chroot was via...
~/Downloads/crouton -t kde
Did that install Trusty or Precise? is there a version that comes a little more ready to roll? but isn't full blown Kubuntu Desktop...

I'm having the same problem.  I've tried launching muon using kdesudo

kdesudo muon-installer

A graphical window opens and requests my root password.  In Konsole, the following appears:

Unable to load library icui18n "Cannot load library icui18n: (libicui18n.so.cannot open shared object file: No such file or directory)"
kdesudo(29997) KDESu::KDESuPrivate::KCookie::getXCookie: No X authenticationo set for display  ":1"

QDBusConnection: session D-Bus connection created before QCoreApplication. Acation may misbehave.
Unable to load library icui18n "Cannot load library icui18n: (libicui18n.so.cannot open shared object file: No such file or directory)"
QDBusConnection: session D-Bus connection created before QCoreApplication. Acation may misbehave.

The same happens when trying to launch the package manager, kdesudo muon.
Any ideas? I've been googling around for ways to install the library refered to in the Terminal message, but not really getting anywhere.

Did anyone find a fix to this? I'm really enjoying the KDE experience in Chromebook, but this authorisation issue is a real pain.


Custom Query string

How can i do custom query like :
@Query('SELECT * FROM MyEntity WHERE :queryString')
List filterMyEntity(String queryString)

Hi! Floor currently only supports binding parameters but not bigger query segments. Can you describe your use case a bit more?

for example at the current version parameter null throw exception and i force to do my query:
String query =
'code = $code'
'${cityId != null ? ' AND cityId = $cityId' : ''}'
'${provinceId != null ? ' AND provinceId = $provinceId' : ''}'
'${agencyName != null && agencyName.isNotEmpty ? ' AND name LIKE %$agencyName%' : ''}'
'${typeCode != null ? ' AND type = $typeCode' : ''}';
List result =
await _repository.filterPartnerLocation(query);

@Query('SELECT * FROM PartnerLocationEntity WHERE :query')
Future<List> filterPartnerLocation(String query);

I think this is high important feature for your library to supprot "rawQuery",
for example if i want to create a search filtter (like a sample which i provided above) need to write 16 functioin in my dao in normal way (becasue of 4 parameter take 4^2 state)

+1
This feature will be very cool !
That will be awesome to build a custom query in real time (directly when the app is running), and then execute it.

Such a feature would be pretty cool, indeed. What are your thoughts about the approach Room takes with raw queries? https://developer.android.com/reference/androidx/room/RawQuery

I like it. But just FYI, as it stands, you can completely write custom query/raw sql as long as it returns one row. Just add this to your DAO, similar to what I wrote above.
Write any query
  @Query('\$q --:q')
  Future<Entity> findByQuery(String q);

Usage example
globals.mydb.myDao.findByQuery("select * from entity").then((value) => print(value.entityfield));

You can even manipulate the results into the model fields select  'Custom Name' as name (no from or anything) which will map to name field.

Do not forget to allow to construct your own sql query string.
Like in a multi fields search, you should construct your where clause based on inputs.
With annotations, this is not possible.
that's why I think room approach is better. (and clean)

Do not forget to allow to construct your own sql query string.
Like in a multi fields search, you should construct your where clause based on inputs.
With annotations, this is not possible.
that's why I think room approach is better. (and clean)

@salvaterra I am trying to implement a BaseDao based on your example but am not able to get it to work. Do you know how I could do something like this? Here is my code:
@Query('\$q --:q')
  Future<T> findByQuery(String q);

  Future<T> getById(String id) {
    return findByQuery("SELECT * FROM ${T.toString()} WHERE id = \'$id\'");
  }

userDao.getById("UUID").then((value) => print(value.entityfield));

At runtime it throws [ERROR:flutter/lib/ui/ui_dart_state.cc(177)] Unhandled Exception: DatabaseException(Cannot bind argument at index 1 because the index is out of range.  The statement has 0 parameters.) sql 'SELECT * FROM User WHERE id = 'UUID'; --?' args [SELECT * FROM User WHERE id = 'UUID']}

@salvaterra I am trying to implement a BaseDao based on your example but am not able to get it to work. Do you know how I could do something like this? Here is my code:
@Query('\$q --:q')
  Future<T> findByQuery(String q);

  Future<T> getById(String id) {
    return findByQuery("SELECT * FROM ${T.toString()} WHERE id = \'$id\'");
  }

userDao.getById("UUID").then((value) => print(value.entityfield));

At runtime it throws [ERROR:flutter/lib/ui/ui_dart_state.cc(177)] Unhandled Exception: DatabaseException(Cannot bind argument at index 1 because the index is out of range.  The statement has 0 parameters.) sql 'SELECT * FROM User WHERE id = 'UUID'; --?' args [SELECT * FROM User WHERE id = 'UUID']}

Should work, I just tested it. Look at the generated code and how you are inheriting the Base dao.
abstract class BaseDao<T> { ...}

@dao
abstract class BookDao extends BaseDao<Book> {...}

Generated code
@override
  Future<Book> getByQuery(String q) async {
    return _queryAdapter.query('$q --?',
        arguments: <dynamic>[q],
...

Should work, I just tested it. Look at the generated code and how you are inheriting the Base dao.
abstract class BaseDao<T> { ...}

@dao
abstract class BookDao extends BaseDao<Book> {...}

Generated code
@override
  Future<Book> getByQuery(String q) async {
    return _queryAdapter.query('$q --?',
        arguments: <dynamic>[q],
...

@salvaterra thanks for getting back to me! I am able to generate the code fine and my project compiles. My UserDao looks like this:
@dao
abstract class UserDao extends BaseDao<User> {}

However at runtime when I go to use userDao.getById('UUID'); it always throws this exception:
[ERROR:flutter/lib/ui/ui_dart_state.cc(177)] Unhandled Exception: DatabaseException(Cannot bind argument at index 1 because the index is out of range. The statement has 0 parameters.) sql 'SELECT * FROM User WHERE id = 'UUID'; --?' args [SELECT * FROM User WHERE id = 'UUID']}

@salvaterra thanks for getting back to me! I am able to generate the code fine and my project compiles. My UserDao looks like this:
@dao
abstract class UserDao extends BaseDao<User> {}

However at runtime when I go to use userDao.getById('UUID'); it always throws this exception:
[ERROR:flutter/lib/ui/ui_dart_state.cc(177)] Unhandled Exception: DatabaseException(Cannot bind argument at index 1 because the index is out of range. The statement has 0 parameters.) sql 'SELECT * FROM User WHERE id = 'UUID'; --?' args [SELECT * FROM User WHERE id = 'UUID']}

Is it iOS or Android? I tested on iOS, could it be the implementation of SQL in Android?

Is it iOS or Android? I tested on iOS, could it be the implementation of SQL in Android?

Oh interesting, I am testing on the Android emulator. I'll see if I get different results on iOS

Oh interesting, I am testing on the Android emulator. I'll see if I get different results on iOS

Yup, that's it. Only works on iOS :(

Yup, that's it. Only works on iOS :(

I'm out of hacking alternatives :) I guess we would need to implement the proper way. Maybe a new @queryRaw tag that doesn't do the binding. Should be easy, just comment the arguments attribute.

I'm out of hacking alternatives :) I guess we would need to implement the proper way. Maybe a new @queryRaw tag that doesn't do the binding. Should be easy, just comment the arguments attribute.

Hey guys,
Looks like the 'RawQuery' feature is still actual. I've started to work on it and plan to implement the same interface as Android Room native. Does anybody have any additional expectations that I should pay attention to?

Here is a draft PR. I've verified it briefly, it's working ok for me, so you can try to switch to my branch until it will be merged to the master branch.
Requires some refactoring and increase test coverage.

I like it. But just FYI, as it stands, you can completely write custom query/raw sql. Just add this to your DAO, similar to what I wrote above.

I just want to add that you should not rely on this behaviour. This is more of a bug than a feature and it might no longer work after #531 is merged. But I agree that rawQuery is absolutely necessary to fill those kinds of Gaps.

Well, it's been so long, but I almost finished rawQuery feature 😁

Hi, can you tell when this feature will be available? floor is awesome btw, I just started working with it.

Hi, can you tell when this feature will be available? floor is awesome btw, I just started working with it.

https://github.com/pinchbv/floor/pull/517

Hi, can you tell when this feature will be available? floor is awesome btw, I just started working with it.

I stopped developing it because there is an option allows you access to performing raw requests.
You can get access to sqflite.DatabaseExecutor if you declare your instance of database as a global variable then you can call :

youDatabaseInstance.database.rawQuery
youDatabaseInstance.database.rawDelete
youDatabaseInstance.database.rawInsert
and so on.

Thanks!


Write citations output to file

Related to https://github.com/allenai/scholar/issues/28994.
As part of getting citations info from scholarphi piped through the s2airs system, we want to get citations info into the annotation store. We can adapt the wrapper system (scholarphi-pipeline) to do this, if it has access to the output that the inner scholarphi code produces.
The simplest way that I found to do this while I was working on https://github.com/allenai/scholar/issues/28297 was to just dump the output as json into a file in a location provided by the caller.
This PR adapts the DatabaseUploadCommand a little, making it more of a 'save output' kind of thing (I haven't renamed it to try to keep this PR small, and avoid changing more than is necessary). The idea is the user can specify whether they want output saved to a db (what we've been doing up until now), saved to a file (what we want for s2airs), or both. To begin with though, only the citations extension of DatabaseUploadCommand will know how to actually save to a file. Other commands extending DatabaseUploadCommand should error out if the user tries to specify that they want output saved to a file for those. To get an idea of how we could extend this to all entities, check out this branch: https://github.com/allenai/scholarphi/compare/chi-2021-demo...chloea-try-producing-json-take-1 (note: it does have some extra stuff around pulling arxiv pdfs that isn't relevant to this, and the saving to file stuff is less neat, and the 'both' option doesn't exist, but hopefully it gets the idea around extending to other entities across).
The plan is that the existing scholarphi-pipeline system would use the 'both' option when processing in citations only mode, so that we can continue to support the current reader, and so that we can put stuff into the annotation store as we build out s2airs.
Testing done:
I added some tests and ran pytest:
# pytest --all
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.7.5, pytest-5.3.1, py-1.10.0, pluggy-0.13.1
rootdir: /data-processing, inifile: pytest.ini, testpaths: tests
plugins: cov-2.5.1
collected 214 items

tests/test_bounding_box.py ..................                                                                                [  8%]
tests/test_colorize_sentences.py .....                                                                                [ 10%]
tests/test_colorize_tex.py ........                                                                                [ 14%]
tests/test_compile.py ........                                                                                [ 18%]
tests/test_extract_definitions.py .................ssss                                                                                [ 28%]
tests/test_locate_symbols.py ...                                                                                [ 29%]
tests/test_match_symbols.py ......                                                                                [ 32%]
tests/test_normalize_tex.py ..............                                                                                [ 38%]
tests/test_parse_equation.py .........................                                                                                [ 50%]
tests/test_parse_tex.py ..............................................                                                                                [ 71%]
tests/test_sanitize_equation.py ....                                                                                [ 73%]
tests/test_scan_tex.py .....                                                                                [ 76%]
tests/test_string.py .............                                                                                [ 82%]
tests/test_unpack.py ....                                                                                [ 84%]
tests/test_visual_validate.py ......                                                                                [ 86%]
tests/common/test_fetch_arxiv.py ......                                                                                [ 89%]
tests/common/test_upload_entities.py ...                                                                                [ 91%]
tests/common/commands/test_database.py .........                                                                                [ 95%]
tests/common/commands/test_fetch_arxiv_sources.py ..                                                                                [ 96%]
tests/common/commands/test_fetch_s2_data.py ........                                                                                [100%]

============================================================================================================== warnings summary ==============================================================================================================
/usr/local/lib/python3.7/dist-packages/wandb/util.py:37
/usr/local/lib/python3.7/dist-packages/wandb/util.py:37
  /usr/local/lib/python3.7/dist-packages/wandb/util.py:37: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    from collections import namedtuple, Mapping, Sequence

/usr/local/lib/python3.7/dist-packages/wandb/vendor/graphql-core-1.1/graphql/type/directives.py:55
  /usr/local/lib/python3.7/dist-packages/wandb/vendor/graphql-core-1.1/graphql/type/directives.py:55: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
    assert isinstance(locations, collections.Iterable), 'Must provide locations for directive.'

entities/definitions/commands/tokenize_sentences.py:111
  /data-processing/entities/definitions/commands/tokenize_sentences.py:111: DeprecationWarning: invalid escape sequence \s
    "EQUATION_DEPTH_0_START\s*(.*?)\s*EQUATION_DEPTH_0_END",

entities/glossary_terms/colorize.py:24
  /data-processing/entities/glossary_terms/colorize.py:24: DeprecationWarning: invalid escape sequence \S
    first_nonspace = re.search("\S", term.tex)

entities/glossary_terms/colorize.py:32
  /data-processing/entities/glossary_terms/colorize.py:32: DeprecationWarning: invalid escape sequence \S
    last_nonspace = re.search("\S(?=\s*$)", term.tex)

tests/test_extract_definitions.py::test_model_extracts_simple_definitions
tests/test_extract_definitions.py::test_model_extracts_simple_definitions
tests/test_extract_definitions.py::test_model_extracts_simple_definitions
tests/test_extract_definitions.py::test_model_extracts_simple_definitions
tests/test_extract_definitions.py::test_model_extracts_simple_definitions
tests/test_extract_definitions.py::test_model_extracts_nickname_before_symbol
tests/test_extract_definitions.py::test_model_extracts_nickname_before_symbol
tests/test_extract_definitions.py::test_model_extracts_nickname_symbol_filter
tests/test_extract_definitions.py::test_model_extracts_nickname_symbol_filter
tests/test_extract_definitions.py::test_model_extract_abbreviation_acronym
tests/test_extract_definitions.py::test_model_extract_abbreviation_acronym
tests/test_extract_definitions.py::test_extract_abbreviation_acronym
tests/test_extract_definitions.py::test_extract_abbreviation_acronym
tests/test_extract_definitions.py::test_model_extract_abbreviation_shortened_word
tests/test_extract_definitions.py::test_model_extract_abbreviation_shortened_word
tests/test_extract_definitions.py::test_extract_abbreviation_shortened_word
tests/test_extract_definitions.py::test_extract_abbreviation_shortened_word
  /usr/local/lib/python3.7/dist-packages/catalogue.py:138: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
    for entry_point in AVAILABLE_ENTRY_POINTS.get(self.entry_point_namespace, []):

tests/test_extract_definitions.py::test_model_extracts_simple_definitions
  /usr/local/lib/python3.7/dist-packages/catalogue.py:126: DeprecationWarning: SelectableGroups dict interface is deprecated. Use select.
    for entry_point in AVAILABLE_ENTRY_POINTS.get(self.entry_point_namespace, []):

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================================================================================== 210 passed, 4 skipped, 24 warnings in 187.01s (0:03:07) ===========================================================================================

I ran one paper through using the file only format and just citations:
root@635a2b2ca566:/data-processing# mkdir hi
root@635a2b2ca566:/data-processing# python scripts/run_pipeline.py --one-paper-at-a-time --arxiv-ids "1601.00978v1" --output-form file --output-dir hi --entities citations
...
root@635a2b2ca566:/data-processing# ls hi/
citations.jsonl
root@635a2b2ca566:/data-processing# cat hi/citations.jsonl
{"version": "v0", "data": [{"id_": "bandeira_automatic_2010-0", "type_": "citation", "bounding_boxes": [{"left": 0.47549019607843135, "top": 0.30808080808080807, "width": 0.0016339869281045752, "height": 0.008838383838383838, "page": 0, "tex_path": "N/A", "entity_id": "bandeira_automatic_2010-0"}], "data": {"key": "bandeira_automatic_2010", "paper_id": "9769ac9d79a75cfefd3731978b9f319d68345ab5"}, "relationships": null}, {"id_": "bandeira_automatic_2010-1", "type_": "citation", "bounding_boxes": [{"left": 0.4035947712418301, "top": 0.3686868686868687, "width": 0.0016339869281045752, "height": 0.008838383838383838, "page": 0, "tex_path": "N/A", "entity_id": "bandeira_automatic_2010-1"}], "data": {"key": "bandeira_automatic_2010", "paper_id": "9769ac9d79a75cfefd3731978b9f319d68345ab5"}, "relationships": null}, {"id_": "bandeira_automatic_2010-2", "type_": "citation", "bounding_boxes": [{"left": 0.8137254901960784, "top": 0.5404040404040404, "width": 0.0016339869281045752, "height": 0.008838383838383838, "page": 1, "tex_path": "N/A", "entity_id": "bandeira_automatic_2010-2"}], "data": {"key": "bandeira_automatic_2010", "paper_id": "9769ac9d79a75cfefd3731978b9f319d68345ab5"}, "relationships": null}, {"id_": "bandeira_automatic_2010-3", "type_": "citation", "bounding_boxes": [{"left": 0.25326797385620914, "top": 0.5896464646464646, "width": 0.0016339869281045752, "height": 0.008838383838383838, "page": 1, "tex_path": "N/A", "entity_id": "bandeira_automatic_2010-3"}], "data": {"key": "bandeira_automatic_2010", "paper_id": "9769ac9d79a75cfefd3731978b9f319d68345ab5"}, "relationships": null}, {"id_": "urbach_automatic_2009-0", "type_": "citation", "bounding_boxes": [{"left": 0.2957516339869281, "top": 0.3686868686868687, "width": 0.004901960784313725, "height": 0.008838383838383838, "page": 0, "tex_path": "N/A", "entity_id": "urbach_automatic_2009-0"}], "data": {"key": "urbach_automatic_2009", "paper_id": "b4caa57f2321298073d651e95507052ccf2ed66d"}, "relationships": null}, {"id_": "urbach_automatic_2009-1", "type_": "citation", "bounding_boxes": [{"left": 0.23202614379084968, "top": 0.6944444444444444, "width": 0.004901960784313725, "height": 0.008838383838383838, "page": 1, "tex_path": "N/A", "entity_id": "urbach_automatic_2009-1"}], "data": {"key": "urbach_automatic_2009", "paper_id": "b4caa57f2321298073d651e95507052ccf2ed66d"}, "relationships": null}, {"id_": "ding_sub-kilometer_2011-0", "type_": "citation", "bounding_boxes": [{"left": 0.4166666666666667, "top": 0.3686868686868687, "width": 0.006535947712418301, "height": 0.008838383838383838, "page": 0, "tex_path": "N/A", "entity_id": "ding_sub-kilometer_2011-0"}], "data": {"key": "ding_sub-kilometer_2011", "paper_id": "447369e1c509e7e552220fa00eec93e474e1167b"}, "relationships": null}, {"id_": "krizhevsky_imagenet_2012-0", "type_": "citation", "bounding_boxes": [{"left": 0.16830065359477125, "top": 0.5202020202020202, "width": 0.006535947712418301, "height": 0.008838383838383838, "page": 0, "tex_path": "N/A", "entity_id": "krizhevsky_imagenet_2012-0"}], "data": {"key": "krizhevsky_imagenet_2012", "paper_id": "abd1c342495432171beb7ca8fd9551ef13cbd0ff"}, "relationships": null}, {"id_": "krizhevsky_imagenet_2012-1", "type_": "citation", "bounding_boxes": [{"left": 0.1568627450980392, "top": 0.38257575757575757, "width": 0.006535947712418301, "height": 0.008838383838383838, "page": 1, "tex_path": "N/A", "entity_id": "krizhevsky_imagenet_2012-1"}], "data": {"key": "krizhevsky_imagenet_2012", "paper_id": "abd1c342495432171beb7ca8fd9551ef13cbd0ff"}, "relationships": null}]}

I ran it with the file only option and a non-citations entity (equations). It failed (as we would expect):
python scripts/run_pipeline.py --one-paper-at-a-time --arxiv-ids "1601.00978v1" --output-form file --output-dir hi --entities equations
...
AssertionError: C does not know how to write to a file.

Note: this PR is for merging into the chi-2021-demo branch, as that is what the scholarphi-pipeline system is still running off of.

@andrewhead thanks for the comments! They were helpful.
I've made a few updates based on your suggestions, and attempted to answer your questions.
I think the one still open question is around using Literal instead of Enum, from this comment.

hi @ca16 ! Awesome PR, I am already using these flags!!
Would it make sense to rename the extension of the output from .jsonl to .json, given that the file contains just one line and can parsed as single JSON object? Sorry, I should have thought to ask this before, though I only realized it when I started working with the file.

Yeah I think that's a good idea! I can make the change.


New version: LaurentPRenedeCotret.pandoc-plot version 1.7.0

Pull request has been created with WinGet Automation :rocket:
Microsoft Reviewers: codeflow:open?pullrequest=https://github.com/microsoft/winget-pkgs/pull/115083&drop=dogfoodAlpha

@wingetbot run


feat: port MeasureTheory.Constructions.BorelSpace.ContinuousLinearMap

[x] depends on: #4018

This PR/issue depends on:

leanprover-community/mathlib4#4018
By Dependent Issues (🤖). Happy coding!

This PR/issue depends on:

leanprover-community/mathlib4#4018
By Dependent Issues (🤖). Happy coding!

bors merge

bors merge

bors r-
bors d+

bore merge

bors merge


chore(test): more logging to debug #5163

@elsirion compilation error, needs a + '_

@elsirion compilation error, needs a + '_

Yeah, just saw it, I wonder why local clippy didn't complain. Maybe I need to refresh my dev env? Haven't restarted it in a few days.

RA using clippy might be your local setting @elsirion
E.g. from my ~/.config/helix/languages.toml
[language-server.rust-analyzer.config]
inlayHints.bindingModeHints.enable = false
inlayHints.closingBraceHints.minLines = 10
inlayHints.closureReturnTypeHints.enable = "with_block"
inlayHints.discriminantHints.enable = "fieldless"
inlayHints.lifetimeElisionHints.enable = "skip_trivial"
inlayHints.typeHints.hideClosureInitialization = false

checkOnSave.command = "clippy"

I'm just using the just clippy command right now since my IDE setup is kinda broken again :sweat_smile: but maybe it's using weird versions or something since I'm running it in the IDE shell.


Create a data conector for Github repos Part 1 (all supported filetypes but no source code)

Create a data connector that pulls data from GIthub repo.
See Adding Data Connectors for details on how to get started. Make sure to include a link to this issue when submitting a PR.
Definition of Done

The checklist has been completed.
The connector is able to process a single repository and recursively the entire documents in a repo.
Should ignore code.
Should process only doc types supported in partition.auto, including markdown files.

Bonus points: the ability to filter by document type.

I'd like to have a go at this! What CLI arguments would you recommend that I introduce?
I imagine something along the lines of:

--repository_url or --github-url.
and optionally
--filetypes or --glob or --fileglob, etc.

Would love to hear your thoughts.

Should process only doc types supported in partition.auto, including markdown files.

How should this be handled? To the best of my knowledge, there is no partition_markdown. Should markdown be parsed using partition_text?


Question about Table 6 in your paper

Hi!
Can you tell me if the performance in table 6 of your paper was measured by pelvis joint relative mpjpe or global coordinate mpjpe?

mpjpe


"Tamil with Grantha" script class: redundant?

Dear @manufrancis and @alevivier , it seems to me that in our Open Theso for scripts, the class "Tamil with Grantha" (https://opentheso.huma-num.fr/opentheso/?idc=57475&idt=th347) is redundant and should be discarded. An inscription in a mix of those scripts would be encoded using both 38776 Tamil and 36768 Grantha in the metadata, and in the edition itself, it would have 38776 Tamil on the edition div, and 36768 Grantha on <seg> elements wherever necessary. See EGD §7.5.5.
Am I right?

Dear Dan,
Sorry my very very slow answer on this issue.
As far as I remember we devised this class "Tamil with Grantha" for inscriptions in Tamil script where words or syllables appear in Grantha (and thus tagged <hi rend="grantha">).
I guess we can dispense with this class "Tamil with Grantha".
BUT
What to do with the text tagged <hi rend="grantha">?
Allow the @rendition with <hi>? Thus <hi rend="grantha" rendition="class:38768 maturity:83213">?
Change <hi rend="grantha"> to <seg rend="grantha" rendition="class:38768 maturity:83213">?
@danbalogh
@michaelnmmeyer
What do you think?

The EGD (§7.5.5) has already been revised so that instead of <hi rend="grantha"> the correct encoding is now <seg rend="grantha" rendition="class:38768 maturity:83213">.  This applies to an edition, whereas (EGD §10.2.1) outside an edition (e.g. in original text cited in a discussion), <hi rend="grantha"> remains available. This is because script is not encoded with rendition outside the edition, but I assume you still wanted a way to display transliterated grantha characters differently.
I do not recall when this revision was made, but certainly not recently. Probably around the time we were discussing the script codes. It was certainly meant to replace the original usage of <hi rend="grantha"> (I have just now corrected a typo caused by incomplete revision: the example <seg rend="grantha" rendition="class:38768 maturity:83213"> was closed by </hi>), and I am sure that this revision would not have taken place without your approval.
So can we keep the EGD as it is now?
If any files still use <hi rend="grantha"> in the edition div (or in apparatus lemmas or readings), these can be mechanically changed to <seg rend="grantha" rendition="class:38768 maturity:83213">.

Dear Dan,
Fine! Let us keep the EGD as it is now.
I will see with Michaël for the automatic change and the specific display (bold) of transliterated Grantha.
PS: I think you have somewhere a list of automatic changes to be implemented on the DHARMA files; maybe add to this list the change of <hi rend="grantha"> to the correct encoding <seg rend="grantha" rendition="class:38768 maturity:83213"> in
 as well as in
, as I might still use <hi rend="grantha"> as a short-hand encoding.

Sorry, I copied and pasted mechanically. The correct form is <seg rendition="class:36768 maturity:83213"> (no @rend="grantha").

Thanks! Noted!


Ability to customize underlying components during setup

It would be great to be able to modify things like the transform and sprite components when the PixelBuffer is created. What if instead of using SpriteBundle, there's PixelBufferBundle that replaces the current tuple bundle and surfaces components users are likely to want to customize. An example is the ShapeBundle in bevy_prototype_lyon: https://github.com/Nilirad/bevy_prototype_lyon/blob/master/src/entity.rs#L23
That way sensible defaults can still be provided and there can still be a builder API, but if you still have access to all of the components that are currently hidden in SpriteBundle.
The exposed API could look something like this:
let size = PixelBufferSize::size((100, 100));
let image = PixelBuffer::create_image(size, |frame| { frame.per_pixel(|_, _| Pixel::RED) });

commands.spawn_bundle(PixelBufferBundle {
  size,
  texture: images.add(image),
  transform: Transform::from_xyz(10., 10., 0.),
  sprite: Sprite {
    anchor: Anchor::BottomLeft,
    ..default()
  },
  ..default()
});

The thing I'm not sure about is how the non-sprite usage of the library fits in with this. What did you have in mind for non-sprite usage? Would it use a custom bevy renderer instead?

What I'm thinking and implementing is this
/// Render setup configuration
#[derive(Clone, Debug)]
pub enum RenderConfig {
    /// Set up a sprite and an optional 2D camera
    AsSprite {
        /// Spawn a 2D camera
        spawn_camera: bool,
        /// Custom sprite bundle parameters.
        ///
        /// Different from [SpriteBundle] because [SpriteBundle] some extra fields that are not customisable.
        sprite_bundle: CustomSpriteBundle,
    },
}

CustomSpriteBundle maps all the customisable fields of SpriteBundle. The onyl not customisable fields are the texture and the sprite custom size.
I'm also making possible to use a bundle instead of the builder pattern.

RenderConfig is an enum to support future alternative ways of rendering. In the future I intended to implement a custom renderer that did't use the bevy one, like pixels does, so if the pixel buffer is the only thing rendering there's no overhead.

For the texture, are there things you could potentially change about the Image and still have the buffer work fine? I wonder if instead of hiding it completely during setup, you could provide a helper function to create an Image that works well for the pixel buffer and maybe assert certain conditions in systems that interact with the Image. I think that works better because nothing is stopping the user from customizing any of these components in their own systems.
Similarly for Sprite.custom_size, maybe it could be added in a startup system?

I don't know too much about custom Bevy renderers, but how would the components be involved? If you were doing it like bevy_pixels, then you wouldn't have a set of components at all, but rather the plugin would expose the frame buffer as a resource and would just define systems for rendering.

That's exactly what I have done. I new utility function and I also implemented a couple of bundles in case the user does not like the builder pattern.
I'm still fixing a couple of edge case bugs, but I think the changes will be pushed soon (tomorrow probably).
Regarding the rederers, neither do I, but having this API allows to add more if needed in the future.

Hmm I'm confused, why do you need CustomSpriteBundle then? I would think with the utility function to create the Image that you could supply the texture directly?

Ok, changes made. CustomSpriteBundle is for the builder. The bundle to create the pixel buffer manually is PixelBufferSpriteBundle that contains a normal bevy SpriteBundle. There are a couple new examples.
I want to rethink the query module before a new release. Shouldn't take too long.

Ohh gotcha, makes sense! 👍

Is this how you would recommend initializing the frame with the bundle approach?
let image_handle = create_image(...);
let mut frame = Frame::extract(&mut images, &image_handle);
frame.per_pixel(...)

commands.spawn_bundle(PixelBufferSpriteBundle { ... });

Yes, that is how to initialize the frame with the bundle approach. But that made me think if create_image returned the image itself instead of the handle. In that case a bundle without initalization would be:
commands.spawn_bundle(PixelBufferSpriteBundle {
  pixel_buffer: ...,
  sprite_bundle: SpriteBundle {
    texture: images.add(create_image(...)), // user needs to add the image
    ...
});

And the initalization:
let mut image = create_image(...);
let mut frame = Frame::get(&mut image);
frame.per_pixel(...);
let image_handle = images.add(image);

commands.spawn_bundle(PixelBufferSpriteBundle { ... });

What do you think is better, how it is currently or if create_image returned the image instead of the handle?

Oh yeah I think returning the Image looks good here. I think it's a bit more flexible and looks better for this case of initializing the frame. In my initial example I had thought about letting the user pass in a frame initializer to create_image itself -- how do you feel about that?
let image = create_image(size, |frame| { frame.per_pixel(|_, _| Pixel::RED) });

Not everyone would want to initalize the image in or just after create_image. I think for most people it would be fine it the image is not initialised and they just get a frame in their regular update system.
For now I will change create_image to return the image and add convenience traits to do something like
let mut image = create_image(...);
image.edit_frame(|frame| frame.per_pixel(|_, _| Pixel::RED));

Could the convenience traits go even further to just implementing the frame functions on Image itself? Something like this:
image.per_pixel(|_, _| Pixel::RED);

Could be done but I prefer to explicitly get the Frame because it's a reinterpretation of the Image. The image has a Vec<u8> for the data and the Frame interprets it as a &[Pixel]

I was actually wondering -- is it expensive to re-create the Frame every tick -- could it be stored in the PixelBuffer component if so?

I believe it is practically free to build the Frame every tick. However, you could benchmark it just in case.


refactor: rename OfflineBasemap -> OfflineArea

Rename OfflineBaseMap to OfflineArea.
This is the first in a few class renames we'll perform in relation to offline base maps functionality in order to make the code clearer.

/gcbrun

Codecov Report

Merging #1036 (d9ec1e1) into master (1dec966) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #1036   +/-   ##
=========================================
  Coverage     30.41%   30.41%
  Complexity      753      753
=========================================
  Files           326      326
  Lines         11479    11479
  Branches       1119     1119
=========================================
  Hits           3491     3491
  Misses         7441     7441
  Partials        547      547

Impacted Files
Coverage Δ

...gnd/persistence/local/room/RoomLocalDataStore.java
74.76% <0.00%> (ø)

...nebasemap/viewer/OfflineBaseMapViewerFragment.java

...droid/gnd/repository/OfflineBaseMapRepository.java

...gnd/ui/offlinebasemap/OfflineBaseMapsFragment.java

...linebasemap/OfflineBaseMapsFragmentDirections.java

...emap/selector/OfflineBaseMapSelectorViewModel.java

...istence/local/room/dao/OfflineBaseMapDao_Impl.java

...ctor/OfflineBaseMapSelectorFragmentDirections.java

...ebasemap/viewer/OfflineBaseMapViewerViewModel.java

...e/local/room/models/OfflineBaseMapEntityState.java

... and 23 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1dec966...d9ec1e1. Read the comment docs.


add two Binary tree algorithms

Make sure to add a description next time.
Great work with the contribution.

Thanks, sure I'll check it out next time
On Sun, 2 Oct, 2022, 1:14 pm Muhammad Umar Waseem, @.***>
wrote:

Make sure to add a description next time.
Great work with the contribution.
—
Reply to this email directly, view it on GitHub
https://github.com/Umar-Waseem/DSA-Hacktober-Fest-2022/pull/42#issuecomment-1264576258,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ASEQBFUCDRHDO563LBY4NY3WBE4M3ANCNFSM6AAAAAAQ2YVHTE
.
You are receiving this because you authored the thread.Message ID:
@.***>


search is not a url command

Thanks for the PR. Question:
if (Regex.IsMatch(arg, @"[0-9A-Za-z_-]{10}[048AEIMQUYcgkosw]")) url = null;
In what situation would this return true? Any url is almost bound to have a character that doesn't match this, such as period or colon

Uri.IsWellFormedUriString does not catch all urls:

Thanks for the PR. Question: if (Regex.IsMatch(arg, @"[0-9A-Za-z_-]{10}[048AEIMQUYcgkosw]")) url = null; In what situation would this return true? Any url is almost bound to have a character that doesn't match this, such as period or colon

I see, cef supports playing just a youtube video id. Didn't actually know that


Support tasks within callouts

Currently I am unable to use Task Collector if a task sits within a callout, for example:
> [!todo] Tasks
> - [ ] Task

It would be great if these tasks could be supported.

oh weird.. I'll look

In the case where completed items are moved, would you expect them to be removed from the callout? (leading >  stripped?)

Hmm, I don't actually use that option, so I can't really answer that. If I was to, I think I'd be quite happy for them to move into a normal non-callout list, but others might have a different set-up.
Thanks for looking into this!

Try with latest BRAT release (0.7.8) .. and let me know

Perfect - it works now! Thank you 🙂


Updated Components topic for OIDC authentication

What this PR does / why we need it:
This PR updates screenshot and text in the Logging into Plutono section of the Components topic with details on logging in with OIDC authentication. It also adds information on programmatic access.
Which issue(s) this PR fixes:
Fixes #545
Special notes for your reviewer:
Release note:

NONE

@nickytd @rickardsjp I have omitted the mention of "programmatic access" in the topic. Is there a need for further updates, or can the PR be merged as is?


Why is my AppImage being uploaded to a new package called gvim-8.0.0118.glibc2.15-x86_64.AppImage at Bintray?

Hi,
I have created a new repo called GVim.AppImage in an attempt to overcome #150 (more like a temporary workaround than a long-term solution) and when it publishes to bintray it creates a new package called the same as the AppImage's filename. For example, the gvim-8.0.0118.glibc2.15-x86_64.AppImage AppImage ended up being uploaded to this package. Any ideas why it isn't being uploaded to the gvim package at Bintray instead?
Thanks for your time,
Brenton

My bintray.sh file is the same as yours.

My attempts to fix this problem have failed. They involved substituting (basename "$FILE") with gvim.

I can confirm that this is a bug, but i don't know the reason yet.

Oh well when ya find a solution please do share. Using transfer.sh isn't much of a long-term solution for storage.

You can also use GitHub Releases, e.g., using uploadtool or a variation thereof.


sql.raw doesn't support buffers when replacing

my workflow is passing a read file into
knex.raw(fs.readFileSync(file), ['replace with this']).then((response) => {doSomething(response)});

You need to pass encoding to your readFileSync or otherwise it won't know how to convert buffer to string.


BenchmarkDotNet usage

Is your feature request related to a problem? Please describe.
I can't use BenchmarkDotNet in notebook because it needs release build to function properly.
Assembly BenchmarkDotNet.Samples which defines benchmarks references non-optimized BenchmarkDotNet
        If you own this dependency, please, build it in RELEASE.
        If you don't, you can create custom config with DontFailOnError to disable our custom policy and allow this b
Assembly BenchmarkDotNet.Samples which defines benchmarks is non-optimized
Benchmark was built without optimization enabled (most probably a DEBUG configuration). Please, build it in RELEASE.

Describe the solution you'd like
I would like to be able to use BenchmarkDotNet in notebooks. It would be incredible because sometimes I want to quickly benchmark piece of code but I am too lazy to create full featured benchmark. Notebooks significantly shorten the feedback loop with ability to draw graphs and so on.
Its either something .NET Interactive doesn't support yet or I don't know how to configure any help is appreciated.

I've been experimenting with this a bit myself. So far I'm really just driving the release build from a PowerShell cell but ideally you could benchmark the code submission directly. When I tried experimenting with that, I hit a blocker pretty quickly that I've just opened an issue for: https://github.com/dotnet/BenchmarkDotNet/issues/1677.
@adamsitnik

@jonsequitur Thank you for taking a look I am going to look forward for any development around this issue 👍


Fix make V=1

See https://github.com/devkitPro/devkitppc-rules/blob/master/base_tools#L41 for more information.

lol, oops.


Use datapackage's base path when loading custom profiles

For example, consider the datapackage:
{
    "name": "test",
    "title": "Test",
    "profile": "my-data-profile.json"
}

The my-data-profile.json should be loaded relative to the datapackage.json path, otherwise we would need to use the absolute path, which isn't portable.

If it doesn't use the data package directory for now - it's a bug.


question about test

it seems that codalab only gives 100MB discs instead of 10GB and one hour now（I fill the form but not apply）. Under this setting, it seems I can not upload databases. So is there any other method to get the test score？

We are facing a similar issue with the new codalab limits, as we can't upload our model weights for evaluation because their size is more than the allowed disk space. Any alternatives for evaluation?

We are facing the same problem too,Have you solved the problem yet?


Status not showing all Linksys routers in network

I have 3 Linksys routers in my home network.  One is downstairs and the other 2 are in the same room as the HSMM-Pi node.  The HSMM-Pi only shows the 2 in the same room on its Status page.
Does HSMM-Pi only show the nodes it is actually connected to or all the nodes in the network?
One of the Linksys nodes is a tunnel.  On the other Linksys nodes I can see the Remote nodes on the other side of the tunnel.  On the HSMM-Pi Status page it only shows the tunnel.
So maybe that is two questions, but they are related.
Gary

The HSMM-Pi status page only shows neighbours; remote nodes are not shown. I'll see about adding remote nodes soon.

Version 0.8.0 now displays remote nodes. Are you able to see all your nodes on the status page now?

Thanks for updating the program.
I am real busy on another project right now.  I will try to check it in a few weeks.
Gary
---- Clayton Smith<EMAIL_ADDRESS>wrote:

Version 0.8.0 now displays remote nodes. Are you able to see all your nodes on the status page now?

You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/urlgrey/hsmm-pi/issues/61#issuecomment-205103388

Ok, I'll close the issue for now. Feel free to re-open it if the issue isn't fixed.


read ECONNRESET

My requirement is to call PowerShell from node.js Lambda.
I packed PowerShell in same zip and change source path for spawn from
powershell to /var/task/powershell
PowerShell is starting but not finishing properly I think commands are executing also but I am not getting a result whether the command is finish or not. I am getting the ECONNRESET error
read ECONNRESET
at _errnoException (util.js:1022:11)
at Pipe.onread (net.js:628:25)

      NPS>  Process 11 started
   Inside ShellWrite echo node-powershell
   NPS>  Command invoke started
   NPS>  echo node-powershell
   inside invoke 1
   Inside ShellWrite
   inside invoke 2
   Inside ShellWrite echo EOI
   inside invoke 3
   Inside ShellWrite
   Error: read ECONNRESET
   at _errnoException (util.js:1022:11)
   at Pipe.onread (net.js:628:25)

I added some logs to confirm which part is making a problem.
I am not getting any error on my local environment

To be clear we are talking about AWS Lambda?

@RayeesMir .
Currently NPS wasn't tested on Lambda, I will be happy if you could do the research and add section in the documentation tips.
Thx

 Lambda POC


Data Binding Lessons Learned the Hard Way

In your roadmap for 2016, you mention confusion over the data binding system as a recurring problem. I have encountered this and have documented my lessons learned, workarounds, and solutions to sibling element data binding issues that I found critical to building a more complex MVC-like app.
Please see https://github.com/kbenjamin/polymer-issues
I welcome corrections, suggestions on better ways to achieve the same results, and any other feedback.
If you find any of the content of value and would like to reuse it in any form to improve Polymer or help others, please feel free.
Once I resolved all the mentioned issues, I've found Polymer 1.2 to be a good functional replacement for Polymer 0.5, which did seem to Just Work. Unfortunately, there were many hours of frustration getting to that point. Hopefully, my experiences will help others avoid all that.

I would like to see a jsbin for this :arrow_down:. I never had to do this.
  created: function(){
    window.addEventListener('WebComponentsReady', function() {
      this.set('appReady', true);
    }.bind(this));
  },

Thanks @kbenjamin! I'm in the process of doing a major rewrite of the data binding section. I think most of the information you refer to is in there somewhere, but it's often very hard to find and not obvious to people learning the system.
I think the queuing workaround you document may no longer be required in 1.3.0 and later. There was a fix in that release allowing data binding to work property on un-upgraded elements, which I think is the issue you were seeing:
https://www.polymer-project.org/1.0/docs/release-notes.html#v1-3-0
The point about initializing a shared object is a good point and one we don't cover currently. Likewise, I'll try and clarify observer timing, and stress the need to declare all of your dependencies in observers and computed properties.

@kaste This was only an issue due to sibling elements not necessarily having been ready before data binding communication was occurring. Child elements were never a problem but siblings were. By listening for WebComponentsReady, I was able to detect when those siblings were able to accept data changes.
@arthurevans Thank you! You were right. I developed the queue under 1.2 to resolve timing issues between as-yet un-upgraded elements. I've just tested and was able to remove the entire queuing system from my app running under Polymer 1.4. I was able to simplify the code so I'm very happy about that.
With this major issue resolved, there remains one area of concern I have which is observer dependencies.
The current options for managing interdependent properties are somewhat problematic and the solution seems magical to the point of being worrisome. Take this snippet:
observers: [ 'somethingChanged(something)' ],

somethingChanged: function(){
  this.processChanges();
},

processChanges: function(){
  if( this.anotherThing === 'hello' && this.thirdThing ){
    // do something
  }
}

This may fail, or it may not. It depends entirely on the state of this.anotherThing and this.thirdThing. The current solution is to use async to fix the problem:
somethingChanged: function(){
 this.async( this.processChanges );
},

What bothers me about using async is that it's effectively a timing fix relying on micro-task completion. Can you 100% guarantee that under all circumstances all outstanding set's will have occurred in the current micro-task? Could something cause a set to occur in the next micro-task? If so, how far out could something get pushed?
I'm not convinced you can rely on micro-task completion but I have no evidence to suggest that you can't. It just seems like a bug waiting to happen that'll be nearly impossible to duplicate or track down.
I'd rather see an ability to define observer dependencies explicitly.
I understand that we can do something like this:
observers: [ 'somethingChanged( something, anotherThing, thirdThing )' ]

But, unless I'm mistaken, that would fire somethingChanged() when any of the three variables change, which, while useful, is rarely what I want in an observer (although often what I want in a computed binding). Usually, I only want to fire the change watcher when a single variable changes.
I propose something like this to explicitly define dependencies:
observers: [ 'somethingChanged( something, [anotherThing, thirdThing] )' ]

This feature should be extended to computed bindings and to property observer as well, although I'm not clear as to what format would make sense for observer and not break things. Perhaps it would have to be another property value:
properties: {
  something: {
    type: String,
    observer: 'somethingChanged',
    observerDeps: '[anotherThing, thirdThing]'
  }
}

Am I missing something here that solves this problem?

Could you please describe with some pedantry what the difference between

observers: [ 'somethingChanged( something, anotherThing, thirdThing )' ] and
observers: [ 'somethingChanged( something, [anotherThing, thirdThing] )' ]

should be?
Is it for the second: somethingChanged is called whenever something changes but only if the other two things are initialized/not undefined :question:

@kaste Yes, you understand my intentions. I would assume the exact same conditions exist between the two definitions which, as I understand things, means that they must not be undefined.
If I want to leverage undefined, I'm still free to do so by simply not including the value in a dependency list.
The other problem I want to see resolved is when changes fire if there are multiple updates occurring. If they were to happen sequentially, i.e. if I set('anotherThing', 'wookies');, then, elsewhere, set('something', true);, I would expect that I can rely on that change to anotherThing having completed prior to somethingChanged being called.
Those sequential conditions must remain true regardless of where the set is called, i.e. in a distant element connected through data binding. The fact that async is required in these kinds of situations suggests to me that I cannot currently trust the order of operations, which, added to the matter of undefined, is really the problem I'm trying to solve here.
Guarding against undefined is easy. Knowing when a set is pending is problematic, to say the least.
Perhaps I'm mistaken in my assumptions about when I need async and it is, in fact, an artifact of my dependence on possibly undefined properties. If the cross-element data binding order of operations is guaranteed, then disregard this aspect of the issue.

@kbenjamin
Regariding:

"But, unless I'm mistaken, that would fire somethingChanged() when any of the three variables change"

You could debounce the somethingChanged call so that it is only called once when either of its dependencies change quickly consecutively.
In my experience explicitly declaring all the dependencies and avoiding this.someproperty access in computed properties/observers works best and is most predictable.

@timeu I agree that explicitly declaring the dependencies is the right way to go. More than best, it's actually required if you want your app to be reliable.
The problem with observers: [ somethingChanged( something, anotherThing, thirdThing );] is that somethingChanged fires whenever any of the three variables change. What if the only trigger I care about is something but I need to use the current values of anotherThing?
If somethingChanged() is fired when thirdThing changes, then somewhere, I need to track the state of something to see if it changed, and return if it didn't. Unfortunately, for observers we can't easily do that without setting a separate variable to maintain the state. Per the docs:

Observers do not receive old values as arguments, only new values. Only single-property observers defined in the properties object receive both old and new values.

The proposed change resolves this common situation by separating change watchers from their dependencies.

I totally agree with you @kbenjamin , I had the same problematic several time, and would like to see a mechanism that helps handle this case properly!

Related Q. Observer:[observe(courseCalandarDaysArray.*).
In top level bind in template repeat compute(courseCalendarDaysArray.*). as day.
Next level compute in repeat
day.courseCalendarDay.eventList
as event.
In top observer and compute as recorded by console log
courseCalendarDaysArray.#[index].courseCalendarDay.eveventList,#[event item index, integer].
Lower compute.
Lower observer
On first run or a run from a null of the dataset. Fine.
Both work.
Run two plus absent null, and some dodgy cloning top level observe and compute track change all the way down to the item level change in second repeat.
 Blank paper items on day cards to be rewritten, color gone, text gone, time gone.  Paper item created with two inputs.
Later p items not rewritten copescetic.
Tried notify piecing the path together.
Cleared the instance and pointer issues
Replicated issue in double repeat using simple example with button pushes.
Question.
How does top level compute and observe see all the way down to second level observe. Know their is a change and yet the second level compute and repeat, designed and tasked with render of changed item, has no idea there was a change?
Could be a dopey question. Sorry if it is. Still just befuddled that observer above passing the item that becomes Dom repeat two knows about 4-5 level down change yet render of obj at that level... No idea?
How fix for truly Dynamic? Our lame work around, as suggested above: null, clone using map and let/ or second constructor, plus an asynchronous call on the re set of array
Plus a this.nofifyPath("fcCourse.courseCalendarDaysArray." + index + ".courseCalendarDay,eventList." + i, cat or pickle);
We use cat to refer to unorthodox work around (more than one way to skin a cat) and pickle to indicate "please change this code when you see it; it was a work around for a dandy of a 'pickle'".
Notify, null, constructor, plus async work if placed at the right point. For a dynamic multi element/input interface with a click happy user.
Can't really use it more than once.
So... Notify, yes... Array slices explicit got it.
Observer limits. Cool.
Need better solution or at least logic on how a-->b-->b'
A knows B and B prime are on a chNge path. B B'.... No dice.
Sent from my iPhone

On Mar 31, 2016, at 6:29 PM, Ken Benjamin<EMAIL_ADDRESS>wrote:
@timeu I agree that explicitly declaring the dependencies is the right way to go. More than best, it's actually required if you want your app to be reliable.
The problem with observers: [ somethingChanged( something, anotherThing, thirdThing );] is that somethingChanged fires whenever any of the three variables change. What if the only trigger I care about is something but I need to use the current values of anotherThing?
If somethingChanged() is fired when thirdThing changes, then somewhere, I need to track the state of something to see if it changed, and return if it didn't. Unfortunately, for observers we can't easily do that without setting a separate variable to maintain the state. Per the docs:
Observers do not receive old values as arguments, only new values. Only single-property observers defined in the properties object receive both old and new values.
The proposed change resolves this common situation by separating change watchers from their dependencies.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

In general, @kbenjamin,  set and all data propagation through the whole system is synchronous, unless an element is not ready for it. That means you can't rely on that fact and must solve the problem as-if everything could be delayed, deferred or gone.
Again in general, if you do something in ready(), and you probably shouldn't do much here, your parent will not yet be ready.

What if the only trigger I care about is something but I need to use the current values of anotherThing?

If you think about this, you will find an efficient solution using just multi-observers and computed properties (and debounce). Actually a computation either depends on a value or not, there is no grey here. Think math: (a, b) => a+b

If somethingChanged() is fired when thirdThing changes, then somewhere, I need to track the state of something to see if it changed, and return if it didn't.

No, you need to map the thirdThing to a subset of values, so it doesn't trigger somethingChanged that often.  c(b) => b === hello;   (a, c) => ...

@kaste Earlier, you asked me about why I might need this:
  created: function(){
    window.addEventListener('WebComponentsReady', function() {
      this.set('appReady', true);
    }.bind(this));
  },

And then say this (emphasis added)?

In general, @kbenjamin, set and all data propagation through the whole system is synchronous, unless an element is not ready for it. That means you can't rely on that fact and must solve the problem as-if everything could be delayed, deferred or gone.

Am I misunderstanding you or isn't that asked and answered? I think, however, the issue of elements not being "ready for it" might have been resolved with 1.3.
Observers will not be run but at least, once they do, the external data-bound properties will have been set to the current value.
I suppose there are situations where multiple property value transitions take place before an element can respond and some of those transitions could be lost without some sort of queuing mechanism. My app doesn't have that kind of thing going on so I didn't see it as a problem but I can see how it might be.
Does it make any sense to make the data binding system synchronous and have it wait until all the elements are loaded before running? You might say it'd be slow but, at least in my testing, I couldn't measure a performance difference between having my queue that waited for WebComponentsReady and not having it as the differences, if any, were hidden within the normal variability.
As for the rest...

If you think about this, you will find an efficient solution using just multi-observers and computed properties (and debounce).

I recall trying exactly this based on feedback from @ebidel and found it complex and not handling all use cases.
I've found that it's easiest to use async before accessing this.thirdThing if I know that it might not be initialized yet. These are mostly edge cases involving startup-related functionality so I do this very sparingly.
I searched my app to see where I used observers arrays and I found only two circumstances that haven't been refactored into single-property observers:

As single-property observers on an unobserved property defined in a behavior
As true multi-property observers where I am interested in any property changing

With Polymer 1.2 my queuing system resolved the problems I described in my original post and, with Polymer 1.3, the setting of properties on un-upgraded elements resolved those issues without the queue.
In almost every situation, I found a way to refactor the code to use a single-property observer. All that said, I'd still like to see something like:
observers: [ 'somethingChanged( something, [anotherThing, thirdThing] )' ]
Doing so would allow me to remove guard code and eliminate async. Before:
properties: {
  selected: {
    type: Boolean,
    value: false,
    observer: 'selectedChanged'
  },
  foo: String
},

// selected is set externally and may true before *this.foo* is initialized

selectedChanged: function( newVal, oldVal ){
  if( oldVal === undefined ){ return; }
  this.async( function(){
     if( this.foo === 'hello' ){ ... }
  });
}

By using observers: [ 'selectedChanged( selected, [foo] )'] this could be reliably refactored to this:
properties: {
  selected: {
    type: Boolean,
    value: false
  },
  foo: String
},
observers: [ 'selectedChanged( selected, [foo] )'],
selectedChanged: function( selected, foo ){
   if( foo === 'hello' ){ ... }
}

@kbenjamin It has been a while, but most of the data-binding updates from Polymer 1 to 2 target the common issues you reported. You can read about the updates at https://www.polymer-project.org/2.0/docs/about_20#data-system Your issues with undefined in observers during initialization should be solved now; you can set multiple properties with setProperties; and most of the gotchas are also documented at https://www.polymer-project.org/2.0/docs/devguide/data-system#data-flow
I am sorry for the late response, but if I understand correctly most of your feedback has been incorporated in Polymer 2. Please let us know if we can close this issue or if you are still experiencing problems with Polymer 2!


Bugfix: Using AR Join and table names with a dash (-)

Bugfix for when using table names with a dash (-) the join would mess up

Can you add something to the changelog about this?


Add Code Coverage

Changes
Add code_coverage workflow
It:

Generates a code coverage report
Uploads the code coverage report to Codecov

Add --coverage option to jest in test:ci
This generates the code coverage report.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@e207c5d). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             master       #18   +/-   ##
==========================================
  Coverage          ?   100.00%
==========================================
  Files             ?         7
  Lines             ?        75
  Branches          ?        17
==========================================
  Hits              ?        75
  Misses            ?         0
  Partials          ?         0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e207c5d...5e14258. Read the comment docs.

:tada: This PR is included in version 1.9.0 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


Please create arm64 provider

Describe the problem/challenge you have
Can't use a provider on apple M1
Describe the solution you'd like
Use provider on apple M1 via terraform.registry.io
Cheers
Marcin

I am looking for vmware-tanzu/caravel for the M1.

No question: we want this.
Carvel maintainers are eyeing #29 for the near future. This need could be addressed concurrently.
That said, will definitely accept contributions around this, too.

Hi @pivotaljohn
Myself and @asalan316 have compiled carvel and tried locally for arm64 and it worked just fine. Not sure though how we can help in releasing it to terraform registry.

As @marcinkubica said, it worked by using local go binary on Apple Silicon. But, probably need some help to release it in order to fix the issue.

Hi, @pivotaljohn. I am ready to contribute. Our team decided to move to ytt from plain templatefile(). But we are facing a problem with the support of recent M1 Macs. Since the release to the registry is not codified, there is not much external contributor can do about.
With the artifact generation as i can see, project uses goreleaser to generate releases, but github action version was not updated for 2 years, so most likely that version does not support recent architectures. Maybe we should start by simply bumping goreleaser version?

Maybe we should start by simply bumping goreleaser version?

Makes sense 💯  @mou Please feel free to make a PR for it :)

@praveenrewar I created a PR #34  which should solve the exact issue. But it is need verification and as I understand publication to the registry is something that should be performed manually by the team of maintainers.

It will be an excellent investment in the development of the carvel project
@mou thx

arm64 support is a show-stopper, tbh.

arm64 is a must-have option here, please, take it into consideration to speed up the PR

Hey, due to a change in Github org, we had to update the provider. Here is the new link https://registry.terraform.io/providers/carvel-dev/carvel/latest

Thank you very much for so quick review and release. You saved a lot of internal Q&A time for our team.


Could you add a bot for impoppable

I'm really struggling to beat dark castle impoppable, could you add a bot for it?

the bot here is able to do it if you manually edit the coordinates, I did some maths for mine and made some more entries myself. You should also edit instructions to place the first towers a bit before round 6, unless you feel like handling it manually to round 6

When I try to run it it starts, places 1 dart monkey and 1 monkey sub and then presses escape, do you know why this is? Or would you be able to share your code?

the only thing i really changed was where the bot would select the game mode. in Static.py there's a list of button positions. By default, the bot uses the CHIMP_MODE. you can choose here, if you want to edit this entry, or make a new entry and change bot.py to use the new entry instead. for the values, i used the info from the other button positions and calculated the position in pixels. Hope this helps

And @Shaksos impoppable also starts at round 6 right?

Impoppable & Chimps start both on round 6. You should basically just need to do as Shaksos says by changing the CHIMPS_MODE cordinates in static.py as CHIMPS are more restricted (Monkey knowledge etc).
In the repo there is a pos.py which you can run to see the cordinates of the button if you feel like changing it.

When I try to run it it starts, places 1 dart monkey and 1 monkey sub and then presses escape, do you know why this is? Or would you be able to share your code?

Are you still experiencing this?

Are you still experiencing this?

Yep, after it sets the first dart monkey to target strong it pauses the game.


podhelper.rb depends on flutter_application_path being magical

Make it non-magical
Related https://github.com/flutter/flutter/issues/32994

@jmagman would be nice to address this simultaneously too

I think this means that you happened to have set it in the 'parent' script in https://github.com/flutter/flutter/pull/36793/files#diff-a6f9a84f8e880e4d2d6051b4f045cb75R3 but that variable being present isn't being read as a global in podhelper.rb right? (it's passed in explicitly in install_flutter_* etc)

flutter_application_path isn't global, and is not required to be set at all.  I just used it to DRY out the load and install_all_flutter_pods lines.
You could do:
load './flutterapp/.ios/Flutter/podhelper.rb'
target 'MyApp' do
  install_all_flutter_pods('flutterapp')
end

SG, thanks!


init not recognised as member of pygame module

Issue Type: Bug
Although the program can run and causes no debug, the auto checker on the text advises that the init function of pygame is not a member of pygame. The simple test code i wrote was just to create a pygame window and close it immediately but pg prior to init is highlighted by error: "Module 'pygame' has no 'init' member pylint(no-member)"
Since it doesn't affect program running this seems like it would be a NULL error realistically but it is odd that despite auto correct providing INIT to be selected, it shows this error once you do select it.
import pygame as pg

pg.init()

gameDisplay = pg.display.set_mode((800,600))
pg.display.set_caption("Test Game")
clock = pg.time.Clock()

dead = False
pg.quit()
quit()

the same issue occurs with quit
Extension version: 2020.6.91350
VS Code version: Code 1.46.1 (cd9ea6488829f560dc949a8b2fb789f3cdc05f5d, 2020-06-17T21:13:20.174Z)
OS version: Windows_NT x64 10.0.18363

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz (8 x 4008)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_off_okwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
7.94GB (1.91GB free)

Process Argv

Screen Reader
no

VM
0%

Hello @VariableZer0, thank you for reaching out.
What you're experiencing is due to the following behaviour from Pylint:

Only C extensions from trusted sources (the standard library)
are loaded into the examining Python process to build an AST
from the live module.

There are existing workarounds for this behaviour, for example in this SO answer.


Add support for resource level locks.

Fixes #3493

The --resource-type parsing has been fixed by #3584 I believe.

Comments addressed, please re-check.
Thanks!

CI is failing for this test.

@tjprescott I see:
CLIError: Please run 'az login' to setup account.

in my scenario test. Why is that there? Do I really need to run az login in the test?

No, you don't.  I would try re-recording the test and see if that fixes it. You'll get that error if there's no recording because the CI will try to run the test live and fail. You could also get it if you had a bad recording which recorded that as the response.

@brendandburns can you get the CI passing so this can be merged?  I'm looking to add support for --ids to these commands and would like to get this PR in beforehand.

@tjprescott recorded tests failed again. These things are annoying to create and super brittle, can you see if you can tell what the problem is?
Thanks

@brendandburns I would suggest creating the lock on a VNet instead of a storage account--it will make it easier to record since VNet name doesn't have to be globally unique.  However, I don't think that is the source of the issue.
There may be an issue with ScenarioTest that is causing this.  You could try writing the test with the older test base (ResourceGroupVCRTestBase) and see if that works. ScenarioTest is newer and there are edge cases the old test base covered that the newer version doesn't.
@troydai do you have any ideas on this one?

We replace the subscription id in the recording to erase credentials. And the test failed because the full zero subscription id gets in the way. I don't know what exactly happen util I can debug into the code.

@tjprescott you requested the scenario test for this PR, if you'd like to merge this, I can move the scenario test out to a different PR...

@brendandburns that sounds perfect.  Please just comment the test out, then we can merge and we can re-enable and debug into it.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@d953a18). Click here to learn what that means.
The diff coverage is 76.92%.

@@           Coverage Diff            @@
##             master   #3591   +/-   ##
========================================
  Coverage          ?   70.9%
========================================
  Files             ?     428
  Lines             ?   27048
  Branches          ?    4119
========================================
  Hits              ?   19179
  Misses            ?    6614
  Partials          ?    1255

Impacted Files
Coverage Δ

...source/azure/cli/command_modules/resource/_help.py
100% <100%> (ø)

...urce/azure/cli/command_modules/resource/_params.py
100% <100%> (ø)

...ource/azure/cli/command_modules/resource/custom.py
62.91% <72.72%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d953a18...b201b31. Read the comment docs.

@tjprescott I think this is ready...

(ah, nm, sorry github hadn't updated to see the merge...)


Prepare user programs for RISCV

Port Biscuit programs to RISCV
Ref: riscv-musl, build pass.
Build Busybox for RISCV
I tried but failed.
Ref: qemu运行riscv linux

@jiegec found a prebuilt binary of Busybox on Debian: busybox-static.
Then we noticed that the syscall ids are different across platforms.

Problem: glibc TLS is not working at the minute.

Syscall id inconsistency is fixed by @jiegec and @equation314

Busybox for RISC-V is available at https://github.com/rcore-os/busybox-riscv-prebuilts

It works now.


It hangs on npm start

It hangs for me on this step. I waited for a few hours then quit the process.

Node v14.16.0
NPM v7.24.0

where is the hang? i see the server is ready & listening for requests
what happens when you navigate to https://localhost:10011?

Sorry, my bad. Was checking wrong port


Missing livereload in alpha 5

Hi.
I'm pretty new in the Ocaml world. So please bear with me if I missed something.
It seems that the last alpha5 package is missing the live reload feature.
I looked into the released commit and it should be available.
https://github.com/aantron/dream/commit/7b46896bf9f9ac7851234d4f3b7e82f8195da671
However, when i downloaded the released package it is missing :

I'm not sure where the issue is. Maybe there is some caching in the CI ?
I hope this helps.
Have a nice day.

Hi! Live reload is in Dream master only for now. Unfortunately, the docs are misleading in this regard, as they were regenerated from master at some point :(

Ok then I close this. Thanks for your work. Cya


Rebuild for R 4.4.1

Comment:
Hello, conda-forge has r-base version 4.4.1, is it possible to build and upload a compatible version of r-essentials? I hope this is the right place to ask. Thank you.

Thanks for the interest! Yes, this is pending and this package will eventually have R 4.4 support. Some core devs are actively addressing several outstanding issues with the r-base feedstock in preparation for rebuilding everything for R 4.4. These will provide general improvements to the Conda R ecosystem. Once those are done then https://github.com/conda-forge/conda-forge-pinning-feedstock/issues/6040 will be merged and R package feedstocks will start migrating to 4.4.

Hello again. I am wondering if you are making progress here. It seems to me the prereq you had mentioned has been completed? Thanks in advance.

This was completed by https://github.com/conda-forge/r-essentials-feedstock/pull/14.


Added Project .NET SDK

Project page for a SDK I've been working on for a little while.

woo, thanks!! @oddvar can you vet & merge please?


Method to increase incorming request size

Method to increase incorming HTTP request size
Description:
I am getting the below error when the incoming request size is greater than 300 MB. The istio proxy sidecar container throws the below error and doesn't allow the request. How to increase the threshold value here.
ERROR:
503 UC upstream_reset_before_response_started{connection_termination}

That error indicates that envoy was proxying the request to an upstream, and the upstream sent back an error. Envoy doesn't limit body size, unless a buffering (or similar) filter is configured.

No filter is configured to limit request body size. But the issue was observed when request size was high and issue reproduced exactly everytime after 2 seconds (from istio proxy sidecar logs). The upstream server is a nodejs application which has its default keepalivetimeout as 5 seconds. No other timeouts have been configured from envoy/istio side. Any way to find and fix this issue ?

The log actually is pretty clear. May be you need to check the server why close the connection.


IMPROVE theme creation

Issue: creating a theme that derived off of light or dark default themes was impractical, and wasn't working quite as users were expecting it to behave.
https://github.com/storybooks/storybook/issues/5824
What I did
ADD ability for users to specify baseTheme, and the theme createFn will merge in the appropriate themeVars.

When I add this line to offcial-storybook/config.js the app hangs:
    theme: create({ base: 'dark', colorPrimary: '#FF4785', colorSecondary: '#1EA7FD' }),

I'm writing tests @shilman

@ndelangen A couple comments, both outside the scope of this PR:

After testing out a buggy version, I had to do localStorage.clear() to get the theme to load. Seems like a bug, but maybe it's a corner case
Seems like the theme serialization should be profiled.. pretty slow to load on my laptop

@shilman
I'd like to remove the need to do theme serialization asap.
theme.color.primary seems to be very rarely used, @domyen came up with the naming and usage of colors.

@ndelangen yea theme.color.primary is not used at all, happy for you to axe it! (the var was mirrored from storybook/frontpage, but we don't need it here)

@domyen @ndelangen seems like an unfortunate choice of names then... I thought the point of this minimal theme var abstraction was to help us preserve backwards compatibility?

We should be making minimal changes at this point though, so I don't really want to change it. Though it makes me cringe a bit.

I’m talking about what’s exposed in the create() function “colorPrimary”.
We still want to have a theme.color.primary as a part of the larger theme
colors (it’s set to storybook pink).
On Mon, Mar 4, 2019 at 12:00 PM Michael Shilman<EMAIL_ADDRESS>wrote:

@domyen https://github.com/domyen @ndelangen
https://github.com/ndelangen seems like an unfortunate choice of names
then... I thought the point of this minimal theme var abstraction was to
help us preserve backwards compatibility?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/storybooks/storybook/pull/5843#issuecomment-469330431,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAQE2S4UQiw8NenGNAS1cndCZ8UwS1NDks5vTVE4gaJpZM4bcCBU
.

@shilman @tmeasday
I did a refactor to ensure the least amount of data needed is transferred over the channel.
I ended up not 'creating' the full theme and then trying to transmit, instead it's much more efficient to transfer the ThemeVars across and construct the actual Theme in the manager.
This means no more hack to make sure the functions make it across, since is a plain flat object.
I know it's not what you were hoping for: it's a bigger change I wish was necessary.

I wrote in such a way, users won't need to change anything.
As long as they are using create()..

I also noticed that setOptions is called a ton, and it's was re-creating the theme on every navigation.
It'd be nice to fix this a bit better, but for now I improved it by deepEqual(theme, updatedTheme).


Section on Ingress Controllers needs table of options

It's hard to understand what the options are, for ingress controllers, because the existing conceptual document doesn't list them all.  I'd like to suggest that it should have a table, something like this:

Type
Name
Comment
Tutorial Link
When to use

Your own controller
Own controller with HTTP only
A controller of your own choice, such as NGINX
https://docs.microsoft.com/en-us/azure/aks/ingress-basic
Not recommended for production use, due to lack of TLS

"
Own controller with internal network
(I don't yet know what this one means myself, but there's a tutorial for it)
https://docs.microsoft.com/en-us/azure/aks/ingress-internal-ip
?

  Or you want to test with a self-signed certificate

"
Own controller with Let's Encrypt cert
A controller of your own choice, with TLS certificates automatically generated by Let's Encrypt
https://docs.microsoft.com/en-us/azure/aks/ingress-tls OR  https://docs.microsoft.com/en-us/azure/aks/ingress-static-ip
You want you own controller, with automatic generation and rotation of TLS certificates

Built-in
Application Gateway Ingress Controller
Azure's native application gateway as an ingress controller. For additional conceptual info see  https://docs.microsoft.com/en-us/azure/application-gateway/ingress-controller-overview
https://docs.microsoft.com/en-us/azure/application-gateway/tutorial-ingress-controller-add-on-existing
You want to leverage built-in Azure capabilites as your ingress controlller OR you are already running the Azure Application Gateway.

"
HTTP Application Routing
Simple built-in Azure capability, suitable for testing. Only works on Services that have a special annotation
https://docs.microsoft.com/en-us/azure/aks/http-application-routing
For simple, non-production testing when you can set the necessary annotation on the service under test (and when you don't need TLS?)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9615c579-305d-2482-7eaf-2d120c75e8d5
Version Independent ID: 064f4a10-bd74-f554-9550-781dd3e838cb
Content: Concepts - Networking in Azure Kubernetes Services (AKS) - Azure Kubernetes Service
Content Source: articles/aks/concepts-network.md
Service: container-service
GitHub Login: @mlearned
Microsoft Alias: mlearned

@JohnRusk
Thanks for the feedback! I have assigned the issue to content author to check and update the document as appropriate.

@mlearned @zr-msft
Can you please check and add your comments on this doc update request as applicable.

@mlearned @zr-msft , any update?

@mlearned @zr-msft , any update?

Thank you @JohnRusk for the feedback.
We've added a task to our backlog and will prioritize accordingly.
#please-close

Thank you @JohnRusk for the feedback.
We've added a task to our backlog and will prioritize accordingly.
#please-close


Feature Request: Values on record iteration

Would you be willing to add a feature that allows the user to iterate json Values for the parser?
My use case is to augment data to the Value without having to re serialize it.

Thanks for adding this. Helps out a lot!


Update release to contain PR #570

Can you make an updated release that contains PR #570?
Our library  has a dependency on mpmath  and our vuln scans fail because the current release does not contain the already merged fix in  #570.  Tagging @rreilly10  to keep him in the loop.

see #550

Issue #550 seems to deal with long term strategic maintenance.  This issue is short term tactical fix.  Or did I miss an implication that no releases will happen until that long term strategy is finalized?

Issue #550 seems to deal with long term strategic maintenance.

Rather it's about there is no maintenance at all for now.

Fixed with 1.3


See https://github.com/JuliaData/CSV.jl/pull/118#discussion_r153446137.

This is a quick fix. Longer-term, there should be another type parameter to say whether the array can contain missing values or not (as for CategoricalArray).

Note that this actually fixes the corruption bug I noted at https://github.com/nalimilan/FreqTables.jl/pull/16. It was due to the fact that the eltype was defined as WeakRefString. After changing the eltype to Union{String, Missing}, WeakRefString objects no longer escape the array and everything is OK.


Don't assume DidChangeTextDocument paths exist

Fixes #5933

My (probably buggy?) acme client causes rust-anayzer to crash a few times per day on this. While clients should do a good job, it is currently trivial to panic rust-analyzer if you send it a ChangeText notification on some random path.

Thank you!
bors r+

Is this correct behavior? Why doesn't he tpath exist? The client needs to gain ownership of the fire first by sending DidOpen which should have a valid URI.

I agree the client should do that. I thought I programmed mine to always send an open first, but apparently it has a bug. However, it should never just panic rust-analyzer.

However, it should never just panic rust-analyzer.

For some historical context, we used to deliberately panic in such cases (to force the us/clients fix all the bugs), but we've changed our stance on this.
That said, I think we need to abstract this away into some kind of log_protocol_violation error, such that we can show an error message to the client or something like that. But that, to be done nicely, requires re-engineering of our event loop I think....


Is it possible to use a model trained on huggingface transformer for getting saliency scores?

Hi,
I wonder if I can use a model trained on huggingface transformer library and get the saliency scores for the token in my dataset? Can I simply modify the code to pass pytorch_model.bin as the args.archive_file instead of tar.gz?
I would appreciate if you could please point me to the right direction.

I have the same question. Did you solve it?

Hi
If you just need gradients, I would suggest using captum.ai library which provides easy access to those. Currently my code modules depend on each other a lot and might end up need a bunch of extra work just to get gradients.


SDK update June 2019

This PR includes the changes for the June release of the Watson Swift SDK.  See the Changelog for the specific features and bug fixes included.

Oh Also we need processingMetrics, processingMetricsInterval, and audioMetrics in the WebSockets recognize

Oh Also we need processingMetrics, processingMetricsInterval, and audioMetrics in the WebSockets recognize

Rats.  Okay ... I'll see if I can get that in quickly.
I think this one is a candidate for reflection. It would be worth calling out changes to the WebSockets interface explicitly in the SDK update issues.

Taj, pls have a look at this commit and let me know if you have any comments or concerns.


Run rake-db migrations programmatically with existing db connection

I would like to run rake-db migrations programmatically, in a beforeAll unit test hook and using the current thread-local transaction.
This is what I would like to work:
beforeAll(async () => {
  await testTransaction.start(db)
  await rakeDb(..., ..., ["migrate"]).promise // just quick testing, I know this is wrong
})

The above code kinda works, but it opens a new database connection, so the tables are not rolled back on testTransaction.close(). I don't see a suitable low-level export in rake-db, moreover I can see that creating the connection is heavily coupled with running the migrations:
https://github.com/romeerez/orchid-orm/blob/cfef93300a6154bc82a7337cebf221f50d42d404/packages/rake-db/src/commands/migrateOrRollback.ts#L57-L69
I think this code could be extended with accepting a readily available db.$adapter instead of AdapterOptions? (Better yet, somehow refactored so that iterating adapters and running migrations through one of the adapters are separate).

I know that the documentation suggests to keep the test database in sync with the development one by providing the array of adapter options to rakeDb during migrations. However, I think the approach above also makes sense. It gives a small overhead during tests, but it reduces the pain of synchronizing, and it also makes sure the migrations are always in a good standing.

That's requiring quite a lot of changes and will take some time.
But, why not use it as designed? So that db migrate will migrate both the dev and test dbs, db rollback will roll back both dbs, and when you run a test the test db is ready to be used.
The main idea and reason for test transactions is to make tests run "blazingly" fast, it's the fastest and simplest way to use a real database in tests, IMO.
With your approach, if you will have, let's say, a hundred test suites, and a hundred migrations, that means running 10k migrations each time when you run all tests, much more work for Postgres, now it will have not only the data but the db structure in memory. It's definitely going to be heavier and slower, but what are the gains?

I actually do have 227 migrations with 48 tests in one of the projects which is more than 10k migrations per test run. That is with Knex/Objection where I didn't have testTransaction so the hooks were creating a new db, running migrations, and destroying the db after each test. At some point I replaced repetitive migrations with executing a single schema.sql (due to the strive for premature optimization, ha-ha) but the gains were very subtle. Overall, in my experience preparing the database(s) only takes a very small fraction of time in the whole test suite run.
As for the benefits, the primary benefit I see is that one does not have to deal with synchronization. Based on my experience, every now and then I f..k up with migrations (especially with non-trivial rollbacks involving data migrations) and have to perform manual maintenance on the local database. Imagine twice the efforts..with a chance of a different mistake... ugh. I tend to think about 'test environment' as of something transient which should require zero efforts.
Anyhow, I understand that it's requiring a lot of changes and is thus not worth it indeed. It's not a big deal. Similar to Knex, I went with using schema.sql with Orchid (generated in afterMigrate, executed in beforeAll with db.$query), that works well.

Imagine twice the efforts..with a chance of a different mistake

But why is it doubling the efforts?
If migration throws error, it's not persisted, both dbs aren't changed.
If migration passes, but you want to edit it, db rollback will rollback both dbs, then do you change, and db migrate will apply it to both dbs.
So it should act and feel like you're managing a single database. And indeed test database requires zero effort for maintaining. This approach worked fine for me on a work project. How it complicates things for you?

A lot of changes and is thus not worth it indeed.

I don't mind doing it, though I'm fine not doing it as well. The change would slightly simplify rakeDb so it can accept a db instance that the user already has instead of connection options. And the fact you could have more control of it is a good thing in general.

The problem I'm usually facing is that the down migration doesn't always fully and consistently rollback. If the up migration is destructuring, the down migration is supposed to put things back (e.g. re-create a table), and sometimes the developer fails to define that properly. They end up with a migration that has been supposedly rolled back yet the schema is broken and needs manual and sometimes non-trivial updates, which will be double work.


SOLR-17457: Backport fix for max-wait-secs

https://issues.apache.org/jira/browse/SOLR-17457
Description
This PR backports the changes from https://github.com/apache/solr/pull/2717 to branch_9_7.
Checklist
Please review the following and check all that apply:

[X] I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
[X] I have created a Jira issue and added the issue ID to my pull request title.
[X] I have given Solr maintainers access to contribute to my PR branch. (optional but recommended, not available for branches on forks living under an organisation)
[ ] I have developed this patch against the main branch.
[ ] I have run ./gradlew check.
[ ] I have added tests for my changes.
[ ] I have added documentation for the Reference Guide

Thanks @malliaridis for the back port to branch_9_7.


Redeploy gh action

This PR introduces a GitHub workflow in .github/workflows/redeploy.yaml that, when triggered, does the following:

Uses the service account credentials stored in the repo secret JSON_GCLOUD_SERVICE_ACCOUNT_JSON to authenticate to the service account<EMAIL_ADDRESS>Stops the VM ontotools-stack, resizes it to an e2-highmem-4 (configurations with less memory have been found to fail), then starts it again.
SSH's into the VM as the user ols_deployer and runs the script pull_and_redeploy.sh. That script first does a git pull in the folder /var/checkouts/ontotools-docker/ on whatever branch is currently checked out (ideally main), then runs the redeploy.sh script. The most recent runs have taken ~40 minutes, but this will vary depending on the input ontologies, I imagine.
Regardless of the prior step's results or if the action is cancelled, the VM is sized back down to an e2-medium to save on runtime costs. This behavior is enabled through the if: always() clause on the "Size down OLS machine for serving" step.

Things to address:

At the moment, if the workflow is triggered again while a prior run is still executing, it will interrupt the previous run. We might consider adding the "Skip Concurrent Workflows" action to address that  (see https://github.com/marketplace/actions/skip-duplicate-actions#skip-concurrent-workflow-runs to block it from re-running or https://github.com/marketplace/actions/skip-duplicate-actions#cancel-outdated-workflow-runs to cancel the prior run)
Longer-term improvement ideas, observations:

We should probably figure out what build artifacts the redeploy.sh script is producing and build them elsewhere, i.e. on a Jenkins server or a building container if we move to Kubernetes. We could then deploy the resulting artifacts to the serving VM and restart it, which should theoretically be quick. That'd also avoid having to resize the serving VM, since it won't have to be large enough to run the build process anymore.

So cooool... Didnt review, but this is fantastic! @twhetzel i have merged my other branch, so we can test this now if possible.


[T4A3][W10-B3]Vincent Seng

Ready for review

Some comments added. Please close the PR after reading comments.