load_dataset for LOCAL CSV files report CONNECTION ERROR

load_dataset for LOCAL CSV files report CONNECTION ERROR

Description:
A local demo csv file:

import pandas as pd
import numpy as np
from datasets import load_dataset
import torch
import transformers

df = pd.DataFrame(np.arange(1200).reshape(300,4))
df.to_csv('test.csv', header=False, index=False)

print('datasets version: ', datasets.__version__)
print('pytorch version: ', torch.__version__)
print('transformers version: ', transformers.__version__)

# output:
datasets version:  1.1.2
pytorch version:  1.5.0
transformers version:  3.2.0

when I load data through dataset:
dataset = load_dataset('csv', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)

Error infos:
ConnectionError                           Traceback (most recent call last)
<ipython-input-17-bbdadb9a0c78> in <module>
----> 1 dataset = load_dataset('csv', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
    588     # Download/copy dataset processing script
    589     module_path, hash = prepare_module(
--> 590         path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
    591     )
    592

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
    266         file_path = hf_github_url(path=path, name=name, dataset=dataset, version=script_version)
    267         try:
--> 268             local_path = cached_path(file_path, download_config=download_config)
    269         except FileNotFoundError:
    270             if script_version is not None:

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/utils/file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
    306             user_agent=download_config.user_agent,
    307             local_files_only=download_config.local_files_only,
--> 308             use_etag=download_config.use_etag,
    309         )
    310     elif os.path.exists(url_or_filename):

~/.conda/envs/py36/lib/python3.6/site-packages/datasets/utils/file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag)
    473         elif response is not None and response.status_code == 404:
    474             raise FileNotFoundError("Couldn't find file at {}".format(url))
--> 475         raise ConnectionError("Couldn't reach {}".format(url))
    476
    477     # Try a second time

ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py

And I try to connect to the site with requests:
import requests

requests.head("https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py")

Similarly Error occurs:
---------------------------------------------------------------------------
ConnectionRefusedError                    Traceback (most recent call last)
~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connection.py in _new_conn(self)
    159             conn = connection.create_connection(
--> 160                 (self._dns_host, self.port), self.timeout, **extra_kw
    161             )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/util/connection.py in create_connection(address, timeout, source_address, socket_options)
     83     if err is not None:
---> 84         raise err
     85

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/util/connection.py in create_connection(address, timeout, source_address, socket_options)
     73                 sock.bind(source_address)
---> 74             sock.connect(sa)
     75             return sock

ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

NewConnectionError                        Traceback (most recent call last)
~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    676                 headers=headers,
--> 677                 chunked=chunked,
    678             )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    380         try:
--> 381             self._validate_conn(conn)
    382         except (SocketTimeout, BaseSSLError) as e:

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
    975         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
--> 976             conn.connect()
    977

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connection.py in connect(self)
    307         # Add certificate verification
--> 308         conn = self._new_conn()
    309         hostname = self.host

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connection.py in _new_conn(self)
    171             raise NewConnectionError(
--> 172                 self, "Failed to establish a new connection: %s" % e
    173             )

NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f3cceda5e48>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
~/.conda/envs/py36/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    448                     retries=self.max_retries,
--> 449                     timeout=timeout
    450                 )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    724             retries = retries.increment(
--> 725                 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
    726             )

~/.conda/envs/py36/lib/python3.6/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    438         if new_retry.is_exhausted():
--> 439             raise MaxRetryError(_pool, url, error or ResponseError(cause))
    440

MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/1.1.2/datasets/csv/csv.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f3cceda5e48>: Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
<ipython-input-20-18cc3eb4a049> in <module>
      1 import requests
      2
----> 3 requests.head("https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py")

~/.conda/envs/py36/lib/python3.6/site-packages/requests/api.py in head(url, **kwargs)
    102
    103     kwargs.setdefault('allow_redirects', False)
--> 104     return request('head', url, **kwargs)
    105
    106

~/.conda/envs/py36/lib/python3.6/site-packages/requests/api.py in request(method, url, **kwargs)
     59     # cases, and look like a memory leak in others.
     60     with sessions.Session() as session:
---> 61         return session.request(method=method, url=url, **kwargs)
     62
     63

~/.conda/envs/py36/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    528         }
    529         send_kwargs.update(settings)
--> 530         resp = self.send(prep, **send_kwargs)
    531
    532         return resp

~/.conda/envs/py36/lib/python3.6/site-packages/requests/sessions.py in send(self, request, **kwargs)
    641
    642         # Send the request
--> 643         r = adapter.send(request, **kwargs)
    644
    645         # Total elapsed time of the request (approximately)

~/.conda/envs/py36/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    514                 raise SSLError(e, request=request)
    515
--> 516             raise ConnectionError(e, request=request)
    517
    518         except ClosedPoolError as e:

ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/1.1.2/datasets/csv/csv.py (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f3cceda5e48>: Failed to establish a new connection: [Errno 111] Connection refused',))

Hi !
The url works on my side.
Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to raw.githubusercontent.com ?

Hi !
The url works on my side.
Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to raw.githubusercontent.com ?

I tried another server, it's working now. Thanks a lot.
And I'm curious about why download things from "github" when I load dataset from local files ?  Dose datasets work if my network crashed?

It seems my network frequently crashed so most time it cannot work.

Hi !
The url works on my side.
Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to raw.githubusercontent.com ?

I tried another server, it's working now. Thanks a lot.
And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?

I download the scripts https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py and move it to the package dir */datasets/ solved the problem. Could you please put the file datasets/datasets/csv/csv.py to datasets/src/datasets/？
Thanks :D

hello, how did you solve this problems?

Hi !
The url works on my side.
Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to raw.githubusercontent.com ?

I tried another server, it's working now. Thanks a lot.
And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?

I download the scripts https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py and move it to the package dir */datasets/ solved the problem. Could you please put the file datasets/datasets/csv/csv.py to datasets/src/datasets/？
Thanks :D

hello, I tried this. but it still failed. how do you fix this error?

你把那个脚本下载到你本地安装目录下，然后 load_dataset(csv_script_path, data_fiels)

hello, how did you solve this problems?

Hi !
The url works on my side.
Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to raw.githubusercontent.com ?

I tried another server, it's working now. Thanks a lot.
And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?

I download the scripts https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py and move it to the package dir */datasets/ solved the problem. Could you please put the file datasets/datasets/csv/csv.py to datasets/src/datasets/？
Thanks :D

hello, I tried this. but it still failed. how do you fix this error?

你把那个脚本下载到你本地安装目录下，然后 load_dataset(csv_script_path, data_fiels)

hello, how did you solve this problems?

Hi !
The url works on my side.
Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to raw.githubusercontent.com ?

I tried another server, it's working now. Thanks a lot.
And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?

I download the scripts https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py and move it to the package dir */datasets/ solved the problem. Could you please put the file datasets/datasets/csv/csv.py to datasets/src/datasets/？
Thanks :D

hello, I tried this. but it still failed. how do you fix this error?

你把那个脚本下载到你本地安装目录下，然后 load_dataset(csv_script_path, data_fiels)

好啦好啦！感谢感谢！我还有一个问题，trainer.evaluate一直OOM，你遇到过吗？

hello, how did you solve this problems?

Hi !
The url works on my side.
Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to raw.githubusercontent.com ?

I tried another server, it's working now. Thanks a lot.
And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?

I download the scripts https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py and move it to the package dir */datasets/ solved the problem. Could you please put the file datasets/datasets/csv/csv.py to datasets/src/datasets/？
Thanks :D

hello, I tried this. but it still failed. how do you fix this error?

你把那个脚本下载到你本地安装目录下，然后 load_dataset(csv_script_path, data_fiels)

好的好的！解决了，感谢感谢！！！

hello, how did you solve this problems?

Hi !
The url works on my side.
Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to raw.githubusercontent.com ?

I tried another server, it's working now. Thanks a lot.
And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?

I download the scripts https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py and move it to the package dir */datasets/ solved the problem. Could you please put the file datasets/datasets/csv/csv.py to datasets/src/datasets/？
Thanks :D

hello, I tried this. but it still failed. how do you fix this error?

你把那个脚本下载到你本地安装目录下，然后 load_dataset(csv_script_path, data_fiels)

我照着做了，然后报错。
ValueError: unable to parse C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets\dataset_infos.json as a URL or as a local path
`---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
 in
----> 1 dataset = load_dataset('C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets/csv.py', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)
C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
588     # Download/copy dataset processing script
589     module_path, hash = prepare_module(
--> 590         path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
591     )
592
C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
296         local_dataset_infos_path = cached_path(
297             dataset_infos,
--> 298             download_config=download_config,
299         )
300     except (FileNotFoundError, ConnectionError):
C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\utils\file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
316     else:
317         # Something unknown
--> 318         raise ValueError("unable to parse {} as a URL or as a local path".format(url_or_filename))
319
320     if download_config.extract_compressed_file and output_path is not None:
ValueError: unable to parse C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets\dataset_infos.json as a URL or as a local path
`

hello, how did you solve this problems?

Hi !
The url works on my side.
Is the url working in your navigator ?
Are you connected to internet ? Does your network block access to raw.githubusercontent.com ?

I tried another server, it's working now. Thanks a lot.
And I'm curious about why download things from "github" when I load dataset from local files ? Dose datasets work if my network crashed?

I download the scripts https://raw.githubusercontent.com/huggingface/datasets/1.1.2/datasets/csv/csv.py and move it to the package dir */datasets/ solved the problem. Could you please put the file datasets/datasets/csv/csv.py to datasets/src/datasets/？
Thanks :D

hello, I tried this. but it still failed. how do you fix this error?

你把那个脚本下载到你本地安装目录下，然后 load_dataset(csv_script_path, data_fiels)

我照着做了，然后报错。
ValueError: unable to parse C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets\dataset_infos.json as a URL or as a local path
`---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
 in
----> 1 dataset = load_dataset('C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets/csv.py', data_files='./test.csv', delimiter=',', autogenerate_column_names=False)
C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, save_infos, script_version, **config_kwargs)
588     # Download/copy dataset processing script
589     module_path, hash = prepare_module(
--> 590         path, script_version=script_version, download_config=download_config, download_mode=download_mode, dataset=True
591     )
592
C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\load.py in prepare_module(path, script_version, download_config, download_mode, dataset, force_local_path, **download_kwargs)
296         local_dataset_infos_path = cached_path(
297             dataset_infos,
--> 298             download_config=download_config,
299         )
300     except (FileNotFoundError, ConnectionError):
C:\Software\Anaconda\envs\ptk_gpu2\lib\site-packages\datasets\utils\file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
316     else:
317         # Something unknown
--> 318         raise ValueError("unable to parse {} as a URL or as a local path".format(url_or_filename))
319
320     if download_config.extract_compressed_file and output_path is not None:
ValueError: unable to parse C:/Software/Anaconda/envs/ptk_gpu2/Lib/site-packages/datasets\dataset_infos.json as a URL or as a local path
`

I also experienced this issue this morning. Looks like something specific to windows.
I'm working on a fix

I also experienced this issue this morning. Looks like something specific to windows.
I'm working on a fix

I opened a PR @wn1652400018

I opened a PR @wn1652400018

I opened a PR @wn1652400018

Thanks you!, It works very well.

I opened a PR @wn1652400018

Thanks you!, It works very well.


Delete all content from unpartitioned table

Run delete from an_unpartitioned_table, it should either fail or delete all content. Right now, it completes but doesn't do anything.

may I claim this one?
seems like delete is optimized to metadataDelete, and in metadataDelete it only deletes partitions
I think exposing partitions in connector is not in the plan, is it?
How about drop the table and delete the data, then create an empty table?

This a correctness issue. We need to fix it before releasing the new delete code.

@zhenxiao Your understanding of how delete works right now is correct. Sorry the title was misleading. When I filed the task, I didn't have anything concrete in my mind on which direction the fix should be (make the unpartitioned delete work, or make it fail loudly). It seems like there is no easy solution to make the unpartitioned delete work. Therefore, I implemented a fix to make it fail loudly.
The concept of partition was removed from presto-main in #2628. If you are wondering whether we want to reintroduce it, the answer is know. If I misunderstood your question, feel free to ask again.

@haozhun thank you for the explanation. Sure. Will ping you when I have other questions.

To implement this, can't we just walk the directory and delete the data files like we do in the rollback methods?

@dain If we are going to implement that, should we implement row-by-row delete instead.

@zhenxiao Go ahead and implement it if you are still interested. What @dain suggested seems to be a good solution.

Thanks @dain @haozhun. sure. I will do it following @dain 's suggestion

Make sure to skip the hidden files.  IIRC they start with . or _.

@haozhun @dain get a PR:
https://github.com/facebook/presto/pull/3794
Your comments are appreciated


feature(distinctUntilSomeChanged)

implemented distinctUntilSomeChanged operator. Added some docs and implemented tests (based off of distinctUntilKeyChanged from rxjs.

We also need to update the docs with this operator

We also need to update the docs with this operator

i have added a snippet . #84


Add an initial Dockerfile 🙃

This is using ubi9/go-toolset as base. This is using multi-stage to
reduce layers size (not shipping go caches used to build the ko binary).
Signed-off-by: Vincent Demeester<EMAIL_ADDRESS>

/ok-to-test

/ok-to-test


Feature: Progress wave

I would like to create an audio wave during recording audio.
It would be cool to be able to put as sample a UInt8List as provided by flutter_sound or even pass a StreamController which gives the data. Instead of the maxDuration and the elapsedDuration it could have only maxDuration which defines how much of the recorded stream is shown.

That's an interesting idea. But, that would add an additional layer of responsibility/dependancy to the package and I believe would steer away from the definition of a UI library which I want this package to be. Letting the package handle only one thing is much better for users and to the people who maintain the package. Updating, adding new features is much easier this way if we have to deal with only single purpose. It's less error prone also.
What you're suggesting can be an independant package though like flutter_sound as you say that would get the raw data for incoming audio.
Though, atm flutter_audio_waveforms doesn't have proper implementation for real-time audio waveform creation. But it's definitely one of the interesting features on the roadmap.🙌

That's an interesting idea. But, that would add an additional layer of responsibility/dependancy to the package and I believe would steer away from the definition of a UI library which I want this package to be. Letting the package handle only one thing is much better for users and to the people who maintain the package. Updating, adding new features is much easier this way if we have to deal with only single purpose. It's less error prone also.
What you're suggesting can be an independant package though like flutter_sound as you say that would get the raw data for incoming audio. Though, atm flutter_audio_waveforms doesn't have proper implementation for real-time audio waveform creation. But it's definitely one of the interesting features on the roadmap.🙌

Ohk! I gotta take back the last statement that flutter_audio_waveforms can't do it!😅 It doesn't matter how the samples are coming in! The package just uses samples and draws the waveform based on that. So if samples are rapidly changing as in case of real-time audio then it will handle it just as usual.
Here's an example that uses real-time audio stream to visualize the audio with waveforms.
https://github.com/rutvik110/flutter_audio_waveforms/tree/experimentation
https://user-images.githubusercontent.com/65209850/147869903-a3b65c3a-176e-481a-9b30-d3b6c397e85a.mov

I believe

I would like to create an audio wave during recording audio. It would be cool to be able to put as sample a UInt8List as provided by flutter_sound or even pass a StreamController which gives the data. Instead of the maxDuration and the elapsedDuration it could have only maxDuration which defines how much of the recorded stream is shown.

I believe, the above example solves the Progressive wave implementation. And also as you're assuming that maxDuration is responsible for showing the waveform but that's not correct. Samples are used to draw the InActiveWaveform. maxDuration and elapsedDuration are basically used to calculate the activeSamples(*internally defined) from the samples that are used to draw ActiveWaveform.
I hope this clears you doubts on how you can use this package to implement your use case. So for now I'm closing the issue.


[CT-1617] Materialization to create external tables (Redshift first)

Describe the feature
When using Redshift Spectrum, dbt is able to stage external tables stored in S3 and do useful things such as declaring the source data format (csv, json, parquet, etc.). The idea is to create a new materialization (potentially "external" would work) option which would handle persisting query results and enable a few capabilties:

Specifying a location to output partitioned results generated by a model's sql
Specifying the output format (csv, json, parquet) and compression where applicable
Control over the partitioning logic (e.g. use a defined set of columns to create partitions in the lake)
For Redshift, ability to CLEAR PATH to overwrite existing partitions in the lake

Describe alternatives you've considered
The implementation of create_external_table here accomplishes this when triggered by a run-operation. The goal here is to make that logic a materialization so that it can become part of the dbt run pipeline.
Additional context
Believe this is relevant for any of the databases currently supported in the external tables package:

Redshift (Spectrum)
Snowflake
BigQuery
Spark
Synapse
Azure SQL

Who will this benefit?
dbt Users who have existing infrastructure that leverages a more data lake centric approach for managing persistence will benefit from this. They can use dbt and the warehouse as an ephemeral compute / transform layer, and then persist the data to a file store, which enables other tools (e.g. AWS Glue / Athena) to query the results using existing analytical patterns.
Are you interested in contributing this feature?
Yes.

@jtcohen6 @matt-winkler my team is investigating building this feature for our own needs, and would be interested in learning about whether we could contribute it back. I've done some digging & experimentation on approach, isolated a few definite next steps, & narrowed down the final approach to two primary options. Would you mind reviewing and weighing in on (a) which option you suggest and (b) whether you'd be open to this getting merged in?
Motivation

My team wants to store model results outside the database (in S3).
dbt, on the other hand, expects to "... [only do] the T in ELT (Extract, Load, Transform) processes – it doesn’t extract or load data, but it’s extremely good at transforming data that’s already loaded into your warehouse." (What is dbt? | dbt Docs)
Redshift lets us pretend that data in S3 is "in the warehouse".

But it has some important differences from standard tables.
Those differences make it not (yet) compatible with dbt for model output materialization.
Can we make it compatible?

Context
Sharing to make sure I'm understanding correctly:
dbt materializations

A dbt materialization is a registered^ adapter-specific^ Jinja macro^.

^registered → declared via {% materialization <name>… %} instead of {% macro <name>… %}
^adapter-specific → different database adapters can have different implementations, e.g.
{% materialization <name>, default %} … {% endmaterialization %}
{% materialization <name>, adapter='redshift' %} ... {% endmaterialization %}
^macro → compiles down to a function with parameters (incl. "context") & return value

Called by dbt run for each model with Jinja context that includes:

sql → compiled-to-string select statement for the current model
pre_hooks / post_hooks → compiled-to-string SQL for current model's pre & post hooks

"Materializations generally take the following shape:"

[1.] Prepare the database for the new model [...]
[2.] Execute any sql required to implement the desired materialization [...]
[3.] Clean up the database as required [...]

Creating new materializations # Anatomy of a materialization | dbt Docs

Naive table materialization
A (hypothetical) naive table materialization might look like:

[1.] Prepare the database for the new model [...]
drop table if exists [m]

[2.] Execute any sql required to implement the desired materialization [...]
create table [m] as (sql)

[3.] Clean up the database as required [...]
// no-op

However this would have some problems:

Prior data unavailable while running CTAS [m].
(Could be a while if m is big.)
Prior data unavailable until manually restored/rerun if CTAS [m] fails.
(Could be a long while, regardless of size of m.)

Pseudo-transactional table materialization
(Hypothetical) improvement:

[1.] Prepare the database for the new model [...]
drop table if exists [m_tmp] &&
drop table if exists [m_bak]

[2.] Execute any sql required to implement the desired materialization [...]
create table [m_tmp] as (sql)
&& alter table [m] rename to [m_bak]
&& alter table [m_tmp] rename to [m]

[3.] Clean up the database as required [...]
drop table if exists [m_bak]

This would resolve the first two problems:

X Prior data only unavailable briefly, for duration of table rename.
X Prior data remain in place if create table [m_tmp] as (sql) fails.

but introduces one of its own:

Leaves garbage behind if create table [m_tmp] as (sql) fails.

Current actual table materialization
According to my reading of the source, the built-in table materialization as implemented today essentially does the following:

[1.] Prepare the database for the new model [...]
drop table if exists [m_tmp] &&
drop table if exists [m_bak]

[2.] Execute any sql required to implement the desired materialization [...]
In transaction: [
create table [m_tmp] as (sql)
&& alter table [m] rename to [m_bak]
&& alter table [m_tmp] rename to [m]
]

[3.] Clean up the database as required [...]
drop table if exists [m_bak]

This addresses all three problems:

Prior data only unavailable briefly, for duration of table rename.
Prior data remain in place if create table [m_tmp] as (sql) fails.
No garbage left behind if create table [m_tmp] as (sql) fails.

Transactional "external table" materialization?

[1.] Prepare the database for the new model [...]
drop table if exists [m_tmp] && drop table if exists [m_bak]

[2.] Execute any sql required to implement the desired materialization [...]
In transaction: [
create external table [m_tmp] as (sql)
&& alter external table [m] rename to [m_bak]
&& alter external table [m_tmp] rename to [m]
]

[3.] Clean up the database as required [...]
drop table if exists [m_bak]

Unfortunately, this doesn't work because:

attempting to "create external table as" in a transaction leads to ERROR: CREATE EXTERNAL TABLE cannot run inside a transaction block, ruling out fully transactional external table creation; and
attempting to rename an external table in any context leads to ERROR: Unsupported operation. [Cannot rename external tables.], ruling out even pseudo-transactional external table creation.

What options do we have?
Option 0. Naive materialization. Accept availability constraints.
Less than ideal; we would like to find a better way.
Option 1. Pseudo-transactional via rename = CET w/ same config && delete old.
Use the external-table-config-DDL-generation script (seemingly written by someone at AWS) to "rename" external tables by creating external table with same config at the new name (not create external table as; just create external table). Manually clean up failures in on-run-end hook.

Pros: No on-cluster storage used.
Cons: Seems very fragile. Generated DDL could break. DDL generation script could break. Underlying views could break. Unclear maintenance ownership.

→ We're not comfortable taking on that maintenance burden / risk. Pretty sure another option would be better.
Option 2a. "Transactional", via table materialization + on-run-end hook
Literally configure the models to materialize as table, then add an on-run-end hook to "create external table as select * from {{ model }}" + "drop table {{ model }}" for every model that got built.
Pros: Simple. No change to materializations.
Cons: dbt manifest's recorded model output locations would be inaccurate, which precludes partial rebuilds and inter-project dependencies. Also, the cluster has to have sufficient capacity to store all model results during the run -- though this storage is reclaimed after the on-run-end hook.
Option 2b. "Transactional", via front-loading option 2a logic into per-model materialization
So the steps would be:

[1.] Prepare the database for the new model [...]
drop table if exists [m_tmp] [in on-cluster schema]

[2.] Execute any sql required to implement the desired materialization [...]
CTAS [m_tmp] [in on-cluster schema]
&& drop table if exists [m] [in external schema]
&& CET [m] [in external schema] as select * from [m_tmp]

[3.] Clean up the database as required [...]
drop table if exists [m_tmp] [from on-cluster schema]

I assume that in order for dbt's manifest to be accurate, the schema returned by generate_schema_name would need to refer to the final destination, correct? In which case, for this option, the "schema" for a model as declared by generate_schema_name should refer to an already-existing external schema. And we would probably specify the name of the temporary on-cluster schema (used for its temporary transactional guarantees) via a model configuration.
Obviously, this option introduces substantial added complexity. But it gets you more "correctness" -- future partial dbt runs would find the outputs in the correct spot, dbt docs' schema names would be accurate, etc.
Option 2c. "Transactional", via 2b but with private per-session temp tables instead of real on-cluster schema?
This option just occurred to me as I was preparing to post this comment! This would obviate the need for the model configuration & having an extra schema lying around.
Are there limitations I'm not aware of that would prevent this approach from working?
Option 3. Pseudo-transactional at schema level
This option would be the most different from what dbt does currently, but it's what I've personally been interested in for a while. Essentially, can we broaden the scope of dbt's "create as temp" + "rename on success" approach to apply at the schema level instead of the individual table level?
Consider the following:

on-run-start = create external schema if not exists [s_tmp]
materialization impl = naive, into [s_tmp] (won't even have to "drop table [m]" though since s_tmp is fresh for this run!)
on-run-end =
on success → drop schema if exists cascade [s] && rename schema [s_tmp → s]
on failure → drop schema if exists cascade [s_tmp]  (or keep it around for troubleshooting if you want!)

This would require:

configurations for S3 location, file type, IAM role, etc. - presumably in model configs?
permission to create new external schemas and new Glue catalogs

Things we'll want to do regardless of approach

Add an adapter configuration parameter to allow setting connection.set_session(autocommit=True) in dbt-postgres connection class? Not sure if that's the right place for this, but that setting will be very helpful to not have to constantly commit; our way out of psycopg2-auto-started transactions.
Would you be open to merging something like this?

Fix dbt-labs/dbt#2289. I think I've got a workable solution for this.

Figure out how to clean up prior runs' data out of S3. The only SQL-runnable option for doing this that I've found is Redshift's UNLOAD statement's CLEANPATH option, with which we could unload an empty select statement to clear an S3 prefix. (Even that leaves a single metadata object in the S3 path though, so we'd still need to use new prefixes for every run. But at least we wouldn't be leaving around whole datasets to store indefinitely.)
Alternatively, we could just run a recurring cleanup job out of band from dbt runs.

Re-opening as #175 has brought the topic back on top (#175 was closed as duplicate).

Our current rule to decide how to solve these issues is:
Can this problem be solved in user land? Is there a decent work around?
Here it seems like yes, as demonstrated in #175:

"materializing the model as view and then use a post hook to create an external table from the view."

That's not the most seamless experience, but it works.
If not, can this problem be solved via a contribution?
You can create your own materialization, as has been discussed in this thread.
Now we need to look again at the 3 objections @vergenzt you brought up at the end of the immensely useful "wall of text" above. Are these absolutely blocking? What can we do about them?
If not, then maybe dbt Labs should solve it
Which here we are considering, not in itself but to support longer plans toward cross-warehouse projects (we will need all sorts of external tables for that) for 1.5+.
A lot of hypothetical here, and we need to think about a common interface for external tables across all warehouses to target most warehouses. I'm sure we'll have discussions on the topic when time comes.

Thanks for the update @vergenzt.
Do you mind sharing why you pivoted away from external tables, and for what alternative? That is if it's not really specific to your own situation/context. Just wondering if your new approach could be recommended to others :)

Hi @Fleid! I've since moved teams so not sure if this is still accurate, but I believe the new plan was to use Databricks/Spark more broadly rather than trying to force external table materialization into dbt-redshift.

@dataders FYI

@dataders opened a Discussion about adding external table capabilities into dbt Core: https://github.com/dbt-labs/dbt-core/discussions/8617


Investigate building connectors for go-cloud

With the release of https://github.com/google/go-cloud we should investigate making connectors so that we can migrate over.

Opened a PR #240 on go-cloud to add modification time to the driver.ObjectAttrs struct

Started working on this here #5


Cleanup dask dependency

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[n/a] Reset the build number to 0 (if the version changed)
[n/a] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[n/a] Ensured the license file is being packaged.

Thank you @jakirkham for the review. What does it need to handle upstream? As I understand the conda-forge dask-core package is the equivalent of installing dask without any extras:
dask: https://github.com/dask/dask/blob/461355e117958354ed30e29893afecd5e9258ff0/pyproject.toml#L29-L45
dask-core-feedstock: https://github.com/conda-forge/dask-core-feedstock/blob/e3cfc3964322f9421c7eed04af30d8e8924edddb/recipe/meta.yaml#L26-L35
The equivalent of the conda-forge dask package is dask['complete'], the conda-forge dask package has more dependencies (cytoolz and bokeh):
https://github.com/dask/dask/blob/461355e117958354ed30e29893afecd5e9258ff0/pyproject.toml#L65-L68
https://github.com/conda-forge/dask-feedstock/blob/c4a13519927e95614a24285ebb06916dd9b369fc/recipe/meta.yaml#L22-L31

Referring to dask-cuda's source as upstream (as indicated by the link above)

Sorry, I don't understdand what needs to be change upstream: dask-cuda have dask (not dask['complete']) as dependency and this would translate to use dask-core in conda-forge.
For example, when installing dask-cuda from pypi, pyarrow and bokeh will not be installed, while it will be installed when installing from conda-forge. This is why dask-core need to be used instead of dask in conda-forge.

Could you please start by raising this as an issue on the dask-cuda repo? Think this will be easier to discuss there

Done!


Springfox doesn't register filter

Hi all,
I have problem with register Filter with Springfox,
My Filter:
public class JwtFilter extends GenericFilterBean {

    @Override
    public void doFilter(final ServletRequest req, final ServletResponse res, final FilterChain chain)
            throws IOException, ServletException {
        final HttpServletRequest request = (HttpServletRequest) req;

        String authHeader = request.getHeader(AUTH_HEADER_KEY);
        if (authHeader == null || !authHeader.startsWith("Bearer ") || authHeader.split(" ").length != 2) {
            throw new ServletException(AUTH_ERROR_MSG);
        }

        try {
            final Claims claims = AuthUtil.decodeToken(authHeader);
            request.setAttribute("claims", claims);
        } catch (final SignatureException e) {
            throw new ServletException(SIGNATURE_INVALID_MSG);
        } catch (final ExpiredJwtException e) {
            throw new ServletException(EXPIRE_ERROR_MSG);
        } catch (final UnsupportedJwtException e) {
            throw new ServletException(JWT_ERROR_MSG);
        } catch (final MalformedJwtException e) {
            throw new ServletException(JWT_INVALID_MSG);
        }
        chain.doFilter(req, res);
    }

}

and Application.java
    @Bean
    public FilterRegistrationBean jwtFilter() {
        final FilterRegistrationBean registrationBean = new FilterRegistrationBean();
        registrationBean.setFilter(new JwtFilter());
        registrationBean.addUrlPatterns("/alepay/*");
        registrationBean.setOrder(1);

        return registrationBean;
    }

But when i run my project, it's not Filter with JwtFilter.
Could you tell me where my mistake and how to fix it? Thanks

I am also interested in this question.

Maybe your url pattern isn't matching


Run z3 in parallel in a naive way

This change adds a 'satpar' stage which tries to run z3 in parallel using a work-pool model. It works but doesn't run any faster. I suspect what's going on is that the overhead of recreating the contexts is drowning out any advantage from parallelism.
I also added a file spinLockStress.cvf which just generates lots of z3 clauses. It's just a sequence of lock / unlocks, derived from spinLock.cvf.
Not sure if this is worth merging at this point, but it might be worth putting in and then coming back to at some point in the future.

Should this be merged @septract ?

I think you can leave it floating here - probably not worth merging since it doesn't improve performance.

I can't see this getting merged any time soon so I'm closing it.


feature: enhance Text.contains("word")

implement feature #1649

support entire word match Text.contains("(word)") and Text.contains("word1|word2|word3")
Text.contains("word") also contain word == propValue

TODO: update search index doc https://hugegraph.github.io/hugegraph-doc/clients/hugegraph-client.html


The title of the quickfix window is sometimes prefixed by an additional colon

vim --version:      VIM - Vi IMproved 8.0  Included patches: 1-1789
Operating system:     Ubuntu 16.04.4 LTS
Terminal emulator:    rxvt-unicode v9.22
Terminal multiplexer: tmux v2.7
$TERM:              tmux-256color
Shell:                zsh 5.3.1

Here's a MWE:
    fu! Replace_qfl() abort
        call setqflist([], 'r',
                     \ {'items': [{ 'filename': $VIMRUNTIME.'/filetype.vim',
                     \              'lnum': 1,
                     \              'valid': 1 }],
                     \  'title': 'some title',
                     \ })
    endfu

    nno  <silent>  cd  :<c-u>call Replace_qfl()<cr>

    let &stl = "%{exists('w:quickfix_title') ? w:quickfix_title : ''}"

If you write this code in /tmp/vimrc, then start Vim like this:
$ vim -Nu /tmp/vimrc

Then, execute a command to populate the quickfix list:
:vim /pattern/ $VIMRUNTIME/**/*.vim | cw

Finally press cd to invoke the custom function Replace_qfl(), the title of the quickfix window as displayed in the status line becomes:
:some title

While it should be:
some title

A colon is added in front of the title of the quickfix list.
If you close and re-open the quickfix window, it disappears.
The issue seems to come from w:quickfix_title, not from the title itself:
echo w:quickfix_title
    → ':some title'
       ^ ✘

echo getqflist({'title':0}).title
    → 'some title'
       ^ ✔

It can probably be solved by checking for an additional colon in w:quickfix_title when the quickfix window is opened for the first time, so I close the issue.

Hi,
On Tue, May 8, 2018 at 11:08 AM, lacygoill<EMAIL_ADDRESS>wrote:

vim --version: VIM - Vi IMproved 8.0 Included patches: 1-1789
Operating system: Ubuntu 16.04.4 LTS
Terminal emulator: rxvt-unicode v9.22
Terminal multiplexer: tmux master
$TERM: tmux-256color
Shell: zsh 5.3.1
Here's a MWE:
fu! Replace_qfl() abort
    call setqflist([], 'r',
                 \ {'items': [{ 'filename': $VIMRUNTIME.'/filetype.vim',
                 \              'lnum': 1,
                 \              'valid': 1 }],
                 \  'title': 'some title',
                 \ })
endfu

nno  <silent>  cd  :<c-u>call Replace_qfl()<cr>

let &stl = "%{exists('w:quickfix_title') ? w:quickfix_title : ''}"

If you write this code in /tmp/vimrc, then start Vim like this:
$ vim -Nu /tmp/vimrc
Then, execute a command to populate the quickfix list:
:vim /pattern/ $VIMRUNTIME/**/*.vim | cw
Finally press cd to invoke the custom function Replace_qfl(), the title of
the quickfix window becomes:
:some title
A colon is added in front of the title of the quickfix list.
If you close and re-open the quickfix window, it disappears.

This problem is addressed by  8.0.1831.

Yegappan

The issue seems to come from w:quickfix_title, not from the title itself:
echo w:quickfix_title
→ ':some title'
^ ✘
echo getqflist({'title':0})
→ 'some title'
^ ✔

Indeed I've just tested the patch, and it works great, thank you very much.


Display preset in issue message

I came across an issue which said " crosses Foot Path". I was confused at first until I realised the first part was referring to a railway line.
It seems the object name replaces the preset name in the issue message. I think it would be a good idea to display both, to tell users what type of object is causing an issue?
For example: Rail () crosses Foot Path (name of path) would be better IMO

Hmm maybe! We do kinda do this for the relation member list. The downside would be longer issues, which is especially a problem for ways with lots of crossings.

Contra Costa County Road 120 (Minor/Unclassified Road) crosses Concord Subdivision (Disused Railway)
Contra Costa County Road 120 (Minor/Unclassified Road) crosses Concord Subdivision (Disused Railway)
Contra Costa County Road 120 (Minor/Unclassified Road) crosses Martinez Subdivision (Disused Railway)

Whoops, seems like my examples got messed up - I had put the railway line name in angled brackets:
"Rail (name of railway line) crosses Foot Path (name of path)" or "Primary Road (ref) crosses Path (name)"
In your example it is currently Contra Costa County Road 120 crosses Concord Subdivision. I think a new/casual mapper seeing this might be confused about what this means without the road/rail.
The other reason I suggested this is because you could have the exact opposite problem, where your issue message is too short. For example two features which don't have a name but have a ref, e.g. "A1 crosses M2" and be confused about what exactly the problem is, or what it is referring to?

Lets close this.. Since clicking on the message takes you to the issue, and hovering the message will highlight the things, the user can clearly see what kind of features they are by looking, and doesn't need any extra guidance in the message.


GD-486: Fix image size errors at startup

Why
On macOS there are errors on startup.
Godot Engine v4.2.1.stable.official (c) 2007-present Juan Linietsky, Ariel Manzur & Godot Contributors.
  core/variant/variant_utility.cpp:1091 - invalid height:16 vs 32
  Invalid image: null
  core/variant/variant_utility.cpp:1091 - invalid height:16 vs 32
  Invalid image: null
  core/variant/variant_utility.cpp:1091 - invalid height:16 vs 32
  Invalid image: null
  core/variant/variant_utility.cpp:1091 - invalid height:16 vs 32
  Invalid image: null
  core/variant/variant_utility.cpp:1091 - invalid height:16 vs 32
  Invalid image: null
  core/variant/variant_utility.cpp:1091 - invalid height:16 vs 32
  Invalid image: null
Loading GdUnit4 Plugin success

What

disable the image size check when merge icons because it has different size on macOS 16vs32 pixels

Not sure, it's only linked to macos. it's also probably linked to the hiDPI monitor.
Pehraps you need to check with EditorInterface.get_editor_scale()
https://docs.godotengine.org/en/stable/classes/class_editorinterface.html#class-editorinterface-method-get-editor-scale

Not sure, it's only linked to macos. it's also probably linked to the hiDPI monitor. Pehraps you need to check with EditorInterface.get_editor_scale() https://docs.godotengine.org/en/stable/classes/class_editorinterface.html#class-editorinterface-method-get-editor-scale In my case, the factor is 2.0 and it's exactly the scale factor encountered in the errors.

Thanks for the hint, I will check this.

Thanks for the feedback, yes you are right.
I checked, and the image is scaled, I use scale 150% also on Windows.
  core/variant/variant_utility.cpp:1091 - invalid height:16 vs 24
  Invalid image: null
check icon size 16 24


Computed property names are const contexts

By analogy with the existing rule that element access argument expressions are a const context. This is useful for template literals, which otherwise have type string by default.
For example, from #45798:
type Person = { id: number }
const persons: Record<`person-${Person["id"]}`, { a: any }> = {
    [`person-${1}`]: { b: "something" }, // should error with excess property check on 'b'
}

The problem in #45798 is that person-${1} is intended to be checked against the index signature person-${Person["id"]}. But it gets the type string, which makes it a late-bound property. With a template type literal, it is instantiated as a string literal and is no longer late-bound.
There are a couple of ways that seem best to fix this:

Make computed property names contextually typed by their index signature. But this requires choosing the index signature that matches (if any), which requires getting the template literal type even if it's later not used.
Make computed property names const locations. This is a new rule, but is quite similar to the existing rule for element access argument expressions.

I don't consider (3), making all template literals produce template literal types, because I still believe the common case is that both string and template literals are intended to be widened to string.
Fixes #45798

@typescript-bot dt test this


"route_delimiter: -" breaks config - Also setting up with Translink

I live in QLD too and am trying to set this up with Translink.
Issue is that "home Assistant" may have had an update recently which may have broke the config. Any ideas on how to get this working again while keeping the "route_delimiter: -"?
I would have just restarted HASS anyway to see if it worked, but restart is prevented with an error in the config.

Hi, Translink is still working fine with me.  I am on HA 2024.10.2.  What version of HA are you using?  Can you post a copy of your configuration?

First I tried to effectively just copy your one with my own info (I have since tried your example directly and had the same issue). This config wouldn't work due to HASS not wanting to launch with route_delimiter: -
  - platform: gtfs_rt
    trip_update_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/TripUpdates'
    vehicle_position_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/VehiclePositions'

    route_delimiter: -
    departures:

    - name: To Indro 433
      route: 433
      stopid: 000007
      icon: mdi:bus
      service_type: Bus

    - name: To the City 433
      route: 433
      stopid: 001592
      icon: mdi:bus
      service_type: Bus

changing route_delimiter to say the following allowed me to reboot, however I didn't get any bus times coming up:
route_delimiter: '-'
I have also tried putting all the numbers in apostrophes in case that was the issue:
  - platform: gtfs_rt
    trip_update_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/TripUpdates'
    vehicle_position_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/VehiclePositions'

    route_delimiter: '-'
    departures:

    - name: 'To Indro 433'
      route: '433'
      stopid: '000007'
      icon: mdi:bus
      service_type: Bus

    - name: 'To the City 433'
      route: '433'
      stopid: '001592'
      icon: mdi:bus
      service_type: Bus

Any ideas?
I'm running HASS 2024.10.1

Try removing the leading zeros from the stop_id.  Unfortunately, the stop id that translink publish in their journey planner ('001592') is actually the stop_code, and not the stop_id, from their GTFS schedule.  The stop id is typically (but not always of course!!!) the same number without the leading zeros.  This is important because only the stop_id is published in the realtime feed.  Here is an extract for your route out of the realtime feed from this afternoon.

id: "TU-30355218-BT 24_25-37701"
trip_update {
  trip {
    trip_id: "30355218-BT 24_25-37701"
    start_time: "18:00:00"
    start_date: "20241015"
    schedule_relationship: SCHEDULED
    route_id: "433-3836"
  }
  stop_time_update {
  stop_sequence: 2
  arrival {
    delay: 112
    time:<PHONE_NUMBER>
    uncertainty: 30
  }
  departure {
    delay: 112
    time:<PHONE_NUMBER>
    uncertainty: 30
  }
  stop_id: "7"
  schedule_relationship: SCHEDULED
}

You can see two things:

The route_id is "433-3836" instead of just "433".  This is why the delimiter is important for translink.
The stop_id is "7" and not "000007"

You should get the stop_ids from the "stops.txt" file in the GTFS static zip file (as mentioned in the documentation) and not the journey planner or app.
Let me know if this helps please.

Hi Mark,
That helps for sure. Thank you. I updated my config with those details. It seems most the route IDs have two numbers for each bus route, eg. 433 has both 433-3838 and 433-3861. I have added both to my config just in case.
Again I tried the config both with and without apostrophes.
  - platform: gtfs_rt
    trip_update_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/TripUpdates'
    vehicle_position_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/VehiclePositions'

    route_delimiter: '-'
    departures:

    - name: 'To Indro 433'
      route: '433-3836'
      stopid: '7'
      icon: mdi:bus
      service_type: bus

    - name: 'To Indro 433 2'
      route: '433-3861'
      stopid: '7'
      icon: mdi:bus
      service_type: Bus

    - name: 'To City433'
      route: '433-3836'
      stopid: '1592'
      icon: mdi:bus
      service_type: Bus

    - name: 'To City 433 2'
      route: '433-3861'
      stopid: '1592'
      icon: mdi:bus
      service_type: Bus

I can still only reboot HASS with route_delimiter: '-' as route_delimiter: - returns an error and prevents rebooting.
Do you think the issue could be there? is there a way I can force route_delimiter: - to work?
I had the theory that maybe HASS running within docker may be preventing the integration from talking to the internet, however the logs are showing that the information is coming through.   At least that is what it is today.  With the next release of the schedule the calendar reference may change.  Because when it changes and what it changes too is completely out of my control I added the route_delimiter option to the configuration so that the code only needed the base route number (i.e. '433') to find the service in the realtime feed.
So, can you please add the route_delimiter option back in to your configuration change the route to '433'.  The following configuration works fine for me
  - platform: gtfs_rt
    trip_update_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/TripUpdates'
    vehicle_position_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/VehiclePositions'
    route_delimiter: '-'
    departures:
    - name: To Indro 433
      route: 433
      stopid: 7
      icon: mdi:bus
      service_type: Bus

Mann, I'm confused then.
I guessed earlier that I might have misunderstood your previous message and tried it without the calendar reference too.
I've also tried your configuration in your previous message directly.. Neither seemed to have helped.
Even trying with a train, I don't seem to get any ETAs.

I've also been checking it through today to see if maybe it just starts working... no luck though.
I've pasted my current config below, I'm now also trying it with a train.
  - platform: gtfs_rt
    trip_update_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/TripUpdates'
    vehicle_position_url: 'https://gtfsrt.api.translink.com.au/api/realtime/SEQ/VehiclePositions'
    route_delimiter: '-'
    departures:
    - name: 4 1 5 To Home
      route: 415
      stopid: 7
      icon: mdi:bus
      service_type: Bus
    - name: express train
      route: BRIP
      stopid: 600019
      icon: mdi:train
      service_type: Train

My current best guess is that maybe it has something to do with how it deals with timezones?
I notice in the logs below, the system data is written out in a date and time format, while the stop time is in Unix.   The route 415/stop 7 works fine for me.  The train route BRIP/stop 600019 doesn't have any services running at the moment so I can't test that at the moment but I have other Tranlink train services configured and they are working fine.
I agree that the issue is likely caused by timezone configuration on your system.  You mentioned docker previously which obviously introduces potential configuration inconsistencies.  I think you need to review your configuration.  All of my testing has been done an a HA OS installation.

Problem Solved!!!
It was due to docker
Looking close at the logs you can see that my log time is very different to the Translink "start time".
As it turns out, even though I have my timezone set to Brisbane within home assistant, my docker stack didn't specify this itself. Because of this, the time used from my logs were too far away from the scheduled arrival time to show in HASS.
To fix this issue I added the following TZ=Australia/Brisbane line to environmental variables in the docker stack. see below:

As soon as I updated my stack, the time values appeared for the translink buses.
(heck, I wonder if this has caused issues anywhere else).
Just one thing that may be of value though (as it was a red herring I was stuck on for a while),
Could you update the Translink example config to show route_delimiter: '-' rather than route_delimiter: -
? Without this, HASS won't restart and apply the config.
Thanks for your help Mark.
So glad someone figured out the translink system.

Glad you got it sorted out.  I have updated the documentation as you suggested.


Debounced keyup handler in search

As discussed in #621 this adds a 750ms debounced event handler in search & fuzzySearch. It makes searching far more usable with very large lists and/or slow devices.
Although perhaps the wait period should be overridden in an option? so the developer could. e.g., make the wait 500ms for desktops/laptops and 1000ms for phones.

Codecov Report

Merging #683 into master will increase coverage by 0.10%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #683      +/-   ##
==========================================
+ Coverage   94.74%   94.85%   +0.10%
==========================================
  Files          19       19
  Lines         799      816      +17
  Branches      187      193       +6
==========================================
+ Hits          757      774      +17
  Misses         29       29
  Partials       13       13

Impacted Files
Coverage Δ

src/fuzzy-search.js
100.00% <100.00%> (ø)

src/search.js
96.34% <100.00%> (+0.34%)
:arrow_up:

src/utils/events.js
86.36% <100.00%> (+11.36%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 038cdfd...334f28f. Read the comment docs.

Hi @sheffieldnick , I have forked this library and started a new project under the name of invisearch.js since this repo seemed a bit inactive. I would really love it if you could push this PR to invisearch.js, and help me keep this awesome library updated. /src files haven't changed in the new project. Thanks

Hi @sheffieldnick , I have forked this library and started a new project under the name of invisearch.js since this repo seemed a bit inactive.

Surely better to talk to @javve with an offer to help maintain this project instead of just copying it and giving it a new name?

Surely better to talk to @javve with an offer to help maintain this project instead of just copying it and giving it a new name?

No doubt. Guess I was a bit too afraid to see this awesome library to be inactive. Am really looking forward for this library. What do you think @sheffieldnick should I do with the repo I have created? Deleting it seems proper.

Codecov Report

Merging #683 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #683   +/-   ##
=======================================
  Coverage   93.88%   93.88%
=======================================
  Files          19       19
  Lines         818      818
  Branches      195      195
=======================================
  Hits          768      768
  Misses         34       34
  Partials       16       16

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 071b1a7...071b1a7. Read the comment docs.

Hey @sheffieldnick, thanks a lot for your PRs and I'm so sorry for not reviewing them until now 😞
I made a big update today. I updated all development dependencies and ran all code via Prettier, which feels great.
However, that also created a bunch of conflicts in your PRs. I'm not really sure why there is so many conflicts or why there is a bunch of random commits included. But I guess it would sort itself our if you could make it so that your PRs was up to date with the new master.
Could you do that? That would be fantastic!

Could you do that? That would be fantastic!

No, but you've got all my code that had no conflicts at the time, so you are very welcome to do it yourself.

That sounds reasonable @sheffieldnick 😅  Sorry for even asking you after all this time. I made it myself and put the fixes in this PR #696
Super nice work! I really appreciate it even though I haven't worked with List.js for a few years now, I feel really bad for you and other people who've made PRs during this time.
However, I hope to spend more time with this lib from now on!


Fix DependencyDB display for type template owners

Motivation
Previously, in the DependencyDB produced by autogen, we would display the owners of type templates using the <Class:...> syntax instead of just the name of the class, so references to a type template Foo in a class Opus::Bar might have a resolved name that looked like [:Opus, :'<Class:Bar>', :Foo], which is never what we want. We in fact worked around this in several places.
This uses shortName for this display, which will always display the original name without any extra stuff. That allows us to sidestep the problems.
This also identified and fixed a problem where we were exiting the test runner too early and not running pipeline tests.
Test plan

See included automated tests. Added a test case for this, although the exercised code is slightly different (i.e. we don't have expectation tests around the msgpack itself, but the toString of the autogen info is a good proxy.)

We have a policy of testing changes to Sorbet against Stripe's codebase before
merging them. I've kicked off a test run for the current PR.  When the build
finishes, I'll share with you whether or how it failed. Thanks!
Stripe employees can see the build results here:
→ https://go/builds/bui_JimB8nf3OKsnG5
→ https://go/builds/bui_JimBnQuos0xCd8


tensorflow meta model can not import to mobile device

Dear ildoonet,
Thanks for your excellent implementation. The model your provided is meta&index model , but the mobile tensorflow only could import .pb model.
Could provide the .pb model on the wget download?
Thanks very much !
Regards!

@Minotaur-CN
Provided model is not trained well, So I'm trying to train a good performance model.
Wait for few weeks...

Thanks!

Could you plz provide the graph.pb file of mobilenet based openpose?


feature #857 : added ability to delete comments

Created delete in QuestionAnswerRepository
Created DELETE disciplineTeachers/:questionId/:disciplineTeacherId/:userId/comment endpoint Created delete method to QuestionAnswerRepository
Created InvalidTypeException
Added permission
Added Swagger documentation
closes fictadvisor/fictadvisor#82

closed as outdated


[SPARK-24884][SQL] Support regexp function regexp_extract_all

What changes were proposed in this pull request?
regexp_extract_all is a very useful function expanded the capabilities of regexp_extract.
There are some description of this function.
SELECT regexp_extract('1a 2b 14m', '\d+', 0); -- 1
SELECT regexp_extract_all('1a 2b 14m', '\d+', 0); -- [1, 2, 14]
SELECT regexp_extract('1a 2b 14m', '(\d+)([a-z]+)', 2); -- 'a'
SELECT regexp_extract_all('1a 2b 14m', '(\d+)([a-z]+)', 2); -- ['a', 'b', 'm']

There are some mainstream database support the syntax.
Presto:
https://prestodb.io/docs/current/functions/regexp.html
Pig:
https://pig.apache.org/docs/latest/api/org/apache/pig/builtin/REGEX_EXTRACT_ALL.html
Note: This PR pick up the work of https://github.com/apache/spark/pull/21985
Why are the changes needed?
regexp_extract_all is a very useful function and make work easier.
Does this PR introduce any user-facing change?
No
How was this patch tested?
New UT

Test build #118091 has started for PR 27507 at commit 06bb690.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22856/
Test PASSed.

Test build #118091 has finished for PR 27507 at commit 06bb690.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
abstract class RegExpExtractBase extends TernaryExpression with ImplicitCastInputTypes
case class RegExpExtractAll(subject: Expression, regexp: Expression, idx: Expression)

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118091/
Test FAILed.

Test build #118096 has started for PR 27507 at commit 7a1c6d3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22861/
Test PASSed.

Test build #118096 has finished for PR 27507 at commit 7a1c6d3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118096/
Test PASSed.

Test build #118274 has started for PR 27507 at commit 1ed159f.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23033/
Test PASSed.

Test build #118274 has finished for PR 27507 at commit 1ed159f.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118274/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23055/
Test PASSed.

Test build #118297 has started for PR 27507 at commit ea29d66.

Test build #118304 has started for PR 27507 at commit e50f010.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23062/
Test PASSed.

Test build #118306 has started for PR 27507 at commit 517251a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23064/
Test PASSed.

Test build #118297 has finished for PR 27507 at commit ea29d66.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118297/
Test FAILed.

I have a high-level question. Do we have huge advantage to generate Java code?
One advantage is to store the result of Pattern.compile() into each global variable for caching while the non-generated code shares one variable for cache.
On the other hand, the size of the result is not small. Which trade-off do we select? Space or performance?

Test build #118304 has finished for PR 27507 at commit e50f010.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118304/
Test PASSed.

Test build #118306 has finished for PR 27507 at commit 517251a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118306/
Test PASSed.

I have a high-level question. Do we have huge advantage to generate Java code?
One advantage is to store the result of Pattern.compile() into each global variable for caching while the non-generated code shares one variable for cache.
On the other hand, the size of the result is not small. Which trade-off do we select? Space or performance?

There is a discussion in https://github.com/apache/spark/pull/26875

Test build #118335 has started for PR 27507 at commit 5e3c092.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23092/
Test PASSed.

Test build #118335 has finished for PR 27507 at commit 5e3c092.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118335/
Test FAILed.

retest this please

Test build #118341 has started for PR 27507 at commit 5e3c092.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23098/
Test PASSed.

Test build #118341 has finished for PR 27507 at commit 5e3c092.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118341/
Test PASSed.

Test build #118363 has started for PR 27507 at commit e9c5ef9.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23120/
Test PASSed.

I misunderstood one thing. In this PR, in my understanding, both cache w/ and w/o codegen can achieve the same advantage.
Do you have performance numbers w/ and w/o codegen? I think that both can achieve a similar performance while we execute the same SQL multiple times.  Am I wrong?

Test build #118363 has finished for PR 27507 at commit e9c5ef9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118363/
Test PASSed.

@kiszk I think one problem is, if we don't implement codegen, the child expression can't be codegened either. Also whole-stage-codegen will be disabled if there are expressions don't support codegen.

@cloud-fan Make sense to support codegen for whole-stage-codegen.
@beliefer Another question: Do we need to generate the same whole code multiple times? Since I cannot see the specialized part in the generated code, how about generating the runtime call. Here is an example.

@kiszk Thanks for your suggestion. Your suggestion is another way. I think the current code is OK.  If we plan to use the method you suggested, I think should make more discussion and not just modify this place.

It would be good to read this.
The code cache is limited. If the code cache is filled, no native code is generated furthremore.

It would be good to read this.
The code cache is limited. If the code cache is filled, no native code is generated furthremore.

Thanks. I learned it.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26718/
Test PASSed.

Test build #123716 has started for PR 27507 at commit ca6c1c5.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/28340/
Test PASSed.

Test build #123716 has finished for PR 27507 at commit ca6c1c5.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123716/
Test FAILed.

retest this please.

Merged build finished. Test PASSed.

Test build #126823 has started for PR 27507 at commit c096de4.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31476/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31482/
Test PASSed.

Test build #126823 has finished for PR 27507 at commit c096de4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126823/
Test PASSed.

Test build #126853 has started for PR 27507 at commit af88ad1.

Test build #126853 has finished for PR 27507 at commit af88ad1.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126853/
Test FAILed.

Test build #126858 has started for PR 27507 at commit 3a98bbe.

Test build #126858 has finished for PR 27507 at commit 3a98bbe.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126858/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31516/
Test PASSed.

Test build #126891 has started for PR 27507 at commit 82a384e.

Test build #126891 has finished for PR 27507 at commit 82a384e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126891/
Test FAILed.

retest this please

Test build #126916 has started for PR 27507 at commit 82a384e.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31541/
Test PASSed.

Test build #126919 has started for PR 27507 at commit 2125bff.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31543/
Test PASSed.

Test build #126919 has finished for PR 27507 at commit 2125bff.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126919/
Test FAILed.

Test build #126916 has finished for PR 27507 at commit 82a384e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126916/
Test PASSed.

retest this please

Test build #126924 has started for PR 27507 at commit 2125bff.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31548/
Test PASSed.

Test build #126924 has finished for PR 27507 at commit 2125bff.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126924/
Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/31567/
Test PASSed.

Test build #126943 has started for PR 27507 at commit 2125bff.

Test build #126943 has finished for PR 27507 at commit 2125bff.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126943/
Test PASSed.

thanks, merging to master!

@cloud-fan @kiszk Thanks for your review.


Atari-py Failed to Build Wheels

Currently when testing master branch, you cannot run the environment due to issues with atari-py and box2d

The solution to this problem was to
brew install swig
or
sudo apt-get install swig on your local machine, alternative if you are still dealing with this issue even though you have swig installed, you can run using docker as the docker environment installs swig.


remove deprecated features that were dropped in 4.x

motivation for this is that @ember/legacy-built-in-components is a peer dep for ember-engines for "ember-source": "^3.24.1 || 4",
in ember-source: 4 sendAction was rightly cleaned up and the named export is no longer resolvable which can cause warnings at best and errors at worst (trying to run an ember-engine while using a LinkTo component on ember source 4.x)
with this PR I propose we close that gap by releasing a new major for @ember/legacy-built-in-components to be used with ember-source >= 4.x

tbh not sure it worth the churn and effort. maybe I'm missing if there is an actual issue this PR fixes?

the actual issue is with running engines with embroider under vite

sounds reasonable!

the alternative would be to pr a change to engines to work without LinkTo component for example or any of the components provided by this addon

it means copy-pasting pasts of this addon.
we tried to brainstorm ideas how to move forward without using something old LinkTo component and came up with https://github.com/emberjs/rfcs/pull/819 which was put on hold as there is larger router initiative as part of Polaris.

I wonder if @mixonic is ok to land this and get new major out.
@void-mAlex I would say, we need to provide peerDeps on ember-source: >= 4 then + update readme as well as ember-try scenarios?

@void-mAlex either should be fine IMO

@mixonic @SergeAstapov I managed to get ci green as well
bumped peer declaration to be ember-source >=4
updated readme to reflect that
ember-beta is running against 5.x series so the loose peer declaration should be correct/ok to leave open for future ember version
this would unblock engines continuing to work through the 4.x and 5.x ember source version especially through embrodier
let me know if I missed something or anything else I need to do to land this.
thank you in advance


Remove autotools dependency

ASAN support
CMake support
Github CI

Disabled testing for Boost and PCRE regex matchers (don't want).

Whoops meant to open in my fork :)


Automate installation

Write a script which install everything to setup this project every docker container and package which is required should be automatically installed after running that script.


v5.0.0-alpha7 error showing searchview under toolbar

Hi. Thanks for this library!
Yesterday I was update library to 5.0.0-alpha7 and now searchview is showing under toolbar.
Main layout - CoordinatorLayout.
Searchview and margins type - menu item.
AppCompat version: 27.0.1.
I call searchview.open(true).
I try configure searchview in xml or code, with behaviour - no results. When I disable toolbar searchview is showing.
Screenshot https://www.dropbox.com/s/2a3yvqo634yyi1z/Screenshot_20171120-083843.png?dl=0

I solved this problem as follows:
public class MySearchViewBehavior extends CoordinatorLayout.Behavior<SearchView> {
    public MySearchViewBehavior() {
        super();
    }
    public MySearchViewBehavior(Context context, AttributeSet attrs) {
        super(context, attrs);
    }

    @Override
    public boolean layoutDependsOn(CoordinatorLayout parent, SearchView child, View dependency) {
        if (dependency instanceof AppBarLayout) {
            ViewCompat.setElevation(child, ViewCompat.getElevation(dependency));
            ViewCompat.setZ(child, ViewCompat.getZ(dependency) + 1);
            return true;
        }
        return super.layoutDependsOn(parent, child, dependency);
    }

    @Override
    public boolean onDependentViewChanged(CoordinatorLayout parent, SearchView child, View dependency) {
        if (dependency instanceof AppBarLayout) {
            child.setTranslationY(dependency.getY());
            return true;
        }
        return super.onDependentViewChanged(parent, child, dependency);
    }
}

I think it happened in the new version of AppCompat, AppBar has a priority Z-coordinate

Can you send me a pull request with these changes for SearchBehavior in library ?:)

I am working on Beta1 which will be released probably next week.

Fixed in private repository >> future Beta1


Fix interpolation for template provider version 2.0.0

Fix interpolation for template version 2.0.0 according the last changelog
Requirements

Fix the interpolation for modules that have the double dollar sign
Pin version of template provider for those modules

Assign to myself. This is blocking some other modules from being verified.

What to do, since our template normally is shell-compatible script:

all variables passed from terraform in form ${tf_var}
all variables assigned and used within the script in form $${sh_var}
all command substitution in form $(cmd)

Branch template_fix.11 contains my modification. Working on validation.

@ketzacoatl for opinion.

I'd like to keep the updates to the nexus example separate from the updates for the template provider, but it otherwise LGTM!

@ketzacoatl Reverted change in nexus example.


+))>Free Steam Gift Card Codes Generator 2024

Steam is one of the famous games app stores from where you can buy games and install them on your system to play them. The store is adding thousands of apps and games to make it the best app/game store for users. We all love playing the game, and most of the famous games are available on it.
CLICK HERE TO FREE GENERATOR
CLICK HERE TO FREE GENERATOR
There are a lot of games that are available for free; however, there are some apps and games which need to be purchased. But we all know that most users don’t want to spend their money on buying games. Either they go for different sources to install the game, or they start searching for the redeem codes through which they can get free money to buy and install the game.
We hope you are also looking for Free Steam Wallet Codes if you have opened this article. We will explain the Steam Wallet Codes who don’t know about it. Also, we will answer some queries of the users that they are searching randomly on the internet related to the Free Steam Wallet Codes. Be with us till the end if you are also excited to try the Free Steam Wallet Codes.
What are Free Steam Wallet Codes?
The users who don’t know about the Steam Wallet Codes, it is a way through which you can gift or recharge your wallet with money to buy any games or apps from the wallet. The players mostly buy the wallet codes, so they don’t have to buy the game from their cards each time when it is released. By having a sufficient balance in the Steam Wallet, you will be able to pay directly for it in a few steps. Thousands of users buy the wallet code to gift it to their family or relatives.
Similarly, the Free Steam Wallet Codes are the codes for which you don’t have to pay any money, and it is available for free to the players. They have to copy and paste the code to use it. There are a lot of sources through which you can get the Free Steam Wallet Codes. We are also going to list them below. Also, the Steam Wallet Codes’ value depends on your account’s region. We have listed the Free Steam Wallet Codes below.


Empty String

I have successfully integrated the project by following this https://github.com/garnele007/SwiftOCR/issues/25
I tried this image

But unfortunately getting an empty string. What could be the reason?

same here i had integrated project using above link, but when i am trying to use it on my iphone its returning empty string or improper output. Do i need to do anything else before running ?

Hmmm...
Could you please check if the app crashes if you change line 16 in SwiftOCR.swift to internal var globalNetwork = FFNN.fromFile(NSBundle(forClass: SwiftOCR.self).URLForResource("OCR-Network", withExtension: nil)!)!.
ps. Does it work if you use the SwiftOCR Debug app?

Well this was not working when using with Xcode 8.0 and Swift 3.0.
When running on Xcode 7.3 with Swift 2.3, it works absolutely fine!

I've opened a PR that ports the project to Swift 3 and also fixes the problem with the empty strings.

Still not working for swift 3.0. When you download the project from branch Swift3.0, it still needs to handle with some convertions to project work. However, after that still appears lot of deprecation warning and no good results with OCR.

Not work for the majority of the tested images. Only once returned two letters that were not related to the text in the image. Application for Mac OS, XCode 8.1, Swift 3.

Still empty string.... :((

any update on this? or any other swift library?

Still same problem tested on iPhone 7 and Xcode 10+ Swift 4.2


How to build binary wheels?

This is more of a question than a bug report. Being new to Nuitka, I'm wondering what the recommend method for building binary wheels, the equivalent of
python setup.py bdist

is. I saw bdist_nuitka somewhere. It works, but will probably not for much longer since Setuptools is phasing out setup.py being used as a the command line tool.

Yes, bdist_wheel is supposed to be a drop-in replacement for binary wheels. It has no documentation at this time, where it comes to how to add options for Nuitka, your best best is currently to check tests/distutils. I have added adding documentation and adding command line options for it to the target to the roadmap.
Regarding your comment, the distutils hooks are not tied to any specific user interface. As long as the values provided are there, they are going to work.
Closing, as we do with answered questions.

Thanks!

Closing, as we do with answered questions.

Perhaps enabling Discussions on GitHub is interesting.

We have the Gitter chat, which works for me. Managing a community takes time. Until I have volunteers for that, I cannot add it to my tasks.


Extend to controller base

Extend the base controller to be able to use middleware for each of the existing or custom actions. #181

Hi. Thanks for submitting this PR but unfortunately I'm going to have to reject it. You can't extend App\Http\Controllers\Controller because that class isn't guaranteed to exist - e.g. the consuming application might be using a different namespace. In addition, as this is a package it should only use classes that it can install via Composer dependencies, so it will have to extend the Illuminate\Routing\Controller and retain the traits that I've already put on the class.
I think it'll be easier if I make these changes, so I'll close this PR.


Component:- Leadspace with Image(Stage):- Copy text is not visible without gradient overlay on mobile and tablet.

Jeff would like this fix included in the v1.9.0 hot fix
Issue Description:- In 'Leadspace with Image' on deselecting 'Gradient Overlay' option copy text and secondary button text is not clearly visible. Happening on 'Default with Image' and 'Small with Image'.
Steps to reproduce the issue

Enter url in the browser.
Select 'Leadspace with Image' component.
In knobs section deselect gradient overlay.

OS:- iOS 12, 13 Android 8,9
Browsers:- Safari, Chrome, UC
URL:- https://ibmdotcom-react-staging.mybluemix.net/?path=/story/components-leadspace--default-with-image
Additional Information

Solution:  Dev should set the gradient on by default in Storybook

Gradient is already defaulted to true for storybook


WeakMap console warning when 0 used as conditional

When using a falsey value other than null, undefined, or false (like 0), there is a console warning:
failed setting the WeakMap cache for args
this should NOT happen, please file a bug on the github repo.

Example sandbox:

Confirmed. Will be fixed after v3 release

why are you sending 0 in the first place? I don't think we should special case for this unless there's a good reason. closing, feel free to reopen if I"m wrong.

@threepointone - I was conditionally applying a style based on a number. Something like numNotifications && styles.badgeVisible.
The only reason I filed this bug is because I read in the documentation that any falsey values will be ignored in calls to css. I think it’s fine if 0 isn’t supported since it’s easy enough to do a true Boolean comparison, but the documentation should reflect that. I’d be happy to open a PR on the docs if you want :smile:
Thanks for all your work on this project, it really is a joy to use!

Now that I’m reading the docs again, I can’t find the reference to doing conditional styling in a call to css. Not sure if it was removed or I’m just remembering completely wrong, so take the previous comment with a grain of salt 😝

no worries. in general I don't think glamor should handle 0, because it points to some underlying problem with the code (eg - it'll fail with 1, but pass on 0, but both probably mean you're sending the wrong thing to glamor)


Please write the whole whitelist of items

Hello, please send all the names of the items in order to be added to the whitelist

You mean like every item in the game? You are better off looking at the wiki for the item names or the spt-aki database


Resolve jsx whitespace quirk

I believe I've addressed every place that was affected by this. A new line does not generate a space after a tag, so we need to have text appear after tags in order to avoid concatenating the contents of the tag with the string/character that precedes or succeeds it.
Related to https://github.com/linode/manager/pull/2439

I didn't pay super close attention to line length so this might make the linter blow up as-is... will fix any lines if it does

Coverage remained the same at 71.281% when pulling 1f162dd6a9d0310a388803c69da082772fe5acb5 on rmcintosh:fix-spacing into 7c350da7fb9097a881f402197153503db8d2e242 on linode:master.


AttributeError: module 'tensorflow.contrib.tpu' has no attribute 'InputPipelineConfig'

AttributeError: module 'tensorflow.contrib.tpu' has no attribute 'InputPipelineConfig'
in tensorflow-gpu==1.7

@djh123 How did you fix it??

used tensorflow-gpu==1.15.2 instead and that seems to solve the issue.


Snyk 2020 06 04

:tada: This PR is included in version 1.2.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


CXXODM-11 Add a CRTP object "model" base class

One last change for the signature of collection(), but otherwise LGTM


Switching from Ubuntu to Chrome OS

name: xenial
encrypted: no
Entering /mnt/stateful_partition/crouton/chroots/xenial...
crouton: version 1-20190120145639~master:5bf96711
release: xenial
architecture: amd64
xmethod: xorg
targets: unity
host: version 11316.165.0 (Official Build) stable-channel snappy
kernel: Linux localhost 4.4.164-15546-gd8c7defc947f #1 SMP PREEMPT Sun Mar 3 22:54:30 PST 2019 x86_64 x86_64 x86_64 GNU/Linux
freon: yes
Unmounting /mnt/stateful_partition/crouton/chroots/xenial...

Just gotten into Chromebook and Linux so a newbie but having issues switching between Chrome OS and Ubuntu.
I installed Ubuntu on my HP Chromebook 14 G5 using the official tutorial on the Ubuntu website. Seemed to install fine, takes a few secs opening things and not as slick as Chrome OS but works fine for what I need it for.
In order for me to start Ubuntu, I have to open chronos and then stype shell and sudo startunity. It boots up fine.
However, there is no other way for me to get into Ubuntu without doing this. I presumed that I could just use control-alt-forward but this does not work as it takes me into the developer console which is not what I want. (interesting, in order for me to get back to Chrome OS from Ubuntu, it's control-alt-SHIFT-forward)
Is it possible to have Ubuntu loaded as the Chromebook starts? Or have I misunderstood
Thanks a lot in advance.

In theory you can have Ubuntu automatically start, but it would be more trouble than it's worth in my opinion.
And the ctl+alt+shift+=> is needed to keep the Chromos and Ubuntu different from eachother. Don't know why, it just is

@UnisexSigntaure

Is it possible to have Ubuntu loaded as the Chromebook starts? Or have I misunderstood

The crouton chroot does not launch with Chrome OS upon startup or sign-in, you have to do it manually by getting into a shell session and running sudo startubuntu as you stated above.
Also, as mentioned above, you can change it to automatically start but it's a bit more complicated and may not be worth it in the end.
Hope this helps,
-DennisLfromGA
Closing. Please re-open if this issue is still a concern.


Can I import this as a package?

Can I import newtonVR into another project as a package or asset?

Just added one to the root of the repo :)


Improve Metaschema specification and documentation for external developer community

Supervisor will find: a plan, documented by GitHub issues, to add or modify Metaschema documentation to be focused on external community developers identified by issues milestone/project/tag linkage; timely and accurate updates issues tracking the documentation changes; completed documentation on the Metaschema website, GitHub repository, or relevant location; analytics tracking reporting indicating no less than 10% improvement in readership of the documentation by Q4.

Completed support in Q2 of 2024 closing as I updated my performance plan objectives in June 2024 and they were finally approved in the upstream system of record.


Refactor test fixtures to more easily add new ones

I went to go put together an implementation for #3, but found the existing test fixture setup hard to extend, with different inputs sharing a single expected output.
This refactors the tests a bit to make it easier to drop in new expected input/output pairs for different scenarios, and I'll follow on with a PR based on this for adding the richer import syntax later today.

Thanks!