Streaming doesn't seem to work properly with explicitly created connections

It looks like the query's result event never gets emitted if connect() is used. Normal (buffered) queries work fine with connect(), and streaming works fine without connect().
I'm using<EMAIL_ADDRESS>and Node 4.2.1. Environment is an Ubuntu Trusty 64-bit VM (VirtualBox/Vagrant) on a Windows 7 64-bit host. Database is MySQL 5.6.
Note that streaming with explicitly created connections worked in previous versions (0.1.x).
This works fine:
// working.js

var Client = require('mariasql');

var c = new Client({
  host: '<IP_ADDRESS>',
  user: 'root',
  password: '',
  db: 'maria_test'
});

var query = c.query('SELECT * FROM users');

query.on('result', function (res) {
  console.log('stream begin');
  res.on('data', function (data) {
    console.log(data.id)
  });
}).on('end', function() {
  console.log('stream end');
});

c.on('end', function () {
  console.log('client end');
});

c.end();

Output:
vagrant@vagrant-ubuntu-trusty-64:~/maria-test$ node working.js
stream begin
1
2
3
stream end
client end
vagrant@vagrant-ubuntu-trusty-64:~/maria-test$

However, this fails (result is never emitted):
// failing.js

var Client = require('mariasql');

var c = new Client();

c.connect({
  host: '<IP_ADDRESS>',
  user: 'root',
  password: '',
  db: 'maria_test'
});

c.on('ready', function () {
  console.log('ready');

  var query = c.query('SELECT * FROM users');

  query.on('result', function (res) {
    console.log('stream begin');
    res.on('data', function (data) {
      console.log(data.id)
    });
  }).on('end', function() {
    console.log('stream end');
  });

  c.end();
});

c.on('end', function () {
  console.log('client end');
});

Output:
vagrant@vagrant-ubuntu-trusty-64:~/maria-test$ node failing.js
ready
client end
vagrant@vagrant-ubuntu-trusty-64:~/maria-test$

Is this a bug or am I doing something wrong?

I'm not able to duplicate the issue.

@mscdex Ok thanks for trying it. Anything I can do to help narrow it down? Is there a debug mode for this? What's your environment?

I personally tested it on Windows and Ubuntu 14.10, but all of the tests use an explicit connect() so that would also include the CI machines too (whatever OS they have).
The only debug statements are in the actual binding right now, you could turn those on I guess. To do that change DEBUG=0 to DEBUG=1 at the top of src/binding.cc and do a node-gyp build in the node_modules/node-mariasql directory.

Only seems to be reproducible on single core machines. I expect @blah238 only has a single core in the VM. #114 fixes it.

Just confirming that PR #115 indeed fixes this issue for me.


Feature Request: Portal standalone with vscode goto definition

Currently launching portal with (p/open {:launcher :vs-code}) does nothing if there is no portal plugin, but if you call "go to definition" in the standalone launcher it tries to do that in emacs.
It would be great to have the launcher option separate from the IDE selection option.
If you want to I can write a PR for this once I find some time.

I really like the idea, let me know if https://github.com/djblue/portal/commit/65baf7c9a34218b3ad3332903385148b532e258d works for you.

Thanks for quick response. I didn't have the time to try it yet, but the code looks good!


Update README.md: adds instructions to add a layout

Adding a layout is required to enable rendering of markdown files. Instructions for that had been missing so far.

Thank you!


Shifter: Control_01: Expose more space switch options

Add an option to configure the space switch to work with rotation, translation, scale or any combination

This issue was moved to mgear-dev/shifter_classic_components#7


MNT: rerender

Hi! This is the friendly automated conda-forge-webservice.
I've rerendered the recipe as instructed in #2.
Here's a checklist to do before merging.

[ ] Bump the build number if needed.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Still picking up conda-forge-pinning version 2<IP_ADDRESS>.27.37. May need to ask to re-render again in a little bit.

@conda-forge-admin, please re-render.

Seems to be working correctly now.


added kubeconfig output

This PR adds an option to output the kubeconfig to a file on the local file system
A user using the module will then need to add the file to his local KUBECONFIG environment variable (as explained in the terraform variable description) and will be able to connect to the cluster
This saves the need to further run az aks get-credentials ... post module execution

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: jungopro sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Metrics report- add table for mouthwash return rate after sort order update

The sort order for mouthwash mailing labels will be changed with the August release, to put those who completed their biospeicmen collection most recently at the top. We are interested in knowing if receiving the home mouthwash kit in closer proximity to the clincial collection visit this has an impact on the mouthwash kit return rate.
In conjunction with the August release, please add a new table, titled: Table 7.5 Baseline- Home Mouthwash Kit Return Rate for Packages Shipped up to 1 Week Ago (kits shipped after 8/28/2024) (The existing table 7.5 and all subsequent tables will need new numbers)
Table will mimic Table 7.4 (shown below), with the following footnote: Note: Sort order was updated on 8/28/2024

Will update after the prod push when there is data

@erincschwartz  Do these number make sense? I'm not sure if the additional amount received to shipped a week ago makes sense

@KELSEYDOWLING7 I reviewed with Michelle, this looks good to move to automation. I will review again once it is in the report next week and let you know if the issue can be closed.

@KELSEYDOWLING7, can we update the titles and/or footnotes for the following tables as indicated below:

Update footnote to: During this time the kit return method was USPS

Update title to: Table 7.4 Baseline- Home Mouthwash Kit Return Rate for Packages Shipped up to 1 Week Ago (kits shipped
between 6/17/2024 and 8/28/2024)
Update footnote to: During this time the kit return method was FedEx; Query for addresses to print pulled oldest clinical collection date/time stamp first.

Update footnote to: Query for addresses to print was updated on 8/28/2024 to pull most recent clinical collection date/time stamp first. Kit return method remained as FedEx.

All set! This will be updated in next week's report

@erincschwartz Good to close after Monday's report?

@KELSEYDOWLING7 In Monday's report, it does not look like any of the titles or footnotes were updated to reflect the requests above.
In reviewing, I realize that I have one additional change to the footnote for Table 7.5, can this be updated to say: Query for addresses to print was updated on 8/28/2024 to pull most recent clinical collection date/time stamp first, with a 5 day lag from date of clinical blood/urine collection date. Kit return method remained as FedEx.

Oh it looks like the changes haven't been merged yet. But yes, all set

Thanks. I will take a look next Monday.
Also, should the sums of the totals shipped in Tables 7.3-7.5 equal the total shipped in Table 7.2? Likewise for the number received?
It seems that Table 7.2 is not including the totals from Table 7.5.

Not quite. Table 7.3 and 7.4 total 7.2, but there's overlap between tables 7.4 & 7.5.
Should 7.4 be those after 6/17/24 but before and including 8/28/24? Right now it's just after 6/17/24

NOW updated with the new titles and footnotes, and Tables 7.3-7.5 total up to Table 7.2

Reviewed the report from Monday, this looks good! We can close this issue.


Port utilities

Also, I didn't review any of the sphinx code, but it looks fine on netlify, so it's good enough for me. I'll leave it to someone more familiar with sphinx to review that side of things.


Documentation has an error

After cloning the repo and installing via npm, the next step is to execute npm run ps but there is no ps script in the package.json.

Thanks for pointing it out! I have updated the instructions on the README.

Thanks.


peers.json is not created

Describe the bug
The creation of peers.json is not robust. The file is not generated.
To Reproduce
Steps to reproduce the behavior:

compile the code
run the simulation
Got error

[
{
"NetAddr": "<IP_ADDRESS>:12002",
"PubKeyHex": "0x0481CDABD15A88A5EAF36A1B78FC203F3284232B793C2B0E6B8B20AD04975682FDB39183510C604EAF86CBFB9A67F4F766DCC23DFC91ACE36954598927D5F636A2"
},
{
"NetAddr": "<IP_ADDRESS>:12003",
"PubKeyHex": "0x04F02664BF95DA2C79BC4E634D18066E8552D39F26D3CC0111F045BFD802D5BBF655DCBE447DF5ECE63A5C2F1B3753E463DEBFE4A818539903723A7014D0C6299D"
},
{
"NetAddr": "<IP_ADDRESS>:12004",
"PubKeyHex": "0x04F08C3C9E25FD7FD65588C952A4CD721CE947B35F8DD11F4579FC64C62CAD8B056874589855DD93662DF7E906F705425F1024A9E3AFA9281AD82246D0A4A33F5B"
}
]
cat: /home/qn/work/src/github.com/Fantom-foundation/go-lachesis/nodes/peers.json: No such file or directory

Hi Quan,
What branch of go-lachesis do you try to run?
Please provide the full output starting from the command line you use to run the simulation and all messages are produced afterwards.

Got message from Quan:

Hi Maxim
￼￼￼￼￼
12:20 PM
you can check out, make install
12:20 PM
maybe glide update
12:20 PM
then run
12:20 PM
I got an error: the peers.json file is not generated.
12:21 PM
you can double-check
12:21 PM
master branch
12:21 PM
Just double-check if you see the same error.
new messages
12:22 PM
if can't reproduce, you can close it.

The correct command to rebuild everything is:
make clean vendor proto build
After that successfully completes on the latest master you can run simulation using this command:
n=25 entry=main logshold=-1,-1,-1,85,0,0 BUILD_DIR="$PWD" ./scripts/multi.bash
Just have checked it with the latest master as of today and it works without any issue.
@quan8 Please try on your system the steps above and report if you still have an issue or the problem is resolved.

@quan8 Please also check that you have the latest batch-ethkey installed:
go get -v github.com/SamuelMarks/batch-ethkey

Steps:
Make a fresh clone:
$git clone xxx
$make clean vendor proto build
$n=3 entry=main logshold=-1,-1,-1,85,0,0 BUILD_DIR="$PWD" ./scripts/multi.bash
Output and Error:
<========== start of output =====
$ n=3 entry=main logshold=-1,-1,-1,85,0,0 BUILD_DIR="$PWD" bash -x ./scripts/multi.bash

set -euo pipefail
declare -i OPTIND=1
declare -i debug=0
declare args=
+++ dirname ./scripts/multi.bash
++ cd ./scripts
++ pwd
declare -r DIR=/home/qn/work/src/github.com/Fantom-foundation/go-lachesis/scripts
getopts d opt
shift 0
'[' '' = -- ']'
'[' 0 == 1 ']'
/home/qn/work/src/github.com/Fantom-foundation/go-lachesis/scripts/multi_build.bash ''

of logical CPUs:  4
of physical CPUS: 2
command-line-arguments
/tmp/go-link-318665185/000002.o: In function mygetgrouplist': /workdir/go/src/os/user/getgrouplist_unix.go:16: warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /tmp/go-link-318665185/000001.o: In function mygetgrgid_r':
/workdir/go/src/os/user/cgo_lookup_unix.go:38: warning: Using 'getgrgid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-318665185/000001.o: In function mygetgrnam_r': /workdir/go/src/os/user/cgo_lookup_unix.go:43: warning: Using 'getgrnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /tmp/go-link-318665185/000001.o: In function mygetpwnam_r':
/workdir/go/src/os/user/cgo_lookup_unix.go:33: warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-318665185/000001.o: In function `mygetpwuid_r':
/workdir/go/src/os/user/cgo_lookup_unix.go:28: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
[
{
"NetAddr": "<IP_ADDRESS>:12002",
"PubKeyHex": "0x044A869F291D672268D52F0F00A71D1DF6EBA757B2FB12D1C71E48A9E48E2951EB87374D3340FA14ED0CC5712E64AA358C80F43B2FBE9964EDF45E27B01697A513"
},
{
"NetAddr": "<IP_ADDRESS>:12003",
"PubKeyHex": "0x04EFC9C9288F7043684BEE8257F7879ABA5CFCE03C2DC3C966467AD64208B1D0FD7384D63AD398F8771CA2DCFF4613CF928422777ED9184CEE946B97E5BA67B1E7"
},
{
"NetAddr": "<IP_ADDRESS>:12004",
"PubKeyHex": "0x04EBE320F92C429E78B4964209F4B2B7C4B8B49C83AD97ECFCA311F064B288B35860CAAB91E0CBFF2EAF28A04754ECFB629E0C5F9BFEB9EA29D7366B89DA2FA7AD"
}
]
cat: /home/qn/work/src/github.com/Fantom-foundation/go-lachesis/nodes/peers.json: No such file or directory
======= end of output ====>

Obviously it should be
go get -u github.com/SamuelMarks/batch-ethkey
to get updated version of batch-ethkey

It is version issue. Fixed with update: go get -u -v github.com/SamuelMarks/batch-ethkey
It'd be good to make it part of the build process (whenever something is updated). You can close it if you think it's not.

batch-ethkey is an external tool mentioned in requirements. You had an old version of it.
Versioning is already proposed, see #53

Thanks.


Add finally function to then returns for Edge

When load EventSourcePolyfill on Edge, we have an error (Unhandled promise rejection TypeError: Object doesn't support property or method 'finally')
This was because the return of our Promise did not include a finally method.
Environment :
browsers : Edge18
event-source-polyfill : 1.0.8
core-js : 3.4.2
webpack : 4.35.3
It seems to be correlated with https://github.com/Yaffle/EventSource/issues/118

Glad to see this has been addressed! @Yaffle Is there a timeline for releasing this fix?

@vibdev , new version is published to npm


[🐛] Deluxe Fraud Challenge does not finish

:bug: Bug report
Description

When applying for a deluxe membership you can either choose to pay by card or wallet.
When your wallet is empty, the button for paying by wallet is disabled. When modifying the HTML entry and delete the disabled property, you can click "Pay", after that the submit button becomes enabled. After Submitting and login in again, you are a deluxe member.
Is this a regression?
Don't know
:microscope: Minimal Reproduction

Follow description above.
:fire: Exception or Error

:deciduous_tree: Your Environment

Using Heroku, so node v12.x

Additional Information

Cool, another unanticipated attack path for that challenge! 😁 ... @Scar26, can you have a look into this please? I suppose it shouldn't be as easy as this, so we probably have to close this simple solution? Or we reduce the difficulty of the challenge?

Nice :laughing: !
I'll look into this in a minute. I guess we should probably block this path otherwise, it would just become another inspect-element challenge and we already have a bunch of those. Is that cool?


Unknown Error Occurred

Have deleted the soon to be depreciated Weatherflow integration and replaced it with this one. However when I try to add the Station ID and the Token it comes back with "Unknown Error Occurred".
Any help greatly appreciated as I now no longer have Tempest data being passed to Home Assistant.
Logger: aiohttp.server
Source: /usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py:421
First occurred: 17:31:00 (1 occurrences)
Last logged: 17:31:00
Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
resp = await request_handler(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/web_app.py", line 543, in _handle
resp = await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/web_middlewares.py", line 114, in impl
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 85, in security_filter_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 80, in ban_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 233, in auth_middleware
return await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/headers.py", line 31, in headers_middleware
response = await handler(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 149, in handle
result = await handler(request, **request.match_info)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/decorators.py", line 63, in with_admin
return await func(self, request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 177, in post
return await super().post(request, flow_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 72, in wrapper
result = await method(view, request, data, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 110, in post
result = await self._flow_mgr.async_configure(flow_id, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 293, in async_configure
result = await self._async_handle_step(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 389, in _async_handle_step
result: FlowResult = await getattr(flow, method)(user_input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/weatherflow_forecast/config_flow.py", line 57, in async_step_user
station_data: WeatherFlowStationData = await weatherflow_api.async_get_station()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pyweatherflow_forecast/wffcst_lib.py", line 228, in async_get_station
station_data = _get_station(json_data)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pyweatherflow_forecast/wffcst_lib.py", line 379, in _get_station
item = api_result["stations"][0]
~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Are you sure you used the STATION ID?
Please see this section from the README file:
Each WeatherFlow Station you setup, will get a unique Station ID, this id is needed during configuration. To get your Station ID, login with your account, select the station on the list, and then click Status. Here you will find your Station ID.

Same issue here
Please wait, starting configuration wizard for WeatherFlow
Then
Unknown error
Station ID is correct as the Forecast integration works

Are you sure you used the STATION ID?
Please see this section from the README file:
Each WeatherFlow Station you setup, will get a unique Station ID, this id is needed during configuration. To get your Station ID, login with your account, select the station on the list, and then click Status. Here you will find your Station ID.

That was exactly what I did!!!!
I used the Device ID rather than the Station ID.  Thank you for the prompt, all now working, and working well.  👍


Album art is not removed when switching to songs that have no images avilable.

When the next or previous song is played the previous thumbnail image is not removed if the song has no album art.
Here is a bit of code I used to fix it. Sorry if its bad code I'm not much of a programmer, but here it is.
media_control.ccp
media_controls& media_controls::set_thumbnail(album_art_data::ptr data) {
	if (data != nullptr)
	{
		if (m_art_stream = SHCreateMemStream(reinterpret_cast<const BYTE*>(data->get_ptr()), data->get_size())) {
			Windows::Storage::Streams::IRandomAccessStream^ stream;

			if (CreateRandomAccessStreamOverStream(m_art_stream, BSOS_DEFAULT, ABI::Windows::Storage::Streams::IID_IRandomAccessStream, reinterpret_cast<void**>(&stream)) == S_OK)
				m_updater->Thumbnail = Windows::Storage::Streams::RandomAccessStreamReference::CreateFromStream(stream);
			else {
				m_updater->Thumbnail = nullptr;
				console::error("Failed to create random access stream over stream.");
			}

		}
		else
		{
			m_updater->Thumbnail = nullptr;
			console::error("Could not create stream.");
		}

	}
	else
	{
		m_updater->Thumbnail = nullptr;
	}

	return *this;
}

If you've fixed it why not make a pull request directly?

No idea how to do that sort of stuff, sorry hope I didn't waste your time.

No need to apologise, thanks for taking the time :)


[Feat]: Adding ARIA to provide additional information to assistive technologies

What feature?
🔖 Issue Description:
Enhance website accessibility by adding Aria attributes. Aria attributes play a crucial role in providing additional information to assistive technologies, ensuring a better user experience for individuals with disabilities.
💡 Key Points:

Implement Aria attributes to important elements throughout the website.
Use descriptive labels that accurately convey the purpose or function of each element.
Focus on interactive elements like buttons, links, and form fields.
Verify compatibility and effectiveness by testing with various screen readers and assistive technologies.

🌟 Contribution Opportunities:

Identify elements requiring Aria attributes and suggest appropriate implementations.
Provide feedback on Aria attribute usage and propose improvements.
Test website accessibility using different screen readers and assistive technologies, reporting any issues or recommendations.

🌐 Useful Resources:

MDN Aria Guide

Feel free to comment, ask questions, or contribute to this issue. Your active participation is highly appreciated!
Show us the magic with screenshots
No response
Agreement to Code of Conduct

[X] I agree to follow this project's Code of Conduct

@PiyushKalyanpy Sir, I would like to solve this Issue. Please do assign it to me under GSSoC'23.
Thank you in advance. 🚀🚀👍🏽😊
Regards,
@vishnu9865

Regarding this Issue, I'm mostly adding role and aria-description to important and isolated components with extra attributes like aria-placeholder, etc for Form controls.
Most accessibility guides emphasize on adding a11y attributes mostly for non-text elements.

Is this okay with you ?


AttributeError: 'list' object has no attribute 'get_shape' when building model

I am trying to use dcpg_train.py, but it fails at the model-building stage like this:
INFO (2017-11-17 11:41:39,196): Building DNA model ...
Traceback (most recent call last):
  File "anaconda/envs/tensorflow/bin/dcpg_train.py", line 828, in <module>
    app.run(sys.argv)
  File "anaconda/envs/tensorflow/bin/dcpg_train.py", line 192, in run
    return self.main(name, opts)
  File "anaconda/envs/tensorflow/bin/dcpg_train.py", line 699, in main
    model = self.build_model()
  File "anaconda/envs/tensorflow/bin/dcpg_train.py", line 591, in build_model
    outputs = mod.add_output_layers(stem.outputs, output_names)
  File "/home/MYUSERNAME/anaconda/envs/deepcpg/lib/python3.6/site-packages/deepcpg-1.0.4-py3.6.egg/deepcpg/models/utils.py", line 260, in add_output_layers
  File "/home/MYUSERNAME/anaconda/envs/deepcpg/lib/python3.6/site-packages/keras/engine/topology.py", line 554, in __call__
    output = self.call(inputs, **kwargs)
  File "/home/MYUSERNAME/anaconda/envs/deepcpg/lib/python3.6/site-packages/keras/layers/core.py", line 840, in call
    output = K.dot(inputs, self.kernel)
  File "/home/MYUSERNAME/anaconda/envs/deepcpg/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 822, in dot
    if ndim(x) is not None and (ndim(x) > 2 or ndim(y) > 2):
  File "/home/MYUSERNAME/anaconda/envs/deepcpg/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 437, in ndim
    dims = x.get_shape()._dims
AttributeError: 'list' object has no attribute 'get_shape'

I am using the latest commit to the master branch here, keras 2.0.2 (and have tried more recent versions) and tensorflow-gpu 1.4.0.
Thanks in advance for any suggestions!

I reran this today having changed nothing obvious and yet it seems now to work. Apologies!


SecKey reference is returning nil, when I try to retrieve the saved key from keychain.

I am using your library to save the public key generated on the server in the iOS keychain and then use that saved key reference to encrypt information and send that encrypted data back to the server.
Now, when I save the public key in keychain using the function - addRSAPublicKey(_ pubkeyBase64: String, tagName: String) throws -> SecKey? , the result constant returns 0 which I guess is confirmation that my key is saved in the keychain. The problem arises when I try to retrieve back the saved key from the keychain then I get the value of SecKey as nil.
Details about the public key,
Encryption Algorithm - RSA
Generated on - Server
Key Size - 4096
Please advice, how should I proceed?

Hi @silicuskushal,
There are 2 things you need to check: First is the tagName, it must match your application id. Second is you may need to run on simulator or real device (running on playground will not work).
The repo has a dummy XCode project to demonstrate it. You can have a look at its code to understand the first thing.
Best regards,
Thanh Nguyen

Thank you so much for the response. Actually, the public key was in the xml format, I had to convert that into a PEM format first before adding it in the keychain. But my concern is, I am able to encrypt the data using the PEM public key and send that data to the server but since the server side is having the private key in xml format, the encrypted data is not getting decrypted at their end. Can you help me out as to how to approach? Can we encrypt using an xml format public key in swift ?

Hi @silicuskushal,
The particular format of the key (XML, JSON, etc) is not really an issue. As long as the key content is correct you can either pre-convert the key to PEM format and use, or convert the key to PEM on-the-fly each time you need to use it.
In your case, it's the job of the server side to decrypt the encrypted message correctly. It has nothing to do with the format of the key.
Best regards,
Thanh Nguyen


generate-table-partitions generates invalid SQL in filter clause when a timestamp field is part of the primary-key

The following command:
data-validation generate-table-partitions -sc rs -tc bq -tbls source.table=target.table --primary-keys _hash_key,_hash_diff,_creation_timestamp,source_last_update_date,delivery_area_id --partition-num 164 --filter-status fail --bq-result-handler pso_data_validator.results -cdir ~/

generates 164 partitioned configs as expected but the none of those can be run using data-validation configs run -c ~/config_X.yaml throwing the follow:
07/31/2023 09:37:36 AM-ERROR: Error (psycopg2.errors.SyntaxError) syntax error at or near "10"
LINE 3: ...2f114e622b' AND (_creation_timestamp > 2023-05-16 10:48:06.8...

It seems the filter: <sql_where_clause> part of the YALM contains timestamp values without the required quotes (see _creation_timestamp below).
  filters:
  - source: (_hash_key > '04b0a057a5ceccd4fff06984b5ed8e0ba1e90591' OR _hash_key =
      '04b0a057a5ceccd4fff06984b5ed8e0ba1e90591' AND (_hash_diff > 'cba27aebaf720f51987e0d3087213bd50fc3a70c'
      OR _hash_diff = 'cba27aebaf720f51987e0d3087213bd50fc3a70c' AND (_creation_timestamp
      > 2023-04-18 10:39:12.808967 OR _creation_timestamp = 2023-04-18 10:39:12.808967 ...

Hi @florisvink, thanks for reporting this! Could you please help us understand how much this is affecting you at the moment? We did a triage for this problem and we will tackle it as soon as we can, since we just merged some updates for this feature on #922.

The problem is in two static methods _less_than_value and. _geq_value in  https://github.com/GoogleCloudPlatform/professional-services-data-validator/blob/develop/data_validation/partition_builder.py. They both have the line
value0 = "'" + values[0] + "'" if isinstance(values[0], str) else str(values[0])

This works correctly if the type is either string or numeric (real, integer etc). It does not work when the value is of type Boolean, datetime or interval(?). The fix is to change this to cover those use cases. While the code change is straightforward, we also need to come up with a test case(s) with primary keys with those column types and test across all the different databases.
Sundar Mudupalli

I also found out that this quoting problem appears when a STRING-field is part of the primary key and it contains a quote itself:
See 'Papa John's' example in error below:
Error (psycopg2.errors.SyntaxError) syntax error at or near "s"
LINE 3: ...'2837642736423' AND (chain_name < 'Papa John's' OR chai..

Hi @florisvink @helensilva14
I have raised PR #945 while solves the problem of timestamp and string with quotes. But I was facing issue with date datatype, for some reasons the pandas dataframe is treating it as timestamp datatype and adds a timestamp formatting to it, like this (col_date >= '2023-07-03 00:00:00') AND (col_date < '2023-07-07 00:00:00'). I am in the process to solve this.

That's very helpful @piyushsarraf . I was fixing those issues now by hand as well by adding an explicit type-cast on date-time columns.

Hi @florisvink! This new PR #962 is the most up-to-date one to fix the filter issue (#950) and our testings are almost finished. But it is addressing this TIMESTAMP issue, partially because of a dependency version we currently use which is
SQLAlchemy==1.4.49. This is the whole picture:
SQLAlchemy==1.4.49 (current version)

we use it because it's the latest that supports Snowflake (one of our recent data source additions)
primary keys: don't support TIMESTAMP (sqlalchemy.exc.CompileError: No literal value renderer is available for literal value "datetime.datetime(2023, 8, 27, 15, 0)" with datatype DATETIME) or DATE (same error, changing only the literal value to 2023, 8, 23, 0, 0)
filters: support DATE, but not TIMESTAMP

SQLAlchemy==2.0.20 (recommended and tested version that works!)

do not support Snowflake (what we understand is not a problem for you, because of your specific Redshift usage)
primary keys: support both TIMESTAMP, DATE
filters: support DATE, but not TIMESTAMP (ValueError: Unable to parse filter arguments)

Steps needed:

Go to your virtual env running DVT
Execute pip uninstall SQLAlchemy==1.4.49 && pip install SQLAlchemy==2.0.20
Ignore the below incompatibility error, since it doesn't affect Redshift on DVT as far as we know

Then you can execute your validation command as stated above for what 2.0.20 supports!

Let us know if you need direct assistance from us for any of this!

After pulling latest develop and upgrading SQLAlchemy as described above I got the following error when generating partitions:
[SQL: show standard_conforming_strings]
(Background on this error at: https://sqlalche.me/e/20/f405)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/floris.vink/Documents/professional-services-data-validator/main.py", line 6, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/floris.vink/Documents/professional-services-data-validator/data_validation/__main__.py", line 584, in main
    partition_and_store_config_files(args)
  File "/Users/floris.vink/Documents/professional-services-data-validator/data_validation/__main__.py", line 507, in partition_and_store_config_files
    config_managers = build_config_managers_from_args(args, consts.ROW_VALIDATION)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/floris.vink/Documents/professional-services-data-validator/data_validation/__main__.py", line 272, in build_config_managers_from_args
    pre_build_configs_list = cli_tools.get_pre_build_configs(args, validate_cmd)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/floris.vink/Documents/professional-services-data-validator/data_validation/cli_tools.py", line 1273, in get_pre_build_configs
    source_client = clients.get_data_client(mgr.get_connection_config(args.source_conn))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/floris.vink/Documents/professional-services-data-validator/data_validation/clients.py", line 253, in get_data_client
    raise exceptions.DataClientConnectionFailure(msg)
data_validation.exceptions.DataClientConnectionFailure: Connection Type "Redshift" could not connect: (psycopg2.errors.UndefinedObject) unrecognized configuration parameter "standard_conforming_strings"

[SQL: show standard_conforming_strings]

@florisvink
Unfortunately, PR #962 introduced the usage of Ibis's to_sql() method which compiles sqlalchemy queries. The to_sql() function only supports timestamps/date/datetime literal rendering starting in version 2.0: https://docs.sqlalchemy.org/en/20/changelog/changelog_20.html#change-206ec1f2af3a0c93785758c723ba356f
The 'standard_conforming_strings' error is because the redshift sqlalchemy connector doesn't support sqlalchemy versions 2.0: https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/264
Due to this package dependency conflict, there isn't a way forward to support Redshift using the to_sql() method. This would probably require reverting the change and adding a condition to format timestamps.

When I used the latest develop but left SQLAlchemy at 1.4.49 it seemed to work with the exception quoting issues in the filters YAML like so:
  filters:
  - source: orderdatetime > '2023-02-15'
    target: orderdatetime > '2023-02-15'
    type: custom
  - source: ' orderdatetime > ''2023-02-15'' AND orderid >= ''6dff79b1c7ffe28f29a44f9476a26b0e53edffa76d1fe06a8f7ce6252a7338df''
      AND orderid < ''6e288c21a16d73502110194c2fad6010868a1d58caf3c0e382b4b0a7262bef56'''
    target: ' ( orderdatetime > ''2023-02-15'' ) AND ( `orderid` >= ''6dff79b1c7ffe28f29a44f9476a26b0e53edffa76d1fe06a8f7ce6252a7338df''
      ) AND ( `orderid` < ''6e288c21a16d73502110194c2fad6010868a1d58caf3c0e382b4b0a7262bef56''
      )'

issue 1: single quote for both YAML string bounds and SQL string
issue 2: double single quotes in SQL

I guess I could try to string-replace the '' with ", or am I mistaking? Just trying to find a quick workaround

Sorry, ignore comment above. Although it looks wrong it does actually works with double single quotes.

@florisvink
As Neha mentioned, we are not able to support timestamp in a composite primary key because RedShift does not work with SQLAlchemy 2.0. I am not sure you need to specify timestamp as a part of the composite primary key.
In a traditional transactional databases a primary key (or composite primary key) is used to uniquely identify a row. This is how we use the term Primary Keys in the data validator. Our documentation says the keys uniquely identify the row. Generally using a timestamp in composite primary key in a transactional database is not a good idea as discussed here
In AWS Redshift, the primary keys are informational only, they are used for query optimization as mentioned in the documentation
Therefore, the only columns you need to mention in the primary key for data validator are the keys that uniquely identify the row. Hopefully you don't need to specify timestamp and therefore this is not an issue for you. At least that is my hope.
Sundar Mudupalli

@florisvink Unfortunately, PR #962 introduced the usage of Ibis's to_sql() method which compiles sqlalchemy queries. The to_sql() function only supports timestamps/date/datetime literal rendering starting in version 2.0: https://docs.sqlalchemy.org/en/20/changelog/changelog_20.html#change-206ec1f2af3a0c93785758c723ba356f
The 'standard_conforming_strings' error is because the redshift sqlalchemy connector doesn't support sqlalchemy versions 2.0: sqlalchemy-redshift/sqlalchemy-redshift#264
Due to this package dependency conflict, there isn't a way forward to support Redshift using the to_sql() method. This would probably require reverting the change and adding a condition to format timestamps.

Hi everyone! Updating this issue to reinforce that we encountered this problem above, but since @florisvink is not blocked anymore, I changed it to a feature request for us to get back here to test the feature when the https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/issues/264 gets solved and Redshift starts to support SQLAlchemy 2.0+.
Thanks a lot everyone for all the effort!

Being able to cast values while generating partitions would also solve a lot of problems while generating partitions. For instance we would have be able to cast a DATE to a string  which when sorted would have the same behaviour


feat: add cloud armor resources

closes #391
cleanup permissions, org policies and provide examples to deploy cloud armor policies
client-setup

add client name hierarchy viewer to policies

client-landing-zone

add org policy exception for lb types on the standard folder

client-project-setup

tier4-sa: remove editor role (too much overlap with security roles) and compute instance admin
tier4-sa: move network user permissions from host project to subnets

examples

add tier3/cloud-armor
add tier3/external-load-balancer
add tier3/remote-access-to-gce
add tier4/managed-instance-group

reviewing 1-3h eta....


AoT for ng2-boostrap

Quick fix from https://github.com/valor-software/ng2-bootstrap/issues/1093

I inline source maps because I let developpers to minimize by themself files (and removing comments). It avoids to have too many files. You can change it as your conveniance.

Without this key mouse events, universal is not working :)

can you leave here only .npmignore and make.js files?
and revert all other changes?

Done

Please check that AoT works fine,
I will play with this one :)
Thanks for quick reaction

My bad, i forget to rollback some files

ok, thing is
If I am planning to roll out this one
everybody will have to, replace
import 'ng2-bootstrap\ng2-bootstrap' to import 'ng2-bootstrap'
what is actually an artifact from ng2 alfas

ok, testing with system.js and ng-cli (webpack)

ng-cli: normal and AoT versions both imports working fine ;)

released as @beta,
check plunker and angular2-quickstart


Update scc-daemonset.yaml

Fixes https://github.com/nginxinc/nginx-ingress-helm-operator/issues/116
Proposed changes
Describe the use case and detail of the change. If this PR addresses an issue on GitHub, make sure to include a link to that issue here in this description (not in the title of the PR).
Checklist
Before creating a PR, run through this checklist and mark each as complete.

[ ] I have read the CONTRIBUTING doc
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked that all unit tests pass after adding my changes
[ ] I have updated necessary documentation
[ ] I have rebased my branch onto master
[ ] I will ensure my PR is targeting the master branch and pulling from my branch from my own fork

@andresm53 I feel like we could do both files at the same time in #117 ? What do you think?

Yes, sorry, will do that. Closing this one.


Igzip support

Depends on merging #32. After that PR base branch can be switched to master.
Tests will fail because there is a bug in igzip https://github.com/intel/isa-l/issues/143/. This causes the append tests to fail because the gzipreader with igzip will produce the wrong output.
This PR will need to be on hold until the bug is fixed. Unfortunately I do not have the know-how to fix it myself. But I guess it will not take long for the upstream developers to fix.
EDIT: I see now that isal is not in ubuntu 18.04, but because of this issue we will p I robably have to use the conda version anyway. I have submitted the package to conda-forge.

Hi Marcel,
I know you are on holiday, there is no rush. Enjoy the holiday!
This morning I was thinking a bit about the bug in igzip that prevents reading of concatenated files. This is annoying, because I am not sure if my fastq files are one big gzip file or a concatenation of gzip files. So using igzip for reading is not safe. Even if the bug is fixed, we do not know if the user has installed the fixed version of igzip on their system.
Checking versions is very elaborate and annoying. But this morning I concluded that by simply testing a concatenated gzip file it would be much easier. So I have added a simple concatenated gzip file to the resources. If igzip can read it correctly, it is fixed. This way we can discriminate between correct and incorrect versions of igzip quite easily.  Writing concatenated gzip files with igzip is not bugged. So the PipedIGzipWriter can be used without issue.
By fixing it this way, we can immediately start using igzip. Writing zip files will become a lot faster. I have packaged the isa-l libraries in conda-forge, so if isa-l is added as a dependency of xopen in the next release we should start seeing the benefits. If the isa-l library is updated to include the fix for concatenated gzip files, this code will automatically become usable without any intervention, as well as being backwards compatible with older versions of igzip.
I have added the "DO NOT MERGE" label as this depends on #32 which should be merged first.

Good things come to those who are patient: a fix has been made available to the igzip repo for the decompression bug. https://github.com/intel/isa-l/pull/147
This bug was found thanks to the extensive test suite of xopen, I am very grateful for that.
Also igzip will be able to diminish compute time for compression level 1 of cutadapt by more than 40%, while delivering smaller filesize as well. Isn't that great? I am very grateful to the igzip program for that.

@marcelm This is now ready for review.

@marcelm. Thanks for the feedback. I have incorporated the changes. It is much better now with the temporary file creation.
I also added an igzip test by creating an igzip environment. In order to do so I switched the distro over to "focal" (20.04) which includes the isal package that includes igzip.
Sorry for my late reply, I have been busy with python-isal but more about that in a separate post. Also, for reasons that I do not want to highlight on the public internet, I will be offline for a while from some point in the coming weeks. I expect to be online again at the end of November. If I do not react to any of your posts it is because of this, and not out of rudeness, neglect or disinterest.

@marcelm Thanks for the quick reply. It has been done.