[balsa] Delay resetting BalsaFrame.

Delay resetting BalsaFrame until the first byte of the next message is parsed. This is necessary for being able to access information about the parsed headers after the message is fully parsed, for example, response status code, and content-length and transfer-encoding headers.
Fixes #34096
Balsa implementation tracking issue: #21245
Commit Message: [balsa] Delay resetting BalsaFrame.
Additional Description:
Risk Level: low
Testing: test/extensions/transport_sockets/http_11_proxy:connect_test
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a
Runtime guard: envoy.reloadable_features.http1_use_balsa_parser (default true)

My only question is if we should runtime guard. cc @RyanTheOptimist for a thoughts before I merge

Yeah this seems wise to me. I think we can use a default true flag so it will take effect immediately, but it just gives us a backup in case there is an edge case here we have not discovered.

My only question is if we should runtime guard. cc @RyanTheOptimist for a thoughts before I merge

Yeah this seems wise to me. I think we can use a default true flag so it will take effect immediately, but it just gives us a backup in case there is an edge case here we have not discovered.

Done.

LGTM modulo ci unhappiness

@alyssawilk @bencebeky shouldn't this kind of fixes be backported to 1.30, 1.29 and 1.28 since Balsa is the default parser?

@sschepens Normally yes, it should. However, it turns out that the way ConnectionImpl is using the Parser interface, this issue never comes in practice. This issue was only discovered when UpstreamHttp11ConnectSocket started to use BalsaParser differently, through SelfContainedParser, and it is believed that this fix does not actually affect HTTP/1 parsing other than in that one extension (which still seems to be under development). Therefore I believe that a backport is not necessary in this case.


Implement control on top of keyboard to enable moving between textfields

Implemented UITextField-Navigation 3rd party library: https://git.io/vi9pB
Solution: https://github.com/yoha/wlkthru/commit/32a7cc0fabff4f7b0ca642f8138e27858268bb9b


Refactor OSv compiler

OSv compiler was misorganized, it was difficult to understand
the relation between osv-java.go and osv-virtualbox.go both sitting
in the same folder. Not any more - first refactored directory structure
of the pkg/compilers/osv to basically look like:
osv
 |- osvbootstrap
   |- virtualbox.go
   |- vmware.go
   |- aws.go
 |- java.go

Directly inside osv folder reside the actual compilers that make
use of Capstan (to compose package for selected language). Then there
is the osvbootstrap folder containing post-processing logic
(called "Bootstrapper") to carry out conversion of Capstan result into
appropriate image format (for the provider specified). The actual compiler
gets reference to the Bootstrapper and invokes its method
Bootstrap() to perform the conversion.
And that's it, code is more readable and managable.

Mr @ilackarms , Mr @gberginc, as suggested in other PR the OSv compiler refactoring is now sitting in this separate PR. I kindly ask for a review.

I'd suggest to review this PR by examining directory structure and files inside pkg/compilers/osv, not the diff. Diff is ugly since I've moved some files and reorganized some functions, but the result is really neat to read IMHO.

Hello, does this refactoring seem reasonable to you guys? 😄

Hey, sorry @miha-plesko. I didn't know who you were addressing with Mr :). I'll take a look.

I'd be happy to see Capstan integration converging towards merge, if possible. I've rebased the PR to the latest master and dropped changing directory structure to simplify the review.
@ilackarms, @gberginc, gentlemen, I would kindly ask you for a review.


Task - Collecting Event - vertical refactor

There area number of collecting event filter requests coming in.   Refactor the horizontal bar to vertical/unified pattern, and add some facets.
@jlpereira let's do this on ce_filter_refactor branch
See also

#1759
#1032 (dupe?)

Closing as done.


invalid configuration: "dependency.assets"

warn Package react-native-notifications contains invalid configuration: "dependency.assets" is not allowed. Please verify it's properly linked using "react-native config" command and contact the package maintainers about this.

I'll provide any needed info


Codecov fails to upload for pull_request_target

Describe the bug
Codecov action fails to upload for pull_request_target triggers.
https://github.com/codecov/codecov-action/issues/155

Maybe we should skip upload for PRs?


Doc file splitting into multiple image files

When uploading .doc or .docx files, the following warnings are displayed:
No acceptable contours found
Contour is not a quadrilateral
lib/python3.10/site-packages/langchain_core/_api/deprecation.py:139: LangChainDeprecationWarning: Since Chroma 0.4.x the manual persistence method is no longer supported as docs are automatically persisted.   warn_deprecated(
After uploading, a single .doc file splits into around 59 documents in .png or .jpeg formats. These are shown in the Doc Counts sidebar and also form the metadata as illustrated in the attached screenshot.

The main issue is the inability of the .doc file to be parsed as a single document, unlike .pdf files. Instead, it splits into multiple .png or .jpeg files, leading to hallucination during querying.
Please address the parsing issue and provide a solution to handle .doc or .docx files correctly without splitting into multiple image files.

We extract images so that they can be processed separately for image question-answer, but I understand if there are many images it might get messy.
I added an ENV so you can control with H2OGPT_DOCX_EXTRACT_IMAGES.  set it to "0" to avoid this step.


[SUPPORT]How to measure the performance of upsert

Describe the problem you faced
I am a novice  and would appreciate your help.
We use Structured Streaming to consume the data in Kafka, and then write the data to the cow table of hudi.I want to know whether the performance of this program is high or low

It dependent on count of cores, configuration, ratio of insert/update, storage type
IMO, it is not slow


Running the evodiff/generate.py script

I've been having trouble with getting the conda environment to work properly, so this may be exacerbating the issue below.
(evodiff3) ubuntu@209-20-159-77:~/evodiff_repo$ python evodiff/generate.py --model-type oa_dm_38M --num-seqs 100
Traceback (most recent call last):
  File "evodiff/generate.py", line 323, in <module>
    main()
  File "evodiff/generate.py", line 40, in main
    data = UniRefDataset('data/uniref50/', 'train', structure=False, max_len=2048)
  File "/home/ubuntu/miniconda3/envs/evodiff3/lib/python3.8/site-packages/sequence_models/datasets.py", line 330, in __init__
    with open(data_dir + 'splits.json', 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'data/uniref50/splits.json'
(evodiff3) ubuntu@209-20-159-77:~/evodiff_repo$

If you're struggling to install EvoDiff locally, feel free to try https://www.tamarind.bio/evodiff, a website which offers a no-code interface for bioinformatics tools including protein design with EvoDiff for free.


Rename amm pallet to xyk pallet

https://medium.com/finnexus/understanding-the-xyk-model-of-pooled-liquidity-7340fdc20d9c

This was done in hydradx-node repository and the pallet will used from there when ready.


Get started with Authentise/pack3d development.

Steps to set up cloned repo on your machine and commit/test as you go.

Meant to open PR on forked repository.


props.columns not updated

Hi,
it seems the props.columns is not updated after the initial Mount.
In my react application I would like to have links to switch between hardcoded sets of columns however it seems that if you modify the columns props the changes aren't repercuted since Griddle only use its internal filteredColumns state from now on.
I guess the proper way would be for Griddle to detect when the props change and modify state.filtredColumns? Or is there any other way to do it?

I have found a way to do this using refs and setColumns but I think this is still an issue, Griddle should either update when changing props.columns or it should be documented that it doesn't happen (maybe rename the property defaultColumns then?).

+1. This is very unexpected. I had to force component remount using key to solve this

This should be fixed in the source but not up in npm just yet.

@ryanlanciaux The fix for this issue seems to prevent changes to the filteredColumns array persisting once componentWillRecieveProps is triggered.  If changes were made via the gridSettings component, they are reverted to the default's that were set via the columns prop in situations like updated external results or onRowClick.


Docs build as a test in the CI

It would be nice if the docs build was present as a test on each PR, so you can see what the docs will look like after the PR / if the build succeeded.
Where would this need to go @norlandrhagen ?

I'm also a bit confused on how its supposed to take for a change to main to be reflected in the public docs build, or whether I need to release or something first?

I think we can close this after our RTD work. New PR's have docs builds now included. see: https://github.com/TomNicholas/VirtualiZarr/pull/53


Added a screenshot

I couldn't find a screenshot of the dashboard running, so I went through the whole process (signup, install, run, etc) just to produce one.
Feel free to add this screenshot.

@jdrago999 thanks!


Adds hidden files toggle button using pop up menu and switch

Description

Added a "show hidden files" option inside a pop up menu to the side of the Add new folder button.

This is usually seen in files applications

No dependency required.

Fixes #9

Screenshots (if appropriate):

Type of change
UI/Backend change
Please tick the relevant option by putting an X inside the bracket

[X] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[X] This change requires a documentation update

Checklist:

[X] My code follows the guidelines of this project
[X] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[X] My changes generate no new warnings
[X] All current GitHub actions pass
[ ] Any dependent changes have been merged and published in downstream modules
[X] I have checked my code and corrected any misspellings
[X] I have read the CONTRIBUTING document

ui will eventually be changed but as previous pr, this serves as a good base. thanks again for your contributions!


[UI] -- Cell Visualizations - Graph w Stat - Base color dropdown does not expand on first click

N.B. Writing a Cypress Test to cover this component.  Need to work around with a second click.  However, if this will be fixed, second click will then close the dropdown as already happens with other dropdown controls... and test will fail.
Steps to reproduce:
List the minimal actions needed to reproduce the behavior.

Create a dashboard
Create a cell with data and save
Open the configure cell menu item
Click the settings cog wheel
Set Graph type to Graph + Single Stat
Scroll to Colorized Thresholds
Click the "Base" Color dropdown.

The dropdown fails to expand on the first click.
Expected behavior:
Since other dropdowns on the page expand with one click, expected this one will expand with 1 click as well.
Actual behavior:
Dropdown failed to expand.  Had to Click a second time.
Environment info:

System info: Run uname -srm and copy the output here Linux 4.4.0-148-generic x86_64

InfluxDB version: Run influxd version and copy the output here
This is a local build.
Boot command
./bin/linux/influxd --store=memory --e2e-testing=true --log-level debug 2>&1 > ~/var/log/influxd.log &
Screenshots

@karel-rehor is this still an issue?


repair can not downlink data,localization :shipit:

The use of brocaar/loraserver can send downlink data packets

Very good!


Allow AsJSON serializer to receive the JSONEncoder class implementation

What this PR does / why do we need it:
Currently we can use the AsJSON serializer which will serialize using json.dumps(obj), but in the obj can be values which are not serializable automatically by JSON default serializer. json.dump() has an extra parameter, cls which can be a custom class which implements json.JSONEncoder
Use case:
A task returns a pandas dataframe which can have some values of type numpy.int64. The AsJSON serializer will fail with  the error: We encountered the following error while attempting to serialize the results of this task: Object of type int64 is not JSON serializable
Which issue(s) does this PR fix:

Fixes #
Release Notes

What type of changes to the API does this change introduce?
PATCH

Checklist

[ ] I've read the Contribution Guidelines
[ ] Updated the appropriate sections of the documentation.
[X] I've added automated tests covering all success and error scenarios.

Hello @raztud,
Thank you for submitting a PR with your proposal.
I have a couple of suggestions:
The first one is that I would not use JSON to serialize Pandas DataFrames. I don't know if you've seen this, but you can write your own serializer implementations very easily: https://larribas.me/dagger/user-guide/serializers/write-your-own/
In particular, you could write a serializer that uses Parquet files or CSVs to serialize dataframes.
... Or you can use the ones in the dagger-contrib library out of the box! 🙂  https://github.com/larribas/dagger-contrib

The second thing is that if we are to accept arbitrary JSONEncoders in the official AsJSON serializer, we would need to also accept JSONDecoder instances.
It is instrumental to the behavior of a serializer that the process is symmetrical.


Update deltask in UG

Currently, the user guide does not have deltask. Let's add deltask in the user guide to allow users to understand its usages.

Codecov Report
Base: 68.03% // Head: 68.03% // No change to project coverage :thumbsup:

Coverage data is based on head (4f5b772) compared to base (07f9d3b).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##             master     #157   +/-   ##
=========================================
  Coverage     68.03%   68.03%
  Complexity      610      610
=========================================
  Files           106      106
  Lines          1980     1980
  Branches        217      217
=========================================
  Hits           1347     1347
  Misses          575      575
  Partials         58       58

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Fix links to pubsub documentation.

It appears that the pubsub doc links have changed on us.
Before: https://cloud.google.com/pubsub/reference/rest
After: https://cloud.google.com/pubsub/docs/reference/rest
See: https://github.com/GoogleCloudPlatform/gcloud-common/issues/190
https://github.com/GoogleCloudPlatform/google-cloud-node/issues/1609

@dhermes, confirmed 1 and 2.

I also, looked at the other services links and it appears that there is a mix of /docs/ and no /docs/ in the paths. I assume that we could expect other ones to change in the future as well.


chore: update logo url ( PROOF-680 )

Rationale for this change
The Blitzar logo was updated right after the previous logo update. This PR updated the logo's URL.
What changes are included in this PR?

The URL is updated to the new logo for light and dark backgrounds.

Are these changes tested?
Yes, on Github and VSCode.

:tada: This PR is included in version 1.1.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


cranelift-fuzzgen: use a different namespace

Otherwise I get a panic with "Duplicate function with name u0:1 found!"
at fuzz/fuzz_targets/cranelift-fuzzgen.rs:76:10.

It's actually because the trampolines and renamed functions are in namespace 0. References to other (non-existent) functions should be fine since they get renamed. I think?

Thanks! And sorry for the mess this created.
The hard coded function names should go away when we start generating random function names.

No worries! I think the only "mess" was my flailing around in different issues trying to figure out what happened. :laughing:


Encountered Errors

Because we encountered some errors for certain setups we decided to write them down in the wiki to collect and share for others, we encounter the same problem.
See https://github.com/hpi-swt2/compass-portal-blue/wiki/Known-Issues

We should probably also specify the supported node versions in the Readme and the package.json (via the engine property).

Node version 10 isn't a good choice because it'S too outdated as @niklasmohrin told me.

Node 12 is the last still maintained version (until 2022-04-30)

best case we would be able to run node 16 which is maintained for 2 years to come (apart from that, for me, there are problems with node 17 anyways :D)
I am having trouble running the version from #137 with node 16 though, the map is all messed up although it works on dev (with node 16) and the pr branch works with node 10. :man_shrugging:


Add github users to Grafana dashboard for CarbonPlan

Context
Re: https://2i2c.freshdesk.com/a/tickets/795

I'd like to view the grafana dashboard for the CarbonPlan hub. The docs recommend reaching out to a 2i2c engineer to activate GitHub login - https://infrastructure.2i2c.org/topic/monitoring-alerting/grafana/#logging-in. Could Raphael Hagan (norlandrhagen) and I (maxrjones) please be given access via GitHub login?

I see that CarbonPlan already has grafana via github organizations set up. Is it also possible to add just individual users using the support.values.yaml or is a different kind of Grafana configuration required?
Proposal
No response
Updates and actions
No response

@jmunroe, I think this one might help you: https://infrastructure.2i2c.org/sre-guide/support/grafana-account/#

Very clear.  Issue resolved!


APEXMALHAR-2374 Calling scan recursively if the current path is a dir…

…ectory
@bhupeshchawda  please review this.

Looks fine to me.


question: can oasgraph be used to generate a graphql api from a loopback 3 swagger?

Describe the bug
Tried this and ended up with multiple:
    {
      "type": "InvalidSchemaType",
      "message": "Request / response schema has no (valid) type: {\"type\":\"object\"}",
      "mitigation": "Fall back to type 'GraphQL String'"
    }

To Reproduce

clone https://github.com/strongloop/loopback-example-access-control
npm install && npm start. Server starts on :3000
run oasgraph http://localhost:3000/explorer/swagger.json`

Expected behavior
run with no error

Why is this issue closed? Is it fixed?

@jtan4nih This "error" is not a true error because we have provided a mitigation for them, "Fall back to type 'GraphQL String'".
To summarize what I wrote before, sometimes there can be a mismatch between GraphQL and the OAS. In GraphQL, you must precisely define what objects look like, i.e. describe every property. In the OAS, you can simply define objects as objects: {"type":"object"}. When this happens, we choose to stringify the object and store it as a GraphQL string type rather than create a new object type, as it would be impossible to do so without having prior knowledge of the properties.
The only way to truly fix this "error" is to define all the properties of the object in the OAS.
The issue is closed because there was never an issue to begin with. The real problem is lacking documentation, which I will try to improve in the next update.

How did it go?

@GitMastStas We have improved schema type identification since then but if a schema type cannot be identified, then we produce the following warning:
{
    "typeKey": "UNKNOWN_TARGET_TYPE",
    "message": "No GraphQL target type could be identified for schema '{stringified schema}'".
    "mitigation": "The response will be stored in an arbitrary JSON type."
}

...which we hope gives the user a better idea of what's going on.


Finish writing the Percival "starter notebook" tutorial

I will finish the tutorial and fill in the missing section with more complex data analyses, as well as nicer examples of data visualizations using Observable Plot.
This starter notebook will be finished when I have time, and I'll advertise the project after that.
See also: #5.
(It would also be good to write a "How it works" section to the README explaining the Rust Datalog compilation and reactive WebAssembly components + Web Workers.)

So excited :D


Do you/can you support Outlets?

Is your feature request related to a problem? Please describe:
I have Z-wave outlets controlled by SmartThings.  The icon shows as a 3-prong outlet, with an on/off switch.
Do you/can you support these, so that they appear as Outlets in HomeKit?

my Zwave outlets are supported. but I don't have 3 gang devices to test

@garyhooper I assume your outlets are looking like switches?  If so, I assume they operate as you would expect?  Other than the icon issue?

Closing due to lack of activity.


[BUG] Invalidation Does Not Work with Filtered GetCollection Requests Using fields[] Apiplatform

Repository https://github.com/g-ra/ISSUE-944-apiplatform/tree/issue-2
Video Demonstration: https://www.loom.com/share/b512959e7a944f6b8bf3bacdf20ac32d
Description:
While testing the Souin caching mechanism, we've encountered an issue where cache invalidation does not work correctly when using filtered GetCollection requests with fields[] parameters. Specifically:
GetCollection requests work correctly.
GetID requests also work correctly.
However, GetCollection requests with fields[] filters trigger invalidation but not deleted in souin.
mode bypass
Steps to Reproduce:
Perform a GetCollection request:
Example: GET http://localhost:81/customers?page=1
Result: Works correctly and cache invalidation occurs as expected.
Perform a GetID request:
Example: GET http://localhost:81/customers/1
Result: Works correctly and cache invalidation occurs as expected.
Perform a GetCollection request with filters:
Example: GET http://localhost:81/customers?page=1&fields[]=name
Result: Cache invalidation does not occur as expected.
Compare with a GetCollection request using query parameters:
Example: GET http://localhost:81/customers?page=1&firstname=John
Result: Works correctly and cache invalidation occurs as expected.
Expected Behavior:
Cache invalidation should occur for all GetCollection requests, including those with fields[] parameters, just as it does for other filtered and unfiltered requests.

Resolved by #545


Passing styles as props

We are currently using Radium in all of our components. We have a use case where Component A is passing styles as props to Component B. Then Component B is using Radium to combine some default styles with the passed in styles:
// Component A
const React = require('react');
const Radium = require('radium');

const styles = {
  red: {
    color: 'red'
  },
  green: {
    color: 'green'
  }
};

const ComponentA = React.createClass({
  render () {
    return (
      <ComponentB style={[styles, this.props.showGreen && styles.green]} />
    );
  }
});

module.exports = Radium(ComponentA);

// Component B
const React = require('react');
const Radium = require('radium');

const styles = {
  color: 'yellow'
};

const ComponentB = React.createClass({
  render () {
    return (

I should be green

    );
  }
});

module.exports = Radium(ComponentB);

Unfortunately this isn't working because this.props.style is an array. If we pass in a single style object as the ComponentB style prop, then it works perfect.
Is there currently a way to address this use case in Radium?

style={[styles].concat(this.props.style)]}?

@ronag thanks for the suggestion! We are actually doing that right now as a temporary fix. It would be nice if Radium could handle nested arrays.

@bsbeeks: I see, then I guess this issue is more about whether Radium should flatten (or not) the array passed into style.

I guess I spoke too soon. It looks like the mentioned use case does work. In our specific components we were placing ...{this.props} after style={[styles, this.props.style]} and so this.props.style was overwriting the Radium generated styles.

Yeah, we recently shipped support for nested style arrays: https://github.com/FormidableLabs/radium/blob/master/CHANGELOG.md#0140-september-15-2015


Add Untag() to oci.Store

Hi.
There is currently a way to tag OCI descriptors, but not to untag them.
This can cause problems when you want to remove container image.
Indeed, in the following scenario, we have two tags pointing to the same underlying descriptor, as shown by the digest:
$ ig image list
REPOSITORY            TAG    DIGEST
docker.io/library/bar latest f959f580ba01
docker.io/library/foo latest f959f580ba01

In this case, we should not try to remove the image, as removing the descriptor will remove it for both image, but rather to untag one image:
$ ig remove docker.io/library/foo:latest
$ ig image list
REPOSITORY            TAG    DIGEST
docker.io/library/bar latest f959f580ba01

To do so, I opened #647 but we should first discuss this here before going further with the PR.
If you want more details on my use case, I invite you to check https://github.com/inspektor-gadget/inspektor-gadget/pull/2162.
Best regards.

I think of a design question regarding this feature: If the given reference string is a digest, what should the behavior of Untag() be? What behavior satisfies your scenario? @eiffel-fl

Hi @eiffel-fl,
We have discussed internally in our team, in general we like the idea of Untag and we think you may continue the work in PR #647. You can add unit tests now, we will review it and get it merged.
Here are a few things we would like your Untag implementation to have:

Your implementation is only for oci.Store for now. Probably we need to have Untag for other types later. So how about you add an Untagger interface in content.resolver.go ? It may just look like the Tagger interface. Its signature would be Untag(ctx context.Context, reference string) error.
For the behavior of the Store.Untag() function, if the input is a valid reference, it should perform untag operations. But if the reference is a valid digest of an existing content, it should do nothing. I think your last implementation is good, you just need to add an extra case of handling digest input. You can use s.tagResolver.Resolve(ctx, reference) and check if reference is a valid digest.

Let us know if there are further concerns. If you want, our team can co-author on PR #647 to speed up progress.


Fix tflint rules (e.g. rename resources)

Description
tflint is used across all terraform-aws-modules and it is currently failing with warnings/notices like:
Notice: resource name `cluster_AmazonEKSClusterPolicy` must match the following format: snake_case (terraform_naming_convention)

Rename of affected resources is a breaking change worth documenting.

This issue has been resolved in version 18.0.0 :tada:


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target_c94c37e2-0f24-4874-b7f7-e0847d321b10/pull/1 is merged into live branch.
Auto archive content to https://github.com/OPS-E2E-PPE/docs-archive-test-target_c94c37e2-0f24-4874-b7f7-e0847d321b10.git

Docs Build status updates of commit 449ba1e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

archiveSourceFolder/TOC.yml
:white_check_mark:Succeeded

archiveSourceFolder/content/someContent.md
:white_check_mark:Succeeded

archiveSourceFolder/content/sql-database-engine.png
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Consider new PointerArith Node type

I think reordering the Left and right children on a '+' node when the left side is a pointer is bad. This adds implicit assumptions about the structure of nodes.
Another way to solve this is to introduce a pointer arith node type. Have you considered that?  Just close this issue if you disagree.

I think enforcing layout is not actually bad. The point is that the + operator in C is not associative -- you cannot swap rhs and lhs in general. It's doable only when both types are arithmetic (non-pointer) types.  Naturally, pointer minus something needs to be represented as a node with the pointer on lhs and the arithmetic value on rhs. Making pointer addition look similar to that feels practical.


COMPLIANCE functionally equiv to S3 COMPLIANCE mode

This seems to be a copy error, based on my comparison of the COMPLIANCE LOCK and S3's Compliance mode (vs S3's Governance mode).

@kula I suspect you are correct about the copy paste error. 🙃 Also that link is redirecting, I'll update that too.
Thank you for finding this! 🎉

Staged for review @ravindk89
http://<IP_ADDRESS>:9000/staging/feorlen-kula-small-doc-fix/linux/administration/object-management/object-retention.html#governance-mode


Migrating and verifying InfluxDB articles

InfluxDB articles from Help.
~[ ] InfluxDB Series Cardinality~ The issue was resolved, Mika confirmed this article can be archived
[ ] InfluxDB Retention Policies
[ ] Getting started with Aiven for InfluxDB
[ ] Available InfluxDB commands
[x] I plan to open a pull request for this

Would be possible to open a separate PR for each of those topics? It makes sense for the review process. Thanks!

I checked with @FloorD and will pick these articles to be migrated.

Fixed here: https://github.com/aiven/devportal/pull/1566/


Can't use timestamps plugin from inside a repo

My repo look like
    class Entries < IngestService::Repository[:entries]
      relations :ingest_steps
      commands :create, use: :timestamps
   end

I expect that calling Entries.new.create(args) will add timestamps to my entry, but that is not the case.
The Timestamp plugin get's applied at runtime when I call create, only then ClassInterface is extended adding timestamps method that can be used to configure the colmuns. Without calling timestamps the InstanceMethods will never be included and thus timestamps never added.
Unfortunately my understanding of the plugins is very limited, they look like being targeted mostly to commands, will love some pointers for the right direction.

One easy way to solve it is for the timestamp_column to have default values, but is not complete as one won't be able to configure them from a repo.

Ah, right. I'm sorry I forgot we didn't make timestamps plugin work with repos yet. This will be addressed in rom 4.0 as we need to improve plugin architecture first.

This issue was moved to rom-rb/rom#467


featurerequest: Prosody or rate control

Dear @synesthesiam,
Thank you so much for creating this and making it available. publicly I apologize if this is not the appropriate place to contact you regarding this, and I should clarify that this is not a true feature request. I have simply tagged it that way so that it has a nicer name in the issue tracker.
I was curious to know if there is any way currently to either tune the network during training, or to adjust larynx2 during operation, to support different rates of speech? I imagine if I were to record an entire speaker's worth of audio at a different rate and train using that, it would create a model that spoke at that different speed. I was just wondering if rate of speech is essentially fixed in these neural model implementations once trained. I am very new to TTS and RNN, and so am not really certain what is technically easy vs feasible but challenging vs impossible.
Thanks to you sharing this code, I am also now seeing what training an RNN might be like and am making headway in setting up training on the LJSpeech dataset to get an idea of how to go about training a voice. How long does your training take per epoch so that I may have a ballpark? At the moment it seems that it will take about 1 hr/epoch for the LJSpeech-1.1 dataset, but I have no frame of reference to determine whether that is fast, slower or about expected. 1000 hours to do 1000 epochs seems like it would be quite a long time to leave this running though. Of course, I know that all of this is relative to the hardware on which things are being run. I am currently running the training through WSL2, with CUDA through a GeForce GTX (not RTX) 1650 on a intel i7-9750. I had to reduce the batch_size down to 4 from the suggested 32 just to enable to training steps to run without PyTorch complaining about RunTime Error: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 4.00 GiB total capacity; 3.37 GiB already allocated; 0 bytes free; 3.45 GiB reserved in total by PyTorch). so I know that there are probably some significant performance hits in general from the WSL2 indirection and from the underspec'ed GPU.
Thank you for any information you have the time to share,
@d-r-a-b

Aha! I have just discovered the --length-scale option which answers my main reason for posting this. Thanks again!
I am still curious about the training questions I posed, but I am working through this either way. Thank you for posting the LJSpeech checkpoints as well. Are they worth using as an initial training checkpoint for other voices? If so, is it better for the voices to be tagged specifically as belonging to another speaker?
Thank you again,
@d-r-a-b

Hi @d-r-a-b, you're welcome :slightly_smiling_face:
I would recommend starting with a checkpoint that is for the language you want to train and use --resume_from_checkpoint to fine-tune instead of starting from scratch.
4GB isn't a lot of VRAM these days, so it wouldn't surprise me to see it taking an hour for an epoch of the LJSpeech dataset. If you fine-tune instead with a smaller dataset, each epoch should be quicker though.
What language/voice are you wanting to train?

Thank you for your kind reply and suggestions. I am interested in training en-US/en-UK and have been looking around for datasets. My first task was to verify I could even run a training epoch and export an ONNX. I am glad to say I was able to do so with a few modifications to requirements.txt. The Lessac ONNX file you provide is already very good, as is the ONNX I was able to export from your LJSpeech/low checkpoint. I must sincerely compliment your work, as I think these voices sound much better already than most TTS I have heard in the past, and certainly better than the default Windows TTS voices or the Festival voices.
I have 2 main uses for TTS at the moment. which I understand deviate from the Rhasspy use case and are likely lower priority from the Rhasspy project perspective. 1) Narrate chapters of eBooks while I do other tasks and 2) dictate passages for practice in shorthand, which is a method of fast writing. I have a bash script which I have used to create files at speeds of 60-140 WPM by feeding larynx2 10-word chunks and inserting silence between those groups to create an effective WPM rate. Where the current Larynx2 voices suffer as compared to a real human, and which I would hope additional training would help improve:

Isolated words sometimes do not sound very good because of missing or altered phonemes.
Sometimes pauses between sentences are shorter than those between words, or to state the reverse, sometimes pauses between words are longer than those between sentences. It can just feel unnatural, although this is emphasized by the contrast in how good Larynx2 already is about this in many places. More noticeable on long passages/books.
Elongated vowels do not seem possible. Like BEEEEEEEEEEEEEEEEEEEP, woooooooooooow, uuuuuuuummmmm, oooookaaaaaaaay. To be honest, I have no expectation of training a solution to this.

As I move forward to pick a dataset and train using a checkpoint, a few questions if you have the time/bandwidth to respond:

Is the Lessac checkpoint also available somewhere? It might be a better starting point if I train a en-UK voice.
Is it a bad idea to train a dataset recorded at 22.050 kHz samplerate, starting from a checkpoint that was training on 16 khz samplerate? In the same way, is it possible to train a multi-speaker dataset at different samplerates or should I take care to re-encode the audio if the rates are different?
If I start with the LJSpeech checkpoint, and want to train e.g. RyanSpeech (a male en-US voice), should I tag all of RyanSpeech as belonging to a different speaker, or just leave it at None? (I had to hack-patch preprocess.py since 3 columns in LJSpeech-1.1 does not have speaker in the 2nd column of metadata)
In terms of how multi-speaker models work, is it basically 1 model for each speaker, or is it 1 unified model that uses all speakers to help model the underlying ascii/IPA --> Phoneme Sound mapping?

Thank you again for any help you can offer,
@d-r-a-b

The Lessac checkpoint is available here now: https://drive.google.com/drive/folders/1h1RH4ktA-ppDBbBYUMi8HKFrI2xnfTcY?usp=share_link
Isolated words and odd pauses are largely due to the datasets, which often are single sentences from books. To get it right, you need a dataset that has lots of single words, single sentences, and some multiple sentences. Without this, the model can't learn what to do with end of sentence punctuation, etc.
I would stay with the sample rate the original dataset was trained at, but I haven't really experimented with it. I don't think multi-speaker would work at different sample rates, since the mel spectrograms are calculated with it in mind.
For female/male changes, you don't have to re-tag a speaker. In fact, you can fine-tune between languages. I've started with the LJSpeech checkpoint to train many of the other voices from different languages. Regarding the preprocess.py hack, I added a --single-speaker option to force it to not interpret the second column as a speaker id.
Multi-speaker models are 1 unified model, but there are extra parameters for speaker embeddings. The gin_channels parameter controls this. I've found that when all of the speakers come from the same recording environment, a multi-speaker model works well. But in other cases where they come from different datasets, it can sound bad.


How to update container environment variables

I'm switching my application's deployment to CodePipeline and the ECS. Is there a way using this API to update the container's environment variables? Previously I was using the CLI to register the task definition as follows:
export ENVIRONMENT=$(<environment.json)
aws ecs register-task-definition --family <family_name> --task-role-arn <iam_role> --container-definitions "$ENVIRONMENT"
aws ecs update-service --cluster <cluster_name> --service <service_name> --task-definition <family_name> --region us-west-2

And whatever variables were set in environment.json would be loaded into the container. I can't find any documentation describing how to set this using CloudFormation. Any ideas?

Hi! The native CodePipeline -> ECS integration will only update the container definitions' image attribute so you cannot use it to manage environment variables. Luckily, you have a couple of other options:

You can use a Lambda function instead to drive your deployment and do something similar to the above to edit both the image and environment attributes.
If you're using CloudFormation to manage your task definition and service, you can use these templates to manage those fields instead of the native integration. See the documentation for more details and a previous iteration of this refarch for an example.

Hope that helps!

Thank you for the quick response!


Add list of professional references

This CV is literally awesome, very close to perfection (at least for my needs).
To make it even better, could anyone help me add a list of professional references by the end of the CV?
I've found a discussion with a potential solution here: (http://tex.stackexchange.com/questions/34881/references-section-in-a-cv). But adapting it to this Awesome-CV template is far beyond my (basic) Latex skills.
Can anyone more proficient give a little hand?
Thank you in advance!

Crosspost

Could you provide with the solution here.. The link doesn't seem to show the comment discussing the topic.

The following code snippet is the modified version of stack exchange code to work with Awesome CV
`\cvsection{References}
\newcommand{\cvdoublecolumn}[2]{%
\begin{minipage}[t]{{.45\linewidth}}#1\end{minipage}%
\hfill%
\begin{minipage}[t]{{.45\linewidth}}#2\end{minipage}%
}
\newcommand{\cvreference}[7]{%
\textbf{#1}\newline% Name
\ifthenelse{\equal{#2}{}}{}{#2\newline}%
\ifthenelse{\equal{#3}{}}{}{#3\newline}%
\ifthenelse{\equal{#4}{}}{}{#4\newline}%
\ifthenelse{\equal{#5}{}}{}{#5\newline}%
\ifthenelse{\equal{#6}{}}{}{\faEnvelope\acvHeaderIconSep~\texttt{#6}\newline}%
\ifthenelse{\equal{#7}{}}{}{#7}}
\cvdoublecolumn{\cvreference{Nicolai Reshetikhin}
{Department of Mathematics}
{University of California}
{Berkeley, CA 94720-3840}
{}
<EMAIL_ADDRESS>{510-643-6234}%
}
{\cvreference{Mikhail Khovanov}
{Department of Mathematics}
{Columbia University}
{990 Broadway}
{New York, NY 10027}
<EMAIL_ADDRESS>{212-854-4186}%
}`

@aklife97 thanks for the code, that's perfect!

Building on top of the code snippet by @aklife97 , I made a few modifications to closer align the looks with the template:
\cvdoublecolumn{\cvreference{Best reference} 	{Department of Superiority} 	{University of the Universe} 	{} 	{Address 42, 4242}<EMAIL_ADDRESS>{1337}% } {\cvreference{Best reference} 	{Department of Superiority} 	{University of the Universe} 	{} 	{Address 42, 4242}<EMAIL_ADDRESS>{1337}% }

Building on top of @aklife97 code snippet, I added a few modifications to closer align the style with the template.

added missing icons
fixed bold name (solution is somewhat hacky)
use font configuration consistent with the rest of the template

\cvsection{References}

\newcommand{\cvdoublecolumn}[2]{%
	\begin{minipage}[t]{{.45\linewidth}}#1\end{minipage}%
	\hfill%
	\begin{minipage}[t]{{.45\linewidth}}#2\end{minipage}%
}

\newcommand{\cvreference}[7]{%
	\entrytitlestyle{#1}\newline% Name
	\ifthenelse{\equal{#2}{}}{}{#2\newline}%
	\ifthenelse{\equal{#3}{}}{}{#3\newline}%
	\ifthenelse{\equal{#4}{}}{}{#4\newline}%
	\ifthenelse{\equal{#5}{}}{}{\faHome\acvHeaderIconSep~#5\newline}%
	\ifthenelse{\equal{#6}{}}{}{\href{mailto:#6}{\faEnvelope\acvHeaderIconSep~#6\newline}}%
	\ifthenelse{\equal{#7}{}}{}{\faMobile\acvHeaderIconSep~#7\newline}}

\cvdoublecolumn{\cvreference{Best reference}
	{Department of Superiority}
	{University of the Universe}
	{}
	{Address 42, 4242}
<EMAIL_ADDRESS>	{1337}%
}
{\cvreference{Best reference}
	{Department of Superiority}
	{University of the Universe}
	{}
	{Address 42, 4242}
<EMAIL_ADDRESS>	{1337}%
}

Result:

@mbuet2ner That's great! Is there any way to decrease the size of the font though? The built in font size macros (\footnotesize, \tiny, \small etc.) don't work within the template.

@StefanVeleski glad you like it. I have no idea to be honest. Maybe check for some predefined text snippet in the class file?

The following code snippet is the modified version of stack exchange code to work with Awesome CV
\cvsection{References}

\newcommand{\cvdoublecolumn}[2]{%
    \begin{minipage}[t]{{.45\linewidth}}#1\end{minipage}%
    \hfill%
    \begin{minipage}[t]{{.45\linewidth}}#2\end{minipage}%
}

\newcommand{\cvreference}[7]{%
    \textbf{#1}\newline% Name
    \ifthenelse{\equal{#2}{}}{}{#2\newline}%
    \ifthenelse{\equal{#3}{}}{}{#3\newline}%
    \ifthenelse{\equal{#4}{}}{}{#4\newline}%
    \ifthenelse{\equal{#5}{}}{}{#5\newline}%
    \ifthenelse{\equal{#6}{}}{}{\faEnvelope\acvHeaderIconSep~\texttt{#6}\newline}%
    \ifthenelse{\equal{#7}{}}{}{#7}}

\cvdoublecolumn{\cvreference{Nicolai Reshetikhin}
    {Department of Mathematics}
    {University of California}
    {Berkeley, CA 94720-3840}
    {}
<EMAIL_ADDRESS>    {510-643-6234}%
    }
    {\cvreference{Mikhail Khovanov}
    {Department of Mathematics}
    {Columbia University}
    {990 Broadway}
    {New York, NY 10027}
<EMAIL_ADDRESS>    {212-854-4186}%
    }

love it. Thanks

@StefanVeleski To change the style you can use predefined styles:
https://github.com/posquit0/Awesome-CV/blob/4be6c213c8631a1fd68d1ad452e88b722b911f78/awesome-cv.cls#L186-L200
or you can define your own.
Here's an example:
\cvsection{References}

\newcommand*{\emaillink}[1]{{\fontsize{8pt}{1em}\bodyfont\scshape\color{awesome} #1}}

\newcommand{\cvdoublecolumn}[2]{%
	\begin{minipage}[t]{{.45\linewidth}}#1\end{minipage}%
	\hfill%
	\begin{minipage}[t]{{.45\linewidth}}#2\end{minipage}%
}

\newcommand{\cvreference}[5]{
	\entrytitlestyle{#1}\newline
	\descriptionstyle{#2}\newline
	\descriptionstyle{#3}\newline
	\faHome\acvHeaderIconSep~\descriptionstyle{#4}\newline
	\href{mailto:#5}{\faEnvelope\acvHeaderIconSep~\emaillink{#5}\newline}}

\cvdoublecolumn{\cvreference{Best reference}
	{Department of Superiority}
	{University of the Universe}
	{Address 42, 4242}
<EMAIL_ADDRESS>}
{\cvreference{Best reference}
	{Department of Superiority}
	{University of the Universe}
	{Address 42, 4242}
<EMAIL_ADDRESS>}

I hope this helps.


Add support for image captions

How about adding captions so images? For my needs, I am using this approach, but it would be nice to have it out of the box in Pixyll.

FWIW, I'm not 100% sure that a theme could add this feature.

you could do something like this:
_![this is a big image](https://i.ytimg.com/vi/gLg6qxkQ94A/maxresdefault.jpg)_
That's going to leave a mark

Check out the citation footer in @johnotander's original example.
Hope this helps.


Cleanup old branches?

This repository has a bunch of branches that have been removed and should probably be cleaned up;

README-updates
naming
no-register-const
better-logo

@antonblanchard - Could you look at this?


Added make command to help with minor release docs generation

Issue #, if available:
Description of changes:
Hacky, kinda bad, but mostly working change to generating minor release doc changes
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm

/approve


revise Bayesian no-pooling models

tests don't pass locally, but i need the working stuff merged to test an mda.streams build.
* checking tests ...
  Running 'testthat.R' ERROR
Running the tests in 'tests/testthat.R' failed.
Last 13 lines of output:

  3. Failure: predict_metab works as expected for bad inputs (@test-predict.R#85)
  all(mp$msgs.fit == "       ") isn't true.

  testthat results ================================================================
  OK: 382 SKIPPED: 2 FAILED: 3
  1. Failure: metab_sim predictions (predict_metab, predict_DO) make sense (@test-metab_sim.R#67)
  2. Failure: predict_metab works as expected for bad inputs (@test-predict.R#69)
  3. Failure: predict_metab works as expected for bad inputs (@test-predict.R#85)

  Error: testthat unit tests failed
  Execution halted
 [19m]
Warning message:
running command '"C:/PROGRA~1/R/R-33~1.1/bin/x64/R" CMD BATCH --vanilla  "testthat.R" "testthat.Rout"' had status 1


[Feature Request]

Is your feature request related to a problem? Please describe.
I would like to use a function from the "Custom JMeter Functions" plugin in my Azure Load Test.
Describe the solution you'd like
In this blog it says I will need to wait for a new release of Azure Load Testing to be able to use JMeter plugins, but in this blog you have to add some JMeter plugins to your local JMeter in order to run the test.
So are some plugins included already in Azure Load Testing, and for others we will need to wait for a new release?
Describe alternatives you've considered
n/a
Additional context
n/a
AB#1587795

@mhoult22 we support plugins that are on https://jmeter-plugins.org/. If you'd like to upload custom jars then that feature is coming soon.

Hi @ninallam, although it is called "Custom JMeter Functions", it is actually a plugin from https://jmeter-plugins.org/ , the link is https://jmeter-plugins.org/wiki/Functions/ .
If I upload the jar along with the JMeter file should it work, or do I need to do something additional?  I tried uploading the jar along with the jmx but it didn't work for me.  Thanks.

For plugins in https://jmeter-plugins.org/ you don't need to upload a jar. It should work out of the box. Is this not working for you?

No, it doesn't seem to be working for me.
I have a jmx with a UDV value of ${__env(mythread,,1)}.  I created this simplified jmx just to test this issue.

If on my Windows laptop I have an environment variable for mythread and set a value then JMeter uses that value.

If there is no environment variable then JMeter uses the default value of 1 in this example.  For this to work locally I had to add the plugin mentioned above.
When I run the same jmx in Azure Load Testing the env function is not processed at all and I just get ${__env(mythread,,1)} in my results.

@mhoult22 can you please drop an email to<EMAIL_ADDRESS>and we'll help you with this?

Closing this since it was resolved over email.


Investigate DB client_api assertion in send_packet_wait_replies_async

While repeatedly running the TCP resume test I occasionally see assertion failures from the db_client lib:
...
Traceback (most recent call last):
  File "/home/kll/terastream/acton/test/./test_db.py", line 349, in test_app_tcp_recovery
    tcp_cmd(self.p, app_port, "INC")
  File "/home/kll/terastream/acton/test/./test_db.py", line 91, in tcp_cmd
    return tcp_cmd(p, port, cmd, retries-1)
  File "/home/kll/terastream/acton/test/./test_db.py", line 78, in tcp_cmd
    raise TcpCmdError(f"Process is dead, returncode: {p.returncode}  stdout: {p.stdout.read()}  stderr: {p.stderr.read()}")
TcpCmdError: Process is dead, returncode: -6  stdout: b'#RTS# Detected 32 CPUs: Using 32 worker threads for 1:1 mapping with CPU affinity set.\n#RTS# Using distributed database backend replication factor of 3\n#RTS# Using distributed database backend (DDB): <IP_ADDRESS>:17400\n#RTS# Using distributed database backend (DDB): <IP_ADDRESS>:17401\n#RTS# Using distributed database backend (DDB): <IP_ADDRESS>:17402\n#RTS# Checking for existing actor state in DDB.\n#RTS# No previous state in DDB; Initializing database...\n#RTS# Database intialization complete.\nGot connection\nConnection from localhost\nRECV GET\nRECV \nConnection from localhost\nGot connection\nRECV INC\n'  stderr: b"ddb_test_server: backend/client_api.c:693: send_packet_wait_replies_async: Assertion `0' failed.\n"

----------------------------------------------------------------------
Ran 1 test in 0.068s

FAILED (errors=1)
make: *** [Makefile:47: test_db_app_tcp] Error 1
kll@Boxy:~/terastream/acton/test$

That function:
int send_packet_wait_replies_async(void * out_buf, unsigned out_len, int64_t nonce, msg_callback ** mc, remote_db_t * db)
{
	int ret = 0;
	*mc = add_msg_callback(nonce, NULL, db);

	if(*mc == NULL)
	{
		assert(0);   // <<<<<<<< this assert

		return -1;
	}

how can that be, when is the message callback null?

I think this has been fixed through one or more of the many fixes from @aagapi in the area of segfaults in the DB. I don't think this issue is actionable, so I'm closing it.


WROVER-B Flash Corrupted in Field (IDFGH-2932)

Environment

Development Kit: none
Module or chip used: ESP32-WROVER-B
IDF version: v3.3.1
Build System: Make
Compiler version: xtensa-esp32-elf-gcc (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a) 5.2.0
Operating System: macOS
Power Supply: Battery

Problem Description
Short version: A small percentage of boards in the field have had their flash corrupted so that they are unable to boot. It appears the first half or more of the flash has been overwritten with random data.
Long version: I started this as a forum thread at ( https://esp32.com/viewtopic.php?f=2&t=14719 ) and there are significant details there.
Customers report a boot loop. Upon receipt of the device the console shows a loop of:
rst:0x10 (RTCWDT_RTC_RESET),boot:0x3b (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun  8 2016 00:22:57

I used esptool.py to pull the flash image off the board and found that the bootloader, partition table, ota_data, first application image and part of the second application image are overwritten with what appears to be random data. The only flash writing that my firmware does is via NVS and OTA APIs. I do not access the flash directly.
I have a hunch that when this has happened it is during rapid power cycles, perhaps due to low battery brownout. I have not been able to confirm that in person yet, but I see some evidence of it.
Reflashing the board via UART recovers it just fine and it operates normally.
A few findings that might be of significance:

I noticed that even the first 0x1000 bytes of flash contain the random data. On good boards I've seen that this is instead 0xff. I don't flash anything to that area, but I don't know if the ESP uses it internally for anything. If it doesn't, it seems odd to me that it would contain data.
3.3v EFUSE is set during provisioning, and I verified it was still set on the board. We use MTDI for other purposes so this is required in our use case.
The entropy (calculated with ent command line tool) of the bad image is twice that of a corresponding good image.

This issue has affected a small but significant number of devices in the field. It results in a completely bricked device that requires return. I'd really appreciate some help or ideas on how this could be happening.
Thanks,
Jason

the first half or more of the flash has been overwritten with random data

One possibility is that it is jumping into some rom flash write loop due to glitching and you may notice that nothing has been erased and all bit changes are 1->0.  If you have the dangerous flash writes blocked then it probably isn't happening due to memory corruption in higher level functions.  Although it may be possible if it is wrapping around from an out of bounds flash address.

Thanks @negativekelvin - I do have dangerous writes blocked.
I do believe it's likely that the writes are all 1->0 as there are no regions of 0xff at all. Additionally, there are no human readable strings in the flash either, so I think it's unlikely that it's writing random chunks of RAM to flash - it really seems to be random.
Given this, do you have any suggestions for what I can do about it? Is there a way to write lock the flash until power is stable or anything like that?

There are some things you could try like go into deepsleep forever when you detect a low voltage if there is a way to hard reboot once someone charges the battery.  But you might have to add a voltage supervisor or pmic.

Hi @vonnieda ,
Are you able to please send me privately a full flash dump from a "bad" device and a flash dump from a "good" device (ideally one which has never had this issue). Can email to angus at espressif com.
You say your device goes into a brownout state due to low battery, can you please give some more details about how the unit is powered, what other voltage regulator or protection circuitry (if any) is present, etc? What VCC range(s) is the ESP32 exposed to while "enabled" (ie EN not pulled low)?

Hi @projectgus, thank you for responding. Unfortunately, sending my images will not be possible due to the proprietary nature of our product.
I do think I could probably send sections of the image, or perhaps images where I've erased every other 16 bytes, or something like that. And I could of course send small sections from specific offsets. Would that be enough of a sample to help?
As for the power situation: We are single cell lithium ion powered with a buck. The board runs at 3.3v. We have an on board IO controller that handles power management and sequencing. We bring up power before EN, but this is where I think that some devices might have had an issue. I believe the IO controller may have had a problem where the output of those signals was unknown or rapidly shifting. I haven't seen this in person - it's just a hunch.
Would you say that a condition where power was unknown and EN was changing, floating, or unknown could cause a situation like this?
Thanks,
Jason

Hi @vonnieda,
Sure. How about a private email with a flash dump of the first 0x2000 bytes of a "good" and a "bad" board? This is mostly bootloader and the "random" data which appears not to be a valid app, so it shouldn't contain anything proprietary.

Would you say that a condition where power was unknown and EN was changing, floating, or unknown could cause a situation like this?

I can think of two possible scenarios here:
One is, the WROVER-B datasheet recommends using an external supply voltage supervisor if there is any possibility the voltage drops below 2.3V, in this case the CHIP_PU/EN pin should be driven low. This is because of the flash chip voltage range. If the chip tries to start up in this condition (ie VCC <2.3V but CHIP_PU not driven low) then it's possible the flash chip became corrupted.
Second possibility, the recommended WROVER-B VCC range is 3.0V - 3.6V. If the Brownout detector is not enabled (or threshold set too low) and the voltage falls below 3.0V while writing to the flash, it may have caused data to be written to an invalid address - similar to what was discussed above. Similarly if there is a lot of electrical noise for some reason. It may also have corrupted enough of the data signal that the values in the flash look random, although this does seem like more of an unlikely possibility.
If the IO controller you mention is causing EN to fluctuate when a brownout occurs or under some other borderline conditions, then it could make either of these scenarios worse.

Thank you @projectgus  - files have been sent.

Hi @vonnieda ,
Got the dumps, thanks for that.
I agree that the values written here look quite random.
One thing I see is that starting from address 0x174c, there are bits set in the corrupt dump which are not set in the "good" dump. This is something that a rogue SPI flash write command cannot do - those sectors of the flash need to be erased first (to all 0xFF) with an erase command, and then the bits cleared by a separate write command.
So my best guess is that the integrated flash chip has been operated out of spec at some point (probably voltage spec) and it's caused physical corruption of the actual flash cells, rather than anything that the ESP32 software has told it to do.
It does seem extremely odd to me that the whole flash chip would fail in this way, rather than just a few corrupt bits flipping. And I can't explain why the setting of random bits starts at address 0x174c rather than 0x1000 (before 0x1000 the bytes are all 0xFF in the "good" dump so there's no possibility to set any new bits.)
I've attached the little Python script that I used to check for set bits in case you'd like to take a look. It also dumps some other simple bitwise operations applied between new and old - I manually had a quick look at those hex dumps to see if any other patterns stood out, but I don't see any.
check_cleared_bits.py.txt
Angus

Thank you @projectgus, for looking into this, and for your help and advice. Based on this we will try to reproduce this by forcing the condition and work on and fix.
Thanks,
Jason

@vonnieda Thanks for reporting. Would you please help share if any updates or details for this issue? Thanks.

@Alvin1Zhang No further updates from my end. I believe the voltage issue is probably correct, but have not yet had time to try to duplicate it in the lab.

Thanks for the update, @vonnieda . I'm going to close this as it seems like the root cause is most likely not specific to ESP-IDF, but if you find further information to the contrary then please let us know or open a new issue if it looks like ESP-IDF is doing something wrong here.

@vonnieda can you please share any information about how you fixed this in your product? We are running into a similar (perhaps same?) issue where we are seeing random flash corruption on our products.

@shreyasbharath I was never able to reproduce this in the lab on a regular basis, but as a protective measure we started adding a voltage supervisor that ensures ESP_ENABLE stays low until the voltage is good, and that seems to have resolved it.

@vonnieda thanks for the response!
It turns out that at least one of the issues we are experiencing is related to the change of flash chip (from Giga Devices to XMC), and an overerase issue in the bootloader.

We went through this with ~3000 chips in the field :-( It cost us a ton of money.
The fix and explanation is on our website (link below)
Good news: You can integrate it in your own next firmware update
Bad news: You have to do it before your modules fail.
https://en.hoerbert.com/technology/esp32-critical-fatal-problem-source-in-some-wrover-e-modules/


[Questions] Only calling Dataloader once per epoch? Training classifier from scratch at each cycle?

at  synse/synse_training.py
def train_one_cycle(cycle_num, cycle_length):
...
    for epoch in range(s_epoch, e_epoch):
...
        (inputs, target) = next(iter(train_loader))

if __name__ == "__main__":
...
            train_one_cycle(num_cycle, cycle_length)
...
            zsl_acc, val_out_embs, cls = train_classifier()

the dataloader is only called once per epoch, so only 64 samples are used for training each epoch, and a classifier is trained from scratch after each cycle.

Did you test training with all training samples per epoch and concluded only training with a few samples per epoch gave better results?

Did you test training the classifier from where it left off last cycle and concluded training from scratch each time gave better results?

I believe we could have used a better naming convention for the variables. The variable 'epoch' here refers to each iteration. Basically a random batch of 64 samples is loaded each time next(iter(train_loader)) is called. Therefore the VAEs sees all the samples while training.
To answer the second question, no we didn't resume training the classifier from where it was left off in the last cycle, because after each cycle the generated embeddings pertaining to the unseen classes are different, and the classifier is trained such that it optimally classifies the embeddings generated at the end of the current cycle.


the function: training_states_and_actions under class TrackerFeaturizer, does not return anything.

Rasa version: 1.2.9
Rasa SDK version (if used & relevant):
Rasa X version (if used & relevant):
Python version:
3.7
Operating system (windows, osx, ...):
windows
Issue:
When I check the KerasPolicy in the code, and walk through the Class and functions, I found out that the function: training_states_and_actions under class TrackerFeaturizer, does not return anything.
I just wonder that if it does not return anything, how kerasPolicy takes in the input training_data?
Error (including full traceback):
NotImplementedError: Featurizer must have the capacity to encode trackers to feature vectors

Thanks for raising this issue, @JustinaPetr will get back to you about it soon✨
Please also check out the docs and the forum in case your issue was raised there too 🤗

Hi @victorbai2. Looks like you are using a very old version of Rasa. Is there a reason for that?


Not working! Was working before v0.2.0

Was working flawlessly before this new version 0.2.0.
I'm using the correct API token from CircleCi but always gives me "No workflow found".

Can you help us all?

@aPinix thanks for reporting, I can take a look. Here are a couple things that could help debug:

When you start the extension is there any output in Debug Console?
Do you have a Git remote set for origin (git config --get remote.origin.url)?
Does the "Open in browser" button at the top take you to the correct Project in CircleCI?

Is this issue still occurring?

Going to close this issue as it appears inactive. Please re-open with the above requested details if the issue continues.


Rework how istio is handled

Description
So the "old" way (what is currently in main) how we handle istio is described in this image: (press the <--> looking icon to view it in big 😅  )
---
title: Old
---
flowchart LR
    dynakube[Dynakube\nName: test\nEnableIstio: true]
    subgraph operator
        direction TB
        reconcile-everything...
        reconcile-istio
    end
    subgraph for-api-url
        direction TB
        se1[ServiceEntry\nName: some-hash \nPurpose: For the api-url]
        vs1[VirtualService\nName: some-hash \nPurpose: For the api-url]
        se1 -.- vs1
    end
    subgraph for-communication-hosts
        direction TB
        se2[ServiceEntry\nName: some-hash \nPurpose: For oneagent IP-based communication-hosts]
        se3[ServiceEntry\nName: some-hash \nPurpose: For oneagent FQDN-based communication-hosts]
        vs2[VirtualService\nName: some-hash \nPurpose: For oneagent FQDN-based communication-hosts]
        se3 -.- vs2
        se2 ~~~ se3
    end

    exp1([Redundant, as for the operator to get to this point, this should already exist]):::bad-red
    exp2([+there is a overcomplicated delete logic, as the names constantly change, because it's hash based on the contents\n+If any error happens, we just log it, Which can silently break functionality]):::bad-red

    dynakube-->operator
    reconcile-istio-->for-api-url
    reconcile-istio-->for-communication-hosts
    for-api-url --> exp1
    for-communication-hosts --> exp2
    exp1-->exp2

    classDef bad-red stroke:#f00

According to this PR how it works:
---
title: New
---
flowchart LR
    dynakube[Dynakube\nName: test\nEnableIstio: true]
    subgraph operator
        direction TB
        reconcile-istio-for-api-url
        reconcile-connectionInfo...
        reconcile-istio-communication-hosts
        reconcile-everything-else...
    end
    subgraph for-api-url
        direction TB
        se1[ServiceEntry\nName: test-dk-fqdn-operator \nPurpose: For the api-url]
        vs1[VirtualService\nName: test-dk-fqdn-operator \nPurpose: For the api-url]
        se1 -.- vs1
    end
    subgraph for-communication-hosts
        direction TB
        se2[ServiceEntry\nName: test-dk-ip-oneagent \nPurpose: For oneagent IP-based communication-hosts]
        se3[ServiceEntry\nName: test-dk-fqdn-oneagent \nPurpose: For oneagent FQDN-based communication-hosts]
        vs2[VirtualService\nName: test-dk-fqdn-oneagent \nPurpose: For oneagent FQDN-based communication-hosts]
        se3 -.- vs2
        se2 ~~~ se3
    end

    exp1([No longer redundant, user no longer need to configure istio for the API Url by hand]):::good-green
    exp2([Update logic is simple as the names are static + names mean something\nincase of error, we actually stop what we are doing]):::good-green

    dynakube-->operator
    reconcile-istio-for-api-url-->for-api-url
    reconcile-istio-communication-hosts-->for-communication-hosts
    for-api-url --> exp1
    for-api-url --> exp2
    for-communication-hosts --> exp2

    classDef good-green stroke:#0f0

TLDR:
How we used to handle the istio integration was less then ideal:

The user had to do extra steps for the operator to even get to the isito reconcile part
The resources we created had a hash for a name, that only complicated things and confused users
The resources we created had weird labels (that we were afraid to change as they were used as labels selector due to the names being unpredictable)
The code of the istio reconciler was "refactored" multiple times, but was never re-thinked, so the code itself was overly convoluted for what it was doing.

Still TODO:

fix e2e tests...

How can this be tested?

have a cluster with istio with REGISTRY_ONLY mode
have istio inject the operator's namespace
deploy a dynakube with EnableIstio: true
it should just work

previously you would need to create a ServiceEntry + VirtualService for your APIURL to get to the istio reconcile part, you DON'T have to do this anymore
The created ServiceEntries + VirtualServices should be named nicely and have nice labels.

Checklist

[x] Unit tests have been updated/added
[x] PR is labeled accordingly with a single label
[x] I have read and understood the contribution guidelines

Codecov Report

Merging #2105 (3dd7793) into main (84be803) will increase coverage by 0.41%.
Report is 30 commits behind head on main.
The diff coverage is 80.21%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@            Coverage Diff             @@
##             main    #2105      +/-   ##
==========================================
+ Coverage   63.13%   63.55%   +0.41%
==========================================
  Files         242      243       +1
  Lines       15321    15386      +65
==========================================
+ Hits         9673     9778     +105
+ Misses       4908     4884      -24
+ Partials      740      724      -16

Files Changed
Coverage Δ

src/kubeobjects/hasher.go
13.95% <0.00%> (-7.48%)
:arrow_down:

src/controllers/dynakube/dynakube_controller.go
58.80% <47.05%> (+1.54%)
:arrow_up:

src/controllers/dynakube/istio/client.go
88.70% <88.70%> (ø)

src/controllers/dynakube/istio/reconciler.go
88.57% <93.90%> (+13.76%)
:arrow_up:

...ers/dynakube/connectioninfo/communication_hosts.go
100.00% <100.00%> (ø)

src/controllers/dynakube/istio/istio.go
100.00% <100.00%> (ø)

src/controllers/dynakube/istio/serviceentry.go
100.00% <100.00%> (+20.53%)
:arrow_up:

src/controllers/dynakube/istio/virtualservice.go
100.00% <100.00%> (+25.21%)
:arrow_up:

... and 15 files with indirect coverage changes

practically LGTM
PASS
ok      github.com/Dynatrace/dynatrace-operator/test/scenarios/cloudnative/istio        509.196s


Productivity percentage should only apply on supported products

Not every product supports productivity modules. Take science 2 for example and enter 1 sciene2 / s and add 40% productivity bonus. 0.715 Inserters/s need only 0.511 iron / s but thats not true, because inserters don't support productivity.
Would be great if productivity bonus is only applied on supported products.
Supported products are listed in this file:
/factorio/data/base/prototypes/item/module.lua
function productivitymodulelimitation()
return
      {
        "sulfuric-acid",
        "basic-oil-processing",
        "advanced-oil-processing",
        "coal-liquefaction",
        "heavy-oil-cracking",
        "light-oil-cracking",
        "solid-fuel-from-light-oil",
        "solid-fuel-from-heavy-oil",
        "solid-fuel-from-petroleum-gas",
        "lubricant",
        "wood",
        "iron-plate",
        "copper-plate",
        "steel-plate",
        "stone-brick",
        "sulfur",
        "plastic-bar",
        "empty-barrel",
        "uranium-processing",
        "copper-cable",
        "iron-stick",
        "iron-gear-wheel",
        "electronic-circuit",
        "advanced-circuit",
        "processing-unit",
        "engine-unit",
        "electric-engine-unit",
        "uranium-fuel-cell",
        "explosives",
        "battery",
        "flying-robot-frame",
        "low-density-structure",
        "rocket-fuel",
        "rocket-control-unit",
        "rocket-part",
        "science-pack-1",
        "science-pack-2",
        "science-pack-3",
        "military-science-pack",
        "production-science-pack",
        "high-tech-science-pack"
      }
end

Thanks for the list, it will be useful!

I handled this by adding another property to the resource list, productivityallowed, as a float.  Defaulted it to 0, but set it to 1 for any item in the above list.  Then just multiplied productivity_bonus in line 441 of factoriojs.ml by resource.productivityallowed.

Done!
Thanks Sadumon for the implementation idea, this helped! I did almost exactly as you suggest.

Is this working on the current published version?? I just tested by hand for science pack 2, any increase in prod% changes the # of yellow inserter assemblies..

Would like to report that speed modules lv 1 suffers from this problem also. It do not support prod modules but people stick x4 extra speed mods and when i change the speed bonus for assembly machines it affects all subitems as well and cause the calculations to be wrong.

@werelord and @Grokzen : it seems to me that it is working fine.
I tested with science pack 2. If I set productivity bonus to 100%, it requires half as many inserters, not because inserters benefit from the productivity bonus but because the science pack does, so producing the same amount of science packs requires half the amount of inserters :)
Or maybe you are not using http://www.doomeer.com/factorio but https://dwahler.github.io/factorio ? I don't know if this clone is up-to-date.


Relationships not being synced since 1.0.10

I updated from 1.0.8 to the latest 1.0.11 and noticed relationships are no longer being sync'd; they show nil.  I tried 1.0.10 with the same results, then 1.0.9 which works.  Did something change between 1.0.9 and 1.0.10 that I'm not aware of that requires an update on my end?  I checked the README and didn't see anything...

Hi @bonebox!
We use the release feature from GitHub to present the changelog.
https://github.com/hyperoslo/Sync/releases
The biggest thing that has changed since 1.0.8 in Sync was how predicates where handled. => https://github.com/hyperoslo/Sync/issues/119 https://github.com/hyperoslo/Sync/pull/120
Maybe this was the issue?

Wait, you're saying that 1.0.9 works fine, right? Nothing critical changed on 1.0.11 so it must be 1.0.10 then.
1.0.10 only includes this change => https://github.com/hyperoslo/NSManagedObject-HYPPropertyMapper/releases/tag/3.3.0

1.0.10 only includes this change => https://github.com/hyperoslo/NSManagedObject-HYPPropertyMapper/releases/tag/3.3.0 which involves hp_dictionary.

This is probably it.  From this, it looks like you are using nested objects as the default, whereas my JSON objects are in array format.  So how do I get Sync to still work with the latter?

This is a bit strange, since before hyp_dictionary also was doing nested objects. Are you using that method in your code? hyp_dictionary.
Would be great if you could help me to figure out what is failing since the Sync unit tests are working fine after the update, there must be something I'm missing.

No, I'm not directly calling methods from any of Sync's dependencies.  The only method I'm calling is changes:inEntityNamed:predicate:dataStack:completion:.  Then when I fetch the newly inserted entity, the attributes are mapped fine, but any relationships are nil.  I'm happy to help, but not sure what would be useful for you.  Let me know if you'd like me to test anything specific.

@bonebox could you provide please some insight of your Core Data model and the used JSON?
It would be super helpful. :heart:

See below for the core data graph and JSON.  I'm importing the JSON as a Form entity.  Both the model and element relationships are nil with 1.0.11 and 1.0.10, and everything gets mapped fine in 1.0.9 and prior.  Let me know if you need any more specifics on the model or anything else.
{
    "model": {
        "uri": "http://tempuri.org/any/path/mymodel/1.0/",
        "type": "http://tempuri.org/any/path/mymodel/",
        "version": "1.0",
        "properties": [
                       {
                       "id": "anyString",
                       "type": "string",
                       "restrictions": [
                                        {
                                        "restriction": "enumeration",
                                        "items": [
                                                  "option1",
                                                  "option2",
                                                  "option3"
                                                  ]
                                        }
                                        ]
                       },
                       {
                       "id": "anyBoolean",
                       "type": "boolean",
                       "restrictions": [
                                        {
                                        "restriction" : "minLength",
                                        "value" : 2
                                        },
                                        {
                                        "restriction" : "maxLength",
                                        "value" : 50
                                        }
                                        ]
                       },
                       {
                       "id": "anyDecimal",
                       "type": "decimal"
                       },
                       {
                       "id": "anyInteger",
                       "type": "integer"
                       },
                       {
                       "id": "anyDateTime",
                       "type": "dateTime"
                       },
                       {
                       "id": "anyDate",
                       "type": "date"
                       },
                       {
                       "id": "anyDuration",
                       "type": "duration"
                       },
                       {
                       "id": "anyMultimedia",
                       "type": "multimedia"
                       },
                       {
                       "id": "anyLocation",
                       "type": "location"
                       }
                       ]
    },

    "uri": "http://tempuri.org/any/path/mydescriptor/1.2?language=en-US",
    "type": "http://tempuri.org/any/path/mydescriptor/",
    "version": "1.2",
    "language": "en-US",
    "element" : {
        "type" : "group",
        "label" : "Any Form",
        "elements" : [
                      {
                      "type": "input",
                      "label": "Any Input",
                      "reference": "anyInteger",
                      "inputConfiguration": {
                      "justAHint" : ["a", "b", "c"]
                      }
                      },
                      {
                      "type" : "group",
                      "label" : "Any Group",
                      "elements" : [
                                    {
                                    "type": "selectSingle",
                                    "label": "Any Single Select",
                                    "reference": "anyString",
                                    "selection": "closed",
                                    "items": [
                                              { "value": "option1", "label": "Option One" },
                                              { "value": "option2", "label": "Option Two" }
                                              ]
                                    },
                                    {
                                    "type": "selectMultiple",
                                    "label": "Any Multiple Select",
                                    "reference": "anyString",
                                    "selection": "open",
                                    "items": [
                                              {
                                              "value": "option3",
                                              "label": "Option Three",
                                              "aliasLabels" : [
                                                               { "label": "Option Three A" },
                                                               { "label": "Option Three B" }
                                                               ]
                                              },
                                              { "value": "option4", "label": "Option Four" }
                                              ]
                                    }
                                    ]
                      },
                      {
                      "type": "textArea",
                      "label": "Any Text Area",
                      "reference": "anyString",
                      "comment": "This is a Comment for Any Text Area"
                      },
                      {
                      "type": "image",
                      "label": "Any Image",
                      "reference": "anyMultimedia",
                      "maximumFileSize" : 12000
                      },
                      {
                      "type": "location",
                      "label": "Any Address Location",
                      "reference": "anyString",
                      "locationRepresentation" : "address"
                      },
                      {
                      "type": "custom",
                      "label": "Any Custom",
                      "reference": "anyString",
                      "customType": "http://tempuri.org/path/to/custom/element"
                      },
                      {
                      "type": "input",
                      "label": "Any Input With Keyboard",
                      "reference": "anyString",
                      },
                      {
                      "type": "form",
                      "label": "Any Form Link",
                      "formURI": "http://tempuri.org/path/to/another/form"
                      }
                      ]
        }
}

So strange :disappointed:
I got this running
- (void)testNilRelationshipsAfterUpdating_Sync_1_0_10 {
    NSDictionary *form = [self<EMAIL_ADDRESS>    DATAStack *dataStack = [self dataStackWithModelName:@"Bug125"];

    [Sync changes:@[form]
    inEntityNamed:@"Form"
        dataStack:dataStack
       completion:nil];

    XCTAssertEqual([self countForEntity:@"Form"
                              inContext:dataStack.mainContext], 1);

    XCTAssertEqual([self countForEntity:@"Element"
                              inContext:dataStack.mainContext], 11);

    XCTAssertEqual([self countForEntity:@"SelectionItem"
                              inContext:dataStack.mainContext], 4);

    XCTAssertEqual([self countForEntity:@"Model"
                              inContext:dataStack.mainContext], 1);

    XCTAssertEqual([self countForEntity:@"ModelProperty"
                              inContext:dataStack.mainContext], 9);

    XCTAssertEqual([self countForEntity:@"Restriction"
                              inContext:dataStack.mainContext], 3);

    [dataStack drop];
}

So strange :disappointed:
I got this running
- (void)testNilRelationshipsAfterUpdating_Sync_1_0_10 {
    NSDictionary *formDictionary = [self<EMAIL_ADDRESS>    DATAStack *dataStack = [self dataStackWithModelName:@"Bug125"];

    [Sync changes:@[formDictionary]
    inEntityNamed:@"Form"
        dataStack:dataStack
       completion:nil];

    XCTAssertEqual([self countForEntity:@"Form"
                              inContext:dataStack.mainContext], 1);

    XCTAssertEqual([self countForEntity:@"Element"
                              inContext:dataStack.mainContext], 11);

    XCTAssertEqual([self countForEntity:@"SelectionItem"
                              inContext:dataStack.mainContext], 4);

    XCTAssertEqual([self countForEntity:@"Model"
                              inContext:dataStack.mainContext], 1);

    XCTAssertEqual([self countForEntity:@"ModelProperty"
                              inContext:dataStack.mainContext], 9);

    XCTAssertEqual([self countForEntity:@"Restriction"
                              inContext:dataStack.mainContext], 3);

    NSArray *array = [self fetchEntity:@"Form"
                             inContext:dataStack.mainContext];
    NSManagedObject *form = [array firstObject];
    NSManagedObject *element = [form valueForKey:@"element"];
    NSManagedObject *model = [form valueForKey:@"model"];
    XCTAssertNotNil(element);
    XCTAssertNotNil(model);

    [dataStack drop];
}

Ok, it seems the problem is in fact related to the predicate. I removed that and the relationships now get sync'd correctly.  But it works with the predicate in 1.0.9.  In the past, I needed the predicate in order to prevent Sync from deleting existing forms in core data whenever I sync a new form.  Has that changed?

@bonebox yes! Could you provide the predicate, please?

let uri = JSON["uri"] as! String
let predicate = NSPredicate(format: "uri == %@", uri)

@bonebox I was able to find the bug, thanks!
Fixed on https://github.com/hyperoslo/Sync/pull/126

Great, thanks!

@bonebox Thanks for your help and sorry for the inconvenience.
https://github.com/hyperoslo/Sync/releases/tag/1.0.12

No worries, sorry you had to rollback some prior changes!

@bonebox Hi Jeremy,
Could you please check if you have the same issue as before on the recently released 1.0.13?
https://github.com/hyperoslo/Sync/releases/tag/1.0.13

Hey @3lvis ,
I can't get it to build.  It looks like you added a new class extension NSArray+Sync.h that is importing DATAStack like import "DATAStack.h" and that is causing the issue.  If seems like I've seen this before when using an Objective-C dependency in a Swift framework (which is what I'm doing).  Anyway, changing it to import <DATAStack/DATAStack.h> let's it build successfully for me.

Hi @bonebox,
I changed the Examples and the Tests suite to use modules now. So after adding use_frameworks to your code everything should work fine.

@3lvis, sure thing and thanks!  Will you be releasing an updated CocoaPod with those changes?  Since I'm using this in a framework with it's own podspec I can't point directly to a specific branch within a podspec like with a Podfile.

@bonebox I did that a while ago, with Sync 1.1.0 => https://github.com/hyperoslo/Sync/blob/master/Sync.podspec

https://github.com/hyperoslo/Sync/releases

Thanks, I see it now. It wasn't showing up for me when I checked earlier.

Ok, updated to 1.1.0, and now I get a crash when calling Sync.changes.  Looks to be an incompatibility with ordered sets.
2015-10-12 16:31:46.229 mForms-Test[5687:468759] -[_NSFaultingMutableOrderedSet hyp_dictionaryWithDateFormatter:parent:usingRelationshipType:]: unrecognized selector sent to instance 0x7fb0db20a760
2015-10-12 16:31:46.231 mForms-Test[5687:468759] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[_NSFaultingMutableOrderedSet hyp_dictionaryWithDateFormatter:parent:usingRelationshipType:]: unrecognized selector sent to instance 0x7fb0db20a760'
*** First throw call stack:
(
	0   CoreFoundation                      0x000000010bb97f65 __exceptionPreprocess + 165
	1   libobjc.A.dylib                     0x000000010d829deb objc_exception_throw + 48
	2   CoreFoundation                      0x000000010bba058d -[NSObject(NSObject) doesNotRecognizeSelector:] + 205
	3   CoreFoundation                      0x000000010baedf7a ___forwarding___ + 970
	4   CoreFoundation                      0x000000010baedb28 _CF_forwarding_prep_0 + 120
	5   NSManagedObject_HYPPropertyMapper   0x000000010b656010 -[NSManagedObject(HYPPropertyMapper) attributesForToOneRelationship:relationshipName:usingRelationshipType:parent:dateFormatter:] + 208
	6   NSManagedObject_HYPPropertyMapper   0x000000010b655ce0 -[NSManagedObject(HYPPropertyMapper) hyp_dictionaryWithDateFormatter:parent:usingRelationshipType:] + 1536
	7   NSManagedObject_HYPPropertyMapper   0x000000010b656010 -[NSManagedObject(HYPPropertyMapper) attributesForToOneRelationship:relationshipName:usingRelationshipType:parent:dateFormatter:] + 208
	8   NSManagedObject_HYPPropertyMapper   0x000000010b655ce0 -[NSManagedObject(HYPPropertyMapper) hyp_dictionaryWithDateFormatter:parent:usingRelationshipType:] + 1536
	9   NSManagedObject_HYPPropertyMapper   0x000000010b6556a0 -[NSManagedObject(HYPPropertyMapper) hyp_dictionaryWithDateFormatter:usingRelationshipType:] + 96
	10  NSManagedObject_HYPPropertyMapper   0x000000010b65557c -[NSManagedObject(HYPPropertyMapper) hyp_dictionaryUsingRelationshipType:] + 92
	11  Sync                                0x000000010b679a99 -[NSArray(Sync) preprocessForEntityNamed:usingPredicate:parent:dataStack:] + 1913
	12  Sync                                0x000000010b67cef2 +[Sync changes:inEntityNamed:predicate:parent:inContext:dataStack:completion:] + 1378
	13  Sync                                0x000000010b67c1c9 __61+[Sync changes:inEntityNamed:predicate:dataStack:completion:]_block_invoke + 121
	14  DATAStack                           0x000000010b63360a __43-[DATAStack performInNewBackgroundContext:]_block_invoke + 58
	15  CoreData                            0x000000010b72bf17 developerSubmittedBlockToNSManagedObjectContextPerform + 199
	16  libdispatch.dylib                   0x000000010f06249b _dispatch_client_callout + 8
	17  libdispatch.dylib                   0x000000010f048988 _dispatch_queue_drain + 2215
	18  libdispatch.dylib                   0x000000010f047ea9 _dispatch_queue_invoke + 601
	19  libdispatch.dylib                   0x000000010f04aaf2 _dispatch_root_queue_drain + 1420
	20  libdispatch.dylib                   0x000000010f04a561 _dispatch_worker_thread3 + 111
	21  libsystem_pthread.dylib             0x000000010f3934f2 _pthread_wqthread + 1129
	22  libsystem_pthread.dylib             0x000000010f391375 start_wqthread + 13
)
libc++abi.dylib: terminating with uncaught exception of type NSException

@bonebox Thanks for giving it a try, I'll check what's going on.

@bonebox I found what was the issue, it should be working again on 1.1.1 thanks for your input, I can't imagine all the bugs if it wasn't for your help. :heart:

Hey @3lvis, glad I can help!  It's the least I can do for getting to use such a nice library :+1:
I tried 1.1.1 and everything looks like it's working properly!  Thanks!


refactor front page

OK I will add something to this PR. I felt it was repetitive but I agree that it may be missing.

I will merge this. It became just a minor change. I think it is better now.


Connects falkonry/tracker#240

Ported to Python3,Rectified test cases

@phagunbaya
I just wanted to check whether my test fixes were working properly on the deployment or not
There are other changes as well which were done in correspondance with falkonry-cli
Kindly wait for the Code review
Will let you know about the same when its completely done


Cannot mount local windows folder using docker for windows

Expected behavior
I have mounted a windows local folder using this sentence:
docker run -v /c/Users/vanay/Downloads/java-example:/eslap/java-example --entrypoint=bash -it name
when listing content of java-example, the folder is void. The volume relationship is listed in Kitematic, and no error is shown in the log.
Actual behavior
Nothing is mounted.
Information about the Issue
Steps to reproduce the behavior

...
...

@vicanfon Make sure you followed the docs on sharing the folder:
https://docs.docker.com/docker-for-windows/#shared-drives


Regex, Header, and Footer Additions for the GUI

Apologies on the delayed pull request for the front-end changes! Here are the changes as per requested.

Hi @slmendez Thanks! I'm aiming to take a look at this this weekend.

Hi @slmendez and everyone --
Thank you again for your hard work on this, it looks awesome. A super useful new set of functionalities. I took a long look at the code and at the interface and I have a few questions, before I start merging, to make sure I understand what stuff is supposed to do. Since you clearly have thought through these new features, I want to make sure I get as much of your thought process as possible.
I love that you all added tests, added social media links to the bottom of the homepage and added comments.
No particular rush; I'll email you all as well.

Does the regex search depend on anything happening during the upload step? (If someone already had PDFs they had uploaded with the current Tabula, then upgraded to Tabula with your changes, would Regex Search work with their older PDFs?
What's the thinking behind making the regex-created selections not deleteable or moveable?
What does PDF Outline button supposed to do? Just hide the sidebar, for more space?
What's this green/blue header stuff for? (Is it right that it excludes areas from regex search?) What's the use case you all had in mind for this?
Can you describe the testing infrastructure that's included here? We obviously didn't have any tests on the Tabula GUI before, but, obviously, it's great. How do I run it? What are the top-level tools you chose to use for the tests? ("we do frontend testing with ____ framework"?) I know basically nothing about testing JavaScript apps like this.

Thanks again, y'all.

I can probably answer the middle three questions well enough, though Shirley may have some feedback/points that I'm not taking into account. We have a "final project documentation" folder available on the shared Google Drive space that may provide more detail on the front-end side of the software that I'm not up-to-date on, myself; I'll email a copy to you and Manuel as well in case that's any more convenient. Let me know if you'd like me to clarify or elaborate on anything below!
- What's the thinking behind making the regex-created selections not deletable or moveable?
I think that the selections are deletable via clicking on their respective searches listed in the box/table to the bottom right of the GUI. Beyond that, we didn't consider the need of moving them with our new functionality.
- What does PDF Outline button supposed to do? Just hide the sidebar, for more space?
I believe that is correct, just more of a minor point that we wanted for viewers' flexibility with screen space.
- What's this green/blue header stuff for? (Is it right that it excludes areas from regex search?) What's the use case you all had in mind for this?
The drag down/up is for headers and footers, primarily to keep Tabula from extracting garbage like page numbers or static annotations/descriptions. It does exclude those areas from the search. When talking with Coleman, we were primarily considering page numbers due to the raw amount of undesired output they create. We'd also discussed the option of incorporating page-by-page header and footer "exclusion regions," though decided that it wouldn't be a good use of development time at this point.

@dbmarsh Thanks! I appreciate all the explanation and the additional documentation on Google. I see now how I can delete a regex search.
More questions on header/footer:
Just in terms of how you all anticipate the header/footer draggables being used:

It's just for excluding headers/footers in regex search selections?  Is it supposed to do anything if I'm doing an old-fashioned click-and-drag selection?
It only makes sense if the regex-defined selection spans pages, right?

And with regex search generally -- and you all may have gone over this in the past, sorry for the repetition -- is the idea that multiple similarly-laid-out PDFs would be processed with the same set of regexes?

- It's just for excluding headers/footers in regex search selections? Is it supposed to do anything if I'm doing an old-fashioned click-and-drag selection?
I can't recall ever working much with the click-and-drag on a machine with our new developments due to our demonstrations primarily being meant to highlight differences from Tabula's original capabilities, but thinking on the back end I believe that the header/footer exclusion checks are carried out alongside regex pair searches so I would say no effect would be desired on click-and-drag functionality. However, I may be mistaken and won't have access to my computer with everything set up to play around with it for the next few days.
- It only makes sense if the regex-defined selection spans pages, right?
There may be some corner cases - e.g. where the regex start or end phrases are found in the header or footer of a single page but these matches are not desired - where the exclusion functionality has the potential to give users additional control over the search capabilities, but in terms of our intentions it was certainly built with multi-page regions in mind.
And with regex search generally -- and you all may have gone over this in the past, sorry for the repetition -- is the idea that multiple similarly-laid-out PDFs would be processed with the same set of regexes?
From our discussion over the course of the semester, it seems that the regex search capabilities would provide the most benefit in two cases: (1) batch processing of similarly-laid-out PDFs; and (2) extremely large PDFs with many instances of matching regions. Most of our testing and demonstration have focused on the former, but there is potential with respect to the latter when considering the alternative would be manually locating and selecting these regions via the click-and-drag approach. In general, though, I agree with the sentiment above.

Hi @jeremybmerrill , sorry for taking me so long to respond back to your questions!
I'll be adding on to what David has answered:
Can you describe the testing infrastructure that's included here? We obviously didn't have any tests on the Tabula GUI before, but, obviously, it's great. How do I run it? What are the top-level tools you chose to use for the tests? ("we do frontend testing with ____ framework"?) I know basically nothing about testing JavaScript apps like this.
The testing framework is built on selenium and all the test cases are written in java. Half of the test cases are just testing the actual page functionality (clicking links, buttons, etc.) and the other half is actual testing with pdf files to produce output and validating the output. The test cases can be run and checked for on Travis CI, Ross had helped me set up the Travis file for it to run and show the results of the test cases on there, please let me know if you have trouble running them!
Also a point to bring up with the test cases is how all of them depend on how long it takes the page to load, so in some cases if out of the blue the page takes too long to load up the test case just fails, despise giving it allotted time for it to wait. I've looked into this issue and it seems to be a pretty common issue with a lot of front-end testing that depend on page load time.
Lastly, in our project final documentation, I believe under the documentation folder we have document files for the front-end and back-end testing. On there it goes more in detail of the pdf files used and what each test case is doing.
Please let me know if you have any more questions!

@slmendez Thanks! I've got the tests running now, I think.
@dbmarsh: Great, thanks! Very helpful to hear the thinking behind this.
Will let y'all know if I have more questions.


[QA] - Transaction is not balanced

submit a not balanced transaction

script for 1.14.0

https://github.com/ArturWieczorek/node-test-scripts/blob/master/python_scripts/e2e_scenarios/transaction_not_balanced_test.py