95 renLuna stuck as WWT on ETH from SOL

sent renLuna from sol to ETH and it just shows as WWT and i cant unwrap or do anything with it. any reasons ?

renLuna is a wrapped token and the wormhole can't unwrap it, you need to send that back and maybe contact solana support to see if there is a workaround

got a fix from the solana discord thanks guys closing this issue!

@anonintern I'm glad it is fix. Btw i'm the same guy as the discord one

renLuna is a wrapped token and the wormhole can't unwrap it, you need to send that back and maybe contact solana support to see if there is a workaround

how do you send a WWT token back to phantom?


Parser.Dep: interpret inline_for_extraction on any decl, not just Vals

This is relevant for Pulse, which uses %splice to generate Vals during
typechecking time, so the parser will miss the qualifier and mess up
the dependency analysis.

This passed our brand spanking new check-world workflow, so I'm merging. https://github.com/FStarLang/FStar/actions/runs/10689266115


MINIFICPP-440 Added LibreSSL as an external project build, statically…

…-linked for cases when it isn't available in the target environment for dynamic linking
Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with MINIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit?

For code changes:

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[x] If applicable, have you updated the LICENSE file?
[x] If applicable, have you updated the NOTICE file?

For documentation related changes:

[x] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Looking into potential Macintosh issue...

@phrocker I inverted the default to the more conservative option for 0.5.0. Would 0.6.0 be the target for switching it back? I created MINIFICPP-441 to track and will set the release of that ticket accordingly.

The travis failure is not related. will merge. Thanks


expandable view in topic page in nsqadmin

tables like coordinator status and message distribution could be expandable.

And also the topic producer connections can be hidden by default.

duplicated with #26 . Closed.


[README] Add System Requirements

Implemented in:

https://github.com/geniusyield/market-maker/pull/30


Error and request

Hi! After a player confirms login in discord, the game doesn't get a successful login message, this doesn't affect the functionality in any way, but I would like something to write in the game after a successful login. Also, could you please add support for hex colours in the minecraft messages. I really love your plugin keep up the good work, thanks!

Thank you for error report and idea


KSQL execution works wrong

Describe the bug

INSERT INTO lets to insert the same data multiple times.
SELECT  from tutorial gives an error message.

Set up

http://issues-1109.internal.kafka-ui.provectus.io/
Steps to Reproduce
Steps to reproduce the behavior:

Navigate to KSQL DB in main menu
Push the button Execute KSQL Request
Execute the query INSERT INTO riderLocations (profileId, latitude, longitude) VALUES ('4ddad000', 37.7857, -122.4011); several times.
Observe that every time the result is Query succeed
Execute the push query SELECT * FROM riderLocations
WHERE GEO_DISTANCE(latitude, longitude, 37.4133, -122.1162) < 5 EMIT CHANGES;
Observe result - empty fields of table PROFILEID STRING LATITUDE DOUBLE   LONGITUDE DOUBLE
Execute the query SELECT * from ridersNearMountainView WHERE distanceInMiles <= 10;
Observe the result "SELECT * from ridersNearMountainView WHERE distanceInMiles <= 10;" Bound on key columns '[DISTANCEINMILES DOUBLE KEY]' must currently be '='. See https://cnfl.io/queries for more info. Add EMIT CHANGES if you intended to issue a push query. Pull queries require a WHERE clause that: - includes a key equality expression, e.g. SELECT * FROM X WHERE <key-column>=Y;. - in the case of a multi-column key, is a conjunction of equality expressions that cover all key columns. If more flexible queries are needed, table scans can be enabled by setting ksql.query.pull.table.scan.enabled=true.
However, it works with = only : SELECT * from ridersNearMountainView WHERE distanceInMiles = 10;
In conclusion no one SELECT (pull query) request from the page https://docs.ksqldb.io/en/latest/developer-guide/ksqldb-reference/select-pull-query/ doesn`t works.

Expected behavior

Screenshots

Additional context

Continuation of ##1109

@agolosen

repeated insert should work without error - there is no concept of uniq key in kafka - following inserts with same primary key just overwrite existing
point 8 -> like message sad to make it work you need to set ksql.query.pull.table.scan.enabled to true. if set - it works fine

For other queries - there are some error in confluent docs, so, need to check every question one by one.


Add TTS Settings

On the rewards screen, add a button to open TTS settings.
Available options should be:

Rate
Volume
Voice

Also add a text box to test the current settings.

The TTS settings have been added, but they do not persist after the application closes. That will be added next.
The options do, however, persist from session to session.

The TTS settings are now persistent. The feature is complete.


fix crash

add

      if(FirebasePlugin.callbackContext == null){
        return;
      }
```
in the first line of
```
 public static void onNotificationOpen(Bundle bundle)
```

Update your plugin it's already fixed since 7 days.


Vulnerability roundup 110: graylog-3.3.16: 2 advisories [9.8]

search, files

[ ] CVE-2021-37759 CVSSv3=9.8 (nixos-21.11, nixos-unstable)
[ ] CVE-2021-37760 CVSSv3=9.8 (nixos-21.11, nixos-unstable)

CVE details
CVE-2021-37759
A Session ID leak in the DEBUG log file in Graylog before 4.1.2 allows attackers to escalate privileges (to the access level of the leaked session ID).
CVE-2021-37760
A Session ID leak in the audit log in Graylog before 4.1.2 allows attackers to escalate privileges (to the access level of the leaked session ID).

Scanned versions: nixos-21.11: 3ddd960a3b5; nixos-unstable: 5aaed40d22f.
Cc @fadenb

Fixed in #197524


Updating R package list

@otarb : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@mgblythe : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Hi @otarb. It looks like you might have a different line ending in your editor b/c this PR show all lines changed. I'll merge this one as-is, but please look into this for next time. Thanks!
@davidiseminger as FYI


Migrate protobuf to google.golang.org.

Hi,
I'd like to know if there's any plan in migrating the usage of github.com/golang/protobuf to google.golang.org/protobuf ?

github.com/golang/protobuf is now implemented in terms of google.golang.org/protobuf, so as far as I know there is no pressing need to migrate.
The github.com/golang/groupcache package certainly cannot be migrated, since that would be a breaking change (and would in turn cause a breaking change in the groupcache.ProtoGetter interface).

Just an outside observer here but github.com/golang/protobuf is deprecated.


Interaction with CSS font matching algorithm

https://drafts.csswg.org/css-fonts/#font-matching-algorithm specifies a fairly precise algorithm. font-kit has Source::select_best_match which claims part of it, but which takes no text or code point and only returns a single font.
Skribo’s input at the moment is a FontCollection which contains FontFamily’s which contains fonts. In CSS the input has an ordered list of family names (each a string, or one of the 5 generic categories). There is also special handing of @font-face fonts v.s. system fonts.
For the use case of implementing CSS, how much of this should be implemented in Skribo v.s. font-kit? How much should be left to the user?
CC @raphlinus, @pcwalton

This definitely needs more thought. Since we have things spread across multiple crates, there are serious questions about what logic should be where.
I unfortunately am too overloaded right now to dig into this, but I can imagine a few approaches. One is to build the entirety of the CSS font matching algorithm into skribo, relying on font-kit for raw data. Another is to figure out an API surface (possibly with extension points such as a match score callback) where skribo can provide enough capability to implement CSS. For example, I can definitely see skribo applying the cluster matching logic.
Basically, somebody needs to sit down and work out a design for this. Unfortunately it won't be me :/ But I'm open to guiding skribo to adapt to what's decided.
I did look at this in the context of DirectWrite; it's pretty clear that DWrite has lots of features to facilitate web work (such as async loading of fonts), but I don't think it quite does everything needed by CSS. In particular, I see "unicode-range" as a fly in the ointment, though I believe it's being de-emphasized for new work.

I'm not sure off the top of my head what the best place for this is. I think Source::select_best_match should be involved somehow, as I set it up to be useful for this purpose; it sounds like we should change font-kit's API to better match what CSS needs.


Serialized folder became "Access Denied"

I don't know if the following behaviour is a bug or a wrong configuration.
We are using Sitecore 82u3 and unicorn 3.3.2
Some times after the serialization process some Feuture or Foundation folder (C:\Progetti\Sitecore<projectname>\Sitecore.\src\Feature\Navigation\serialization\Feature.Navigation.Media) became unaccessible with "Access Denied" message.
The only way to unlock is to use a Linux USB instance and rename/delete the folder.
There is some one experienced in this behaviour.
Dante

This can occur if a folder is placed into a 'pending delete' state by the
file system. Usually killing w3wp.exe will resolve it. I've never seen
needing to use a Linux USB to fix it however, or outside of a
reserialization.
Rainbow 1.4.2 and later greatly mitigate the chances of this happening.
On Mon, Jul 17, 2017 at 1:18 AM danteciambella<EMAIL_ADDRESS>wrote:

I don't know if the following behaviour is a bug or a wrong configuration.
We are using Sitecore 82u3 and unicorn 3.3.2
Some times after the serialization process some Feuture or Foundation
folder
(C:\Progetti\Sitecore\Sitecore.\src\Feature\Navigation\serialization\Feature.Navigation.Media)
folder became unaccessible with "Access Denied" message.
The only way to unlock is to use a Linux USB instance and rename/delete
the folder.
There is some one expeerieced in this behaviour.
Dante
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/kamsar/Unicorn/issues/239, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGU_e0XQKmvcO2LzQPtqVLY2QuVqbEsks5sOxjqgaJpZM4OZqkf
.

Ok Kamsar thank you very much for your answer.
Next time I'll try stopping w3wp.exe, I'll keep you updated.
Dante


add github ISSUE_TEMPLATE files

What is the change?

add bug_report.md to bug-report.yml to enable Github's form based issue template

https://youtu.be/qQE1BUkf2-s?t=23

add config.yml file to help direct users to the helpful pages

Motivation

encourage's bug reporter's to put more care into their bug report before submission
this may help maintainer's receive more detailed & higher quality bug report's
adds helpful tips for user's during the process of creating a bug/issue report

Demo of Change

this PR is similar to this one I created for remix-auth repo: https://github.com/sergiodxa/remix-auth/pull/87

Thanks for adding this helpful template!


Unable to use FeatureColumn with Keras Functional API

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Y
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): Source
TensorFlow version (use command below):  2.0.0-alpha0
Python version: 3.7
Bazel version (if compiling from source): N/A
GCC/Compiler version (if compiling from source): N/A
CUDA/cuDNN version: N/A
GPU model and memory: N/A

Describe the current behavior
I followed the guide: Classify structured data. The author used TF Feature Column and TF Data alongside a Sequential model, which worked out just fine.
Then, I've tried implementing the same using Keras Functional API, but was greeted by the error message below:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-19-9647c70de900> in <module>
----> 1 inputs = layers.Input(tensor=feature_layer, name='features')
      2 x = layers.Dense(128, activation='relu')(inputs)
      3 x = layers.Dense(64, activation='relu')(x)
      4
      5 baggage_pred = layers.Dense(1, activation='sigmoid', name='baggage')(x)

~/.local/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_layer.py in Input(shape, batch_size, name, dtype, sparse, tensor, **kwargs)
    231       dtype=dtype,
    232       sparse=sparse,
--> 233       input_tensor=tensor)
    234   # Return tensor including `_keras_history`.
    235   # Note that in this case train_output and test_output are the same pointer.

~/.local/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_layer.py in __init__(self, input_shape, batch_size, dtype, input_tensor, sparse, name, **kwargs)
     77         dtype = backend.floatx()
     78       else:
---> 79         dtype = backend.dtype(input_tensor)
     80     elif input_tensor is not None and input_tensor.dtype != dtype:
     81       raise ValueError('`input_tensor.dtype` differs from `dtype`: %s vs. %s' %

~/.local/lib/python3.7/site-packages/tensorflow/python/keras/backend.py in dtype(x)
   1025   ```
   1026   """
-> 1027   return x.dtype.base_dtype.name
   1028
   1029

AttributeError: 'str' object has no attribute 'base_dtype'

Describe the expected behavior
If I understood correctly, TF Keras is supposed to be interoperable with Feature Column. And the way to achieve that is to wrap a list of feature columns with tf.keras.layers.DenseFeatures() and parse it to the input layer as a tensor like so:
feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
inputs = layers.Input(tensor=feature_layer, name='features')
Code to reproduce the issue
Here's the code to reproduce the error:
from __future__ import absolute_import, division, print_function

import numpy as np
import pandas as pd

#!pip install tensorflow==2.0.0-alpha0
import tensorflow as tf

from tensorflow import feature_column
from tensorflow import keras
from tensorflow.keras import layers
from sklearn.model_selection import train_test_split

URL = 'https://storage.googleapis.com/applied-dl/heart.csv'
dataframe = pd.read_csv(URL)
dataframe.head()

train, test = train_test_split(dataframe, test_size=0.2)
train, val = train_test_split(train, test_size=0.2)
print(len(train), 'train examples')
print(len(val), 'validation examples')
print(len(test), 'test examples')

# A utility method to create a tf.data dataset from a Pandas Dataframe
def df_to_dataset(dataframe, shuffle=True, batch_size=32):
  dataframe = dataframe.copy()
  labels = dataframe.pop('target')
  ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
  if shuffle:
    ds = ds.shuffle(buffer_size=len(dataframe))
  ds = ds.batch(batch_size)
  return ds

batch_size = 5 # A small batch sized is used for demonstration purposes
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

age = feature_column.numeric_column("age")
age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])
thal = feature_column.categorical_column_with_vocabulary_list(
      'thal', ['fixed', 'normal', 'reversible'])
thal_one_hot = feature_column.indicator_column(thal)
thal_embedding = feature_column.embedding_column(thal, dimension=8)
thal_hashed = feature_column.categorical_column_with_hash_bucket(
      'thal', hash_bucket_size=1000)
crossed_feature = feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000)

feature_columns = []

# numeric cols
for header in ['age', 'trestbps', 'chol', 'thalach', 'oldpeak', 'slope', 'ca']:
  feature_columns.append(feature_column.numeric_column(header))

# bucketized cols
age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])
feature_columns.append(age_buckets)

# indicator cols
thal = feature_column.categorical_column_with_vocabulary_list(
      'thal', ['fixed', 'normal', 'reversible'])
thal_one_hot = feature_column.indicator_column(thal)
feature_columns.append(thal_one_hot)

# embedding cols
thal_embedding = feature_column.embedding_column(thal, dimension=8)
feature_columns.append(thal_embedding)

# crossed cols
crossed_feature = feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000)
crossed_feature = feature_column.indicator_column(crossed_feature)
feature_columns.append(crossed_feature)

batch_size = 32
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

feature_layer = tf.keras.layers.DenseFeatures(feature_columns)

inputs = layers.Input(tensor=feature_layer, name='features')
x = layers.Dense(128, activation='relu')(inputs)
x = layers.Dense(64, activation='relu')(x)

baggage_pred = layers.Dense(1, activation='sigmoid')(x)

model = keras.Model(inputs=inputs,outputs=baggage_pred)

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

Other info / logs
N/A

i've been struggling with this as well.
I found 2 workarounds but hope there is easier way..
First method is you subclass tf.keras.models.Model  there, you have inputs parameter in def call method, and you feed it to DenseFeatures
But subclassing keras model has shortcommings in that it can't do model.save()
Second method is you define input tensors for all your feature (Input(name="feature_name")) when you use functional api to build keras model. and build feature_inputs = {"feature_name" : input_tensor} dictionary, and feed the dictionary to tf.keras.layers.DenseFeatures(feature_columns)(feature_inputs) .
So you have to give input tensors to feature_layer ...
It would be much easier if one could create input tensors from feature_columns

@littlehome-eugene : I've referred to the documentation of DenseFeatures and came across the example:
price = numeric_column('price')
keywords_embedded = embedding_column(
    categorical_column_with_hash_bucket("keywords", 10K), dimensions=16)
columns = [price, keywords_embedded, ...]
feature_layer = DenseFeatures(columns)

features = tf.parse_example(..., features=make_parse_example_spec(columns))
dense_tensor = feature_layer(features)
for units in [128, 64, 32]:
  dense_tensor = tf.layers.dense(dense_tensor, units, tf.nn.relu)
prediction = tf.layers.dense(dense_tensor, 1).

It seems features, in this regard, refers to the input tensor. It is fed to the feature layer in dense_tensor = feature_layer(features), and finally to the model dense_tensor = tf.layers.dense(dense_tensor, units, tf.nn.relu).
I have tried the same by defining:
...
feature_layer = tf.keras.layers.DenseFeatures(get_cols())
# get_cols() returns a list of feature columns

features = tf.io.parse_example(features=tf.feature_column.make_parse_example_spec(get_cols()))
dense_tensor = feature_layer(features)

inputs = keras.layers.Input(tensor=dense_tensor, name='features')
x = layers.Dense(128, activation='relu')(inputs)
x = layers.Dense(64, activation='relu')(x)
...

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-12-60a3dbfaf487> in <module>
      1 feature_layer = tf.keras.layers.DenseFeatures(get_cols())
      2
----> 3 features = tf.io.parse_example(features=tf.feature_column.make_parse_example_spec(get_cols()))
      4 dense_tensor = feature_layer(features)
      5

TypeError: parse_example_v2() missing 1 required positional argument: 'serialized'

I'm having trouble defining serialized in tf.io.parse_example. Any pointers?

oh, I was just saying I'm trying to solve the same problem, I do have partial solutions as I outlined.
So you could try what I did before tensorflow has the definitive answer to this.
With my limited knowledge of tensorflow (have worked with only a few weeks),
but I don't think parse_example would help here, the serialized arguments is the actual data like training data, so you can't use it to define your functional api-based model.

@littlehome-eugene: No issue. :) Okay, so parse_example was only meant for demo purposes.
Digging deeper:
features = tf.feature_column.make_parse_example_spec(get_cols())
print(features)
returns me a schema/placeholder:
{'aaa': VarLenFeature(dtype=tf.string),
 'bbb': VarLenFeature(dtype=tf.string),
 'count': VarLenFeature(dtype=tf.int64),
...
 'prob_aaa': FixedLenFeature(shape=(1,), dtype=tf.float32, default_value=None)}

so was thinking if it is possible to congregate these placeholders and use it as the input to Input(shape=...)...

I think I found another way..
it's hacky.. but you can try
I extracted symbolic tensor ( I think it's tensor definition not associated with a data)
and gave them to keras.Input() as a parameter
here's a sample

 import copy
 import tensorflow as tf
 import pandas as pd
 from tensorflow.keras.layers import Input, Embedding, concatenate, Dense, Flatten
 from tensorflow import feature_column
 from tensorflow.python.keras.engine import training_utils

 # tf.compat.v1.disable_eager_execution()

 def df_to_dataset(dataframe, shuffle=True, batch_size=32):
   dataframe = dataframe.copy()
   labels = dataframe.pop('target')
   d = dict(dataframe)

   ds = tf.data.Dataset.from_tensor_slices((d, labels))
   if shuffle:
     ds = ds.shuffle(buffer_size=len(dataframe))
   ds = ds.batch(batch_size)
   return ds

 # import pdb; pdb.set_trace()
 # feature_layer = tf.keras.layers.DenseFeatures(feature_columns)

 class FeatureModel(tf.keras.Model):

   def __init__(self, *args, **kwargs):
     feature_columns = kwargs.pop('feature_columns', None)
     super().__init__(*args, **kwargs)

     self.feature_columns = feature_columns
     self.feature_layer = tf.keras.layers.DenseFeatures(self.feature_columns)
     self.dense1 = Dense(1, activation='sigmoid')

   def call(self, inputs, training=False):
       a_inputs = {k:v for k, v in inputs.items() if k in ['child_month_young', 'child_gender_young']}
       model_inputs = training_utils.ModelInputs(a_inputs)
       feature_inputs = model_inputs.get_symbolic_inputs()

       self.feature_inputs = feature_inputs

       features = self.feature_layer(inputs)

       o = self.dense1(features)
       return o

 feature_columns = []
 child_month_young = feature_column.numeric_column("child_month_young")
 kid_age_youngest_buckets = feature_column.bucketized_column(child_month_young, boundaries=[0, 12, 24, 36, 72, 96])
 feature_columns.append(kid_age_youngest_buckets)

 child_gender_young = feature_column.categorical_column_with_vocabulary_list(
   'child_gender_young', ['boy', 'girl', ''])
 child_gender_young_one_hot = feature_column.indicator_column(child_gender_young)
 feature_columns.append(child_gender_young_one_hot)

 class FModel(object):
     def get_model(self, feature_inputs):
         first_input = Input(shape = (1,), name='first_input')
         second_input = Input(shape = (1,), name='second_input' )

         feature_inputs_d = {}
         for name, tensor in feature_inputs.items():
             input_t = Input(shape=tensor.shape, name=name, tensor=tensor)
             setattr(self, name, input_t)

             feature_inputs_d[name] = input_t

         embedding_layer = Embedding(input_dim=10, output_dim=3, input_length=1)

         first_input_encoded = embedding_layer(first_input)
         first_input_encoded = tf.keras.layers.Reshape((3,))(first_input_encoded)
         second_input_encoded = embedding_layer(second_input)
         second_input_encoded = tf.keras.layers.Reshape((3,))(second_input_encoded)

         feature_layer = tf.keras.layers.DenseFeatures(feature_columns)

         features = feature_layer(feature_inputs_d)
         o = concatenate([first_input_encoded, second_input_encoded, features])
         o = Dense(1)(o)

         inputs = [first_input, second_input] + list(feature_inputs_d.values())
         model = tf.keras.models.Model(inputs=inputs, outputs=o)
         return model

 df = pd.DataFrame(
     [[3,4,5, 'boy', 0],
     [2,6,7, 'girl', 1]], columns=['first_input', 'second_input', 'child_month_young', 'child_gender_young', 'target'])

 def get_feature_inputs(train_ds):

     feature_model = FeatureModel(feature_columns=feature_columns)
     feature_model.compile(optimizer='adam',
                           loss='binary_crossentropy',
                           metrics=['accuracy'])

     feature_model.fit(train_ds, epochs=1)

     return dict(feature_model.feature_inputs)

 train_ds = df_to_dataset(df)
 val_ds = df_to_dataset(df)

 # feature_inputs, _, _ = training_utils.extract_tensors_from_dataset(train_ds)
 # feature_inputs, _, _ = training_utils.unpack_iterator_input(train_ds)

 feature_inputs = get_feature_inputs(train_ds)
 f = FModel()
 model = f.get_model(feature_inputs)

 model.compile(optimizer='adam',
               loss='binary_crossentropy',
               metrics=['accuracy'])

 model.fit(
     train_ds,
     validation_data=val_ds,
     epochs=1
 )

DenseFeatures.dtype is None, so it can't be passed into Input in existing code.
nice workaround @littlehome-eugene !
         feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
         features = feature_layer(feature_inputs_d)

any syntax sugar to create map[string]Input feature_inputs_d per column or from feature_column directly?
then it doesn't need to fit FeatureModel in this way

I am having the same issue. Is there any plan to make feature columns work with Keras functional API?

Hello. Has there been any progress on this issue? I wanted to use the Feature Columns with the Functional API as well. Is there a good workaround to this issue?

I also wanted to use the Feature Columns with the Functional API. Waiting for a good workaround to this issue.

Apologies for the delay. Will take a look today.
On Thu, Jun 13, 2019 at 1:18 AM admu<EMAIL_ADDRESS>wrote:

I also wanted to use the Feature Columns with the Functional API. Waiting
for a good workaround to this issue.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/27416?email_source=notifications&email_token=AKEVL2AFAV4GS6SNAN2E5R3P2H7EJA5CNFSM4HC55GZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXS4PTI#issuecomment-501598157,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKEVL2HA25XU7SY6EYU2QETP2H7EJANCNFSM4HC55GZA
.

How to build a wide-and-deep model using Keras in TensorFlow 2.0

I am also trying to make feature column work with Keras function API. waiting for a reasonable solution.

As unfortunate as this is, for now you will have to make it work by assigning tf.keras.Input to each original feature column that you have, i.e., those numeric feature column and categorical feature columns. Here's the code snippet that will make it work:
from __future__ import absolute_import, division, print_function

import numpy as np
import pandas as pd

#!pip install tensorflow==2.0.0-alpha0
import tensorflow as tf

from tensorflow import feature_column
from tensorflow import keras
from tensorflow.keras import layers
from sklearn.model_selection import train_test_split

URL = 'https://storage.googleapis.com/applied-dl/heart.csv'
dataframe = pd.read_csv(URL)
dataframe.head()

train, test = train_test_split(dataframe, test_size=0.2)
train, val = train_test_split(train, test_size=0.2)
print(len(train), 'train examples')
print(len(val), 'validation examples')
print(len(test), 'test examples')

# A utility method to create a tf.data dataset from a Pandas Dataframe
def df_to_dataset(dataframe, shuffle=True, batch_size=32):
  dataframe = dataframe.copy()
  labels = dataframe.pop('target')
  ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
  if shuffle:
    ds = ds.shuffle(buffer_size=len(dataframe))
  ds = ds.batch(batch_size)
  return ds

batch_size = 5 # A small batch sized is used for demonstration purposes
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

age = feature_column.numeric_column("age")

feature_columns = []
feature_layer_inputs = {}

# numeric cols
for header in ['age', 'trestbps', 'chol', 'thalach', 'oldpeak', 'slope', 'ca']:
  feature_columns.append(feature_column.numeric_column(header))
  feature_layer_inputs[header] = tf.keras.Input(shape=(1,), name=header)

# bucketized cols
age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])
feature_columns.append(age_buckets)

# indicator cols
thal = feature_column.categorical_column_with_vocabulary_list(
      'thal', ['fixed', 'normal', 'reversible'])
thal_one_hot = feature_column.indicator_column(thal)
feature_columns.append(thal_one_hot)
feature_layer_inputs['thal'] = tf.keras.Input(shape=(1,), name='thal', dtype=tf.string)

# embedding cols
thal_embedding = feature_column.embedding_column(thal, dimension=8)
feature_columns.append(thal_embedding)

# crossed cols
crossed_feature = feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000)
crossed_feature = feature_column.indicator_column(crossed_feature)
feature_columns.append(crossed_feature)

batch_size = 32
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
feature_layer_outputs = feature_layer(feature_layer_inputs)

x = layers.Dense(128, activation='relu')(feature_layer_outputs)
x = layers.Dense(64, activation='relu')(x)

baggage_pred = layers.Dense(1, activation='sigmoid')(x)

model = keras.Model(inputs=[v for v in feature_layer_inputs.values()], outputs=baggage_pred)

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

model.fit(train_ds)

@tanzhenyu
Thank you so much for the code example. Appreciate it!
I noticed that inside of the code, embedding columns and crossed columns are not added to feature_layer_inputs, is that intentional?
if feature_layer_inputs doesn't include the two left-out columns, the respective data won't flow through the network, right?
feature_layer_outputs = feature_layer(feature_layer_inputs)

Right, that's why I mentioned it needs to be "originating columns", i.e. those which is actually your input that you will feed in. embedding is derived from you categorical column, and your input (from your data pipeline or dataframe in this case) doesn't contain that, so no need to create input for it.

@tanzhenyu
Thank you so much for the code example. Appreciate it!
I noticed that inside of the code, embedding columns and crossed columns are not added to feature_layer_inputs, is that intentional?
if feature_layer_inputs doesn't include the two left-out columns, the respective data won't flow through the network, right?
feature_layer_outputs = feature_layer(feature_layer_inputs)

To add on to @tanzhenyu's response, note the below line:
feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
This handles the feature column transformation required during model definition. Meanwhile, during model compilation:
keras.Model(inputs=[v for v in feature_layer_inputs.values()], outputs=baggage_pred)
feature_layer_inputs passes in the actual input (original features) to the model.
Hope it helps ;)
Thank you @tanzhenyu for the codes! I shall try this out when I have the chance to. Kudos!

I saved and loaded it using:
model.save('./model/structed_data_model.h5')
new_model = keras.models.load_model('./model/structed_data_model.h5')
It prompt error：
ValueError: Unknown layer: DenseFeatures
https://github.com/tensorflow/tensorflow/issues/27008
So,how can i load it correctly

@admu thanks for the heads up. It is going to be a blocking issue if we are not able to save/load models.
Another issue I hit is that after update tensorflow installation to 2.0.0-beta1, the code example from @tanzhenyu is broken with the following errors.
Tensorflow team, could you please comment on this breakage? Thank you.

49 validation examples
61 test examples
Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-5282677307998123760.py", line 326, in
exec(code)
File "", line 44, in
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/base_layer.py", line 662, in call
outputs = call_fn(inputs, *args, **kwargs)
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/autograph/impl/api.py", line 166, in wrapper
), args, kwargs)
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/autograph/impl/api.py", line 340, in converted_call
if inspect_utils.isbuiltin(f):
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/autograph/pyct/inspect_utils.py", line 84, in isbuiltin
if f in six.moves.builtins.dict.values():
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/pandas/core/generic.py", line 1478, in nonzero
.format(self.class.name))
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-5282677307998123760.py", line 333, in
raise Exception(traceback.format_exc())
Exception: Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-5282677307998123760.py", line 326, in
exec(code)
File "", line 44, in
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/base_layer.py", line 662, in call
outputs = call_fn(inputs, *args, **kwargs)
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/autograph/impl/api.py", line 166, in wrapper
), args, kwargs)
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/autograph/impl/api.py", line 340, in converted_call
if inspect_utils.isbuiltin(f):
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/autograph/pyct/inspect_utils.py", line 84, in isbuiltin
if f in six.moves.builtins.dict.values():
File "/usr/lib/envs/env-1923-ver-2635-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/pandas/core/generic.py", line 1478, in nonzero
.format(self.class.name))
ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Since this issue is closed. I opened a new one for discussing the compatibility issue: https://github.com/tensorflow/tensorflow/issues/29923

Given the model trained this way, did anyone successfully feed data into model.predict()? Could you share the code snippet as well.
The following code snippet doesn't seem to work:
def df_to_dataset_for_inference(dataframe, batch_size=32):
  dataframe = dataframe.copy()
  ds = tf.data.Dataset.from_tensor_slices(dict(dataframe))
  ds = ds.batch(batch_size)
  return ds

model.predict(df_to_dataset_for_inference(test))

Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-669659483191650312.py", line 331, in
exec(code)
File "", line 6, in
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 1143, in predict
callbacks=callbacks)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training_generator.py", line 257, in model_iteration
batch_outs = batch_function(*batch_data)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training_generator.py", line 531, in predict_on_batch
return model.predict_on_batch(x)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 1368, in predict_on_batch
ops.convert_to_tensor(val, dtype=K.floatx()) for val in inputs]
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 1368, in
ops.convert_to_tensor(val, dtype=K.floatx()) for val in inputs]
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1050, in convert_to_tensor
return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1108, in convert_to_tensor_v2
as_ref=False)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1128, in internal_convert_to_tensor
value = _TensorTensorConversionFunction(value, dtype=dtype)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 984, in _TensorTensorConversionFunction
(dtype.name, t.dtype.name, str(t)))
ValueError: Tensor conversion requested dtype float32 for Tensor with dtype int32: 'tf.Tensor(\n[[2]\n [2]\n [3]\n [1]\n [1]\n [2]\n [1]\n [1]\n [2]\n [2]\n [1]\n [2]\n [2]\n [1]\n [1]\n [2]\n [1]\n [2]\n [1]\n [2]\n [1]\n [2]\n [2]\n [1]\n [2]\n [1]\n [1]\n [1]\n [2]\n [2]\n [3]\n [1]], shape=(32, 1), dtype=int32)'

As unfortunate as this is, for now you will have to make it work by assigning tf.keras.Input to each original feature column that you have, i.e., those numeric feature column and categorical feature columns. Here's the code snippet that will make it work:
from __future__ import absolute_import, division, print_function

import numpy as np
import pandas as pd

#!pip install tensorflow==2.0.0-alpha0
import tensorflow as tf

from tensorflow import feature_column
from tensorflow import keras
from tensorflow.keras import layers
from sklearn.model_selection import train_test_split

URL = 'https://storage.googleapis.com/applied-dl/heart.csv'
dataframe = pd.read_csv(URL)
dataframe.head()

train, test = train_test_split(dataframe, test_size=0.2)
train, val = train_test_split(train, test_size=0.2)
print(len(train), 'train examples')
print(len(val), 'validation examples')
print(len(test), 'test examples')

# A utility method to create a tf.data dataset from a Pandas Dataframe
def df_to_dataset(dataframe, shuffle=True, batch_size=32):
  dataframe = dataframe.copy()
  labels = dataframe.pop('target')
  ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
  if shuffle:
    ds = ds.shuffle(buffer_size=len(dataframe))
  ds = ds.batch(batch_size)
  return ds

batch_size = 5 # A small batch sized is used for demonstration purposes
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

age = feature_column.numeric_column("age")

feature_columns = []
feature_layer_inputs = {}

# numeric cols
for header in ['age', 'trestbps', 'chol', 'thalach', 'oldpeak', 'slope', 'ca']:
  feature_columns.append(feature_column.numeric_column(header))
  feature_layer_inputs[header] = tf.keras.Input(shape=(1,), name=header)

# bucketized cols
age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])
feature_columns.append(age_buckets)

# indicator cols
thal = feature_column.categorical_column_with_vocabulary_list(
      'thal', ['fixed', 'normal', 'reversible'])
thal_one_hot = feature_column.indicator_column(thal)
feature_columns.append(thal_one_hot)
feature_layer_inputs['thal'] = tf.keras.Input(shape=(1,), name='thal', dtype=tf.string)

# embedding cols
thal_embedding = feature_column.embedding_column(thal, dimension=8)
feature_columns.append(thal_embedding)

# crossed cols
crossed_feature = feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000)
crossed_feature = feature_column.indicator_column(crossed_feature)
feature_columns.append(crossed_feature)

batch_size = 32
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
feature_layer_outputs = feature_layer(feature_layer_inputs)

x = layers.Dense(128, activation='relu')(feature_layer_outputs)
x = layers.Dense(64, activation='relu')(x)

baggage_pred = layers.Dense(1, activation='sigmoid')(x)

model = keras.Model(inputs=[v for v in feature_layer_inputs.values()], outputs=baggage_pred)

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

model.fit(train_ds)

@tanzhenyu
The solution works in TF2.0 beta, except it has two major problems:

You cannot save the model. Adding model.save('wide_n_deep') to the end of the code fails and gives you this error:

    TypeError: An op outside of the function building code is being passed
    a "Graph" tensor. It is possible to have Graph tensors
    leak out of the function building context by including a
    tf.init_scope in your function building code.
    For example, the following function will fail:
      @tf.function
      def has_init_scope():
        my_constant = tf.constant(1.)
        with tf.init_scope():
          added = my_constant * 2
    The graph tensor has name: model/dense_features/age_bucketized_X_thal_indicator/thal_lookup/Const:0

You cannot run it in non-eager mode: tf.compat.v1.disable_eager_execution(). It gives this error:

2019-06-24 19:02:02.217642: W tensorflow/core/framework/op_kernel.cc:1546] OP_REQUIRES failed at lookup_table_op.cc:809 : Failed precondition: Table not initialized.
Traceback (most recent call last):
  File "fc_test.py", line 91, in <module>
    model.fit(train_ds)
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 643, in fit
    use_multiprocessing=use_multiprocessing)
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 664, in fit
    steps_name='steps_per_epoch')
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 294, in model_iteration
    batch_outs = f(actual_inputs)
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/keras/backend.py", line 3353, in __call__
    run_metadata=self.run_metadata)
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1458, in __call__
    run_metadata_ptr)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Table not initialized.
	 [[{{node dense_features/age_bucketized_X_thal_indicator/hash_table_Lookup/LookupTableFindV2}}]]

Do you have recommendations on how to fix these?

Given the model trained this way, did anyone successfully feed data into model.predict()? Could you share the code snippet as well.
The following code snippet doesn't seem to work:
def df_to_dataset_for_inference(dataframe, batch_size=32):
  dataframe = dataframe.copy()
  ds = tf.data.Dataset.from_tensor_slices(dict(dataframe))
  ds = ds.batch(batch_size)
  return ds

model.predict(df_to_dataset_for_inference(test))

Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-669659483191650312.py", line 331, in
exec(code)
File "", line 6, in
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 1143, in predict
callbacks=callbacks)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training_generator.py", line 257, in model_iteration
batch_outs = batch_function(*batch_data)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training_generator.py", line 531, in predict_on_batch
return model.predict_on_batch(x)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 1368, in predict_on_batch
ops.convert_to_tensor(val, dtype=K.floatx()) for val in inputs]
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 1368, in
ops.convert_to_tensor(val, dtype=K.floatx()) for val in inputs]
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1050, in convert_to_tensor
return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1108, in convert_to_tensor_v2
as_ref=False)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1128, in internal_convert_to_tensor
value = _TensorTensorConversionFunction(value, dtype=dtype)
File "/usr/lib/envs/env-2007-ver-2424-a-4.2.9-py-3.5.3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 984, in _TensorTensorConversionFunction
(dtype.name, t.dtype.name, str(t)))
ValueError: Tensor conversion requested dtype float32 for Tensor with dtype int32: 'tf.Tensor(\n[[2]\n [2]\n [3]\n [1]\n [1]\n [2]\n [1]\n [1]\n [2]\n [2]\n [1]\n [2]\n [2]\n [1]\n [1]\n [2]\n [1]\n [2]\n [1]\n [2]\n [1]\n [2]\n [2]\n [1]\n [2]\n [1]\n [1]\n [1]\n [2]\n [2]\n [3]\n [1]], shape=(32, 1), dtype=int32)'

This should get fixed, can you try install tf 2.0 nightly?

As unfortunate as this is, for now you will have to make it work by assigning tf.keras.Input to each original feature column that you have, i.e., those numeric feature column and categorical feature columns. Here's the code snippet that will make it work:
from __future__ import absolute_import, division, print_function

import numpy as np
import pandas as pd

#!pip install tensorflow==2.0.0-alpha0
import tensorflow as tf

from tensorflow import feature_column
from tensorflow import keras
from tensorflow.keras import layers
from sklearn.model_selection import train_test_split

URL = 'https://storage.googleapis.com/applied-dl/heart.csv'
dataframe = pd.read_csv(URL)
dataframe.head()

train, test = train_test_split(dataframe, test_size=0.2)
train, val = train_test_split(train, test_size=0.2)
print(len(train), 'train examples')
print(len(val), 'validation examples')
print(len(test), 'test examples')

# A utility method to create a tf.data dataset from a Pandas Dataframe
def df_to_dataset(dataframe, shuffle=True, batch_size=32):
  dataframe = dataframe.copy()
  labels = dataframe.pop('target')
  ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
  if shuffle:
    ds = ds.shuffle(buffer_size=len(dataframe))
  ds = ds.batch(batch_size)
  return ds

batch_size = 5 # A small batch sized is used for demonstration purposes
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

age = feature_column.numeric_column("age")

feature_columns = []
feature_layer_inputs = {}

# numeric cols
for header in ['age', 'trestbps', 'chol', 'thalach', 'oldpeak', 'slope', 'ca']:
  feature_columns.append(feature_column.numeric_column(header))
  feature_layer_inputs[header] = tf.keras.Input(shape=(1,), name=header)

# bucketized cols
age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])
feature_columns.append(age_buckets)

# indicator cols
thal = feature_column.categorical_column_with_vocabulary_list(
      'thal', ['fixed', 'normal', 'reversible'])
thal_one_hot = feature_column.indicator_column(thal)
feature_columns.append(thal_one_hot)
feature_layer_inputs['thal'] = tf.keras.Input(shape=(1,), name='thal', dtype=tf.string)

# embedding cols
thal_embedding = feature_column.embedding_column(thal, dimension=8)
feature_columns.append(thal_embedding)

# crossed cols
crossed_feature = feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000)
crossed_feature = feature_column.indicator_column(crossed_feature)
feature_columns.append(crossed_feature)

batch_size = 32
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
feature_layer_outputs = feature_layer(feature_layer_inputs)

x = layers.Dense(128, activation='relu')(feature_layer_outputs)
x = layers.Dense(64, activation='relu')(x)

baggage_pred = layers.Dense(1, activation='sigmoid')(x)

model = keras.Model(inputs=[v for v in feature_layer_inputs.values()], outputs=baggage_pred)

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

model.fit(train_ds)

@tanzhenyu
The solution works in TF2.0 beta, except it has two major problems:

You cannot save the model. Adding model.save('wide_n_deep') to the end of the code fails and gives you this error:

    TypeError: An op outside of the function building code is being passed
    a "Graph" tensor. It is possible to have Graph tensors
    leak out of the function building context by including a
    tf.init_scope in your function building code.
    For example, the following function will fail:
      @tf.function
      def has_init_scope():
        my_constant = tf.constant(1.)
        with tf.init_scope():
          added = my_constant * 2
    The graph tensor has name: model/dense_features/age_bucketized_X_thal_indicator/thal_lookup/Const:0

You cannot run it in non-eager mode: tf.compat.v1.disable_eager_execution(). It gives this error:

2019-06-24 19:02:02.217642: W tensorflow/core/framework/op_kernel.cc:1546] OP_REQUIRES failed at lookup_table_op.cc:809 : Failed precondition: Table not initialized.
Traceback (most recent call last):
  File "fc_test.py", line 91, in <module>
    model.fit(train_ds)
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 643, in fit
    use_multiprocessing=use_multiprocessing)
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 664, in fit
    steps_name='steps_per_epoch')
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 294, in model_iteration
    batch_outs = f(actual_inputs)
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/keras/backend.py", line 3353, in __call__
    run_metadata=self.run_metadata)
  File "/temp/v36_test/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1458, in __call__
    run_metadata_ptr)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Table not initialized.
	 [[{{node dense_features/age_bucketized_X_thal_indicator/hash_table_Lookup/LookupTableFindV2}}]]

Do you have recommendations on how to fix these?

The first one is very likely a bug, can you create a new github issue for this? Thanks,
The second one requires you to call sess.run(tf.tables_initializer())

I use the following hack too solve the prediction issue:

Use DenseFeatures layer as a feature transformation that takes inputs from feature columns.
Use outputs of DenseFeatures layer as the input to the model.

As a result, model load/save as well as predict API works. The following is the complete example:
import numpy as np
import pandas as pd
import tensorflow as tf
from tensorflow import feature_column
from tensorflow import keras
from tensorflow.data import Dataset
from tensorflow.keras import layers
print("numpy version: ", np.version)
print("pandas version: ", pd.version)
print("Tensorflow version: ", tf.version)
def train_test_split(data: pd.DataFrame, test_fraction: float = 0.2) -> (pd.DataFrame, pd.DataFrame):
assert 0.0 < test_fraction < 1.0
split = int(data.shape[0] * (1.0 - test_fraction))
return data[:split], data[split:]
def load_google_heart_example_data() -> (pd.DataFrame, pd.DataFrame, pd.DataFrame):
""" Loads google heart data and split the data into train/validation/test sets."""
URL = "https://storage.googleapis.com/applied-dl/heart.csv"
dataframe = pd.read_csv(URL)
train, test = train_test_split(dataframe, test_fraction=0.1)
train, val = train_test_split(train, test_fraction=0.1)
return train, val, test

def df_to_dataset(features: np.ndarray, labels: np.ndarray, shuffle=True, batch_size=32) -> Dataset:
ds = Dataset.from_tensor_slices(({"feature": features}, {"target": labels}))
if shuffle:
ds = ds.shuffle(buffer_size=len(features))
ds = ds.batch(batch_size)
return ds
def get_feature_transform() -> layers.DenseFeatures:
""" builds a DenseFeatures layer as feature transformation.
The function handles all feature transformation such as bucketizing, vectorizing (one-hot encoding), etc.
"""
feature_columns = []

# Numeric columns.
for feature in ["trestbps", "chol", "thalach", "oldpeak", "slope", "ca"]:
    feature_columns.append(feature_column.numeric_column(feature))

# Bucketized column.
age = feature_column.numeric_column("age")
age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])
feature_columns.append(age_buckets)

# Indicator columns.
thal = feature_column.categorical_column_with_vocabulary_list(
    "thal", ["fixed", "normal", "reversible"])
thal_one_hot = feature_column.indicator_column(thal)
feature_columns.append(thal_one_hot)

return layers.DenseFeatures(feature_columns)

def get_model(input_shape: tuple, learning_rate: float = 0.01) -> keras.Model:
""" Constructs a model using various layers and compiles the model with proper optimizer/loss/metrics."""
inputs = keras.Input(shape=input_shape, name="feature")
x = layers.Dense(8, kernel_initializer="normal", activation="relu", name="hidden_layer_1")(inputs)
x = layers.Dropout(0.2, name="dropout_1")(x)
x = layers.Dense(6, kernel_initializer="normal", activation="relu", name="hidden_layer_2")(x)
x = layers.Dropout(0.2, name="dropout_2")(x)
baggage_pred = layers.Dense(1, activation="sigmoid", name="target")(x)

model = keras.Model(inputs=inputs, outputs=baggage_pred, name="keras/tensorflow_example")
model.compile(optimizer=keras.optimizers.Adam(learning_rate=learning_rate),
              loss="binary_crossentropy",
              metrics=["accuracy"])
return model

def main() -> None:
# Loads data from internet.
train, val, test = load_google_heart_example_data()
print("Sample training data:")
print(train.head(10))
# Transforms all features into dense tensors.
feature_transform = get_feature_transform()
train_features = feature_transform(dict(train)).numpy()
val_features = feature_transform(dict(val)).numpy()
test_features = feature_transform(dict(test)).numpy()

# Defines datasets on the input data.
batch_size = 8
train_ds = df_to_dataset(train_features, train["target"].values, batch_size=batch_size)
val_ds = df_to_dataset(val_features, val["target"].values, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test_features, test["target"].values, shuffle=False, batch_size=batch_size)

# Compiles a model, prints the model summary, and saves the model diagram into a png file.
model = get_model(input_shape=(train_features.shape[1],), learning_rate=0.01)
model.summary()
keras.utils.plot_model(model, "keras_model.png", show_shapes=True)

# Trains the model.
history = model.fit(train_ds, epochs=10, validation_data=val_ds)
print("Training history")
print(history.history)

# Evaluates on test data.
test_results = model.evaluate(test_ds)
print("test result")
print(test_results)

# Runs prediction on test data.
predictions = model.predict({"feature": test_features})
print("Predictions on test data:")
print(predictions)

# Saves and reloads model.
model.save("./model.h5")
model_from_saved = keras.models.load_model("./model.h5")
model_from_saved.summary()

# Runs test data through the reloaded model to make sure the results are same.
predictions_from_saved = model_from_saved.predict({"feature": test_features})
np.testing.assert_array_equal(predictions_from_saved, predictions)

if name == "main":
main()

@HubberDev  Your solution works, but it confuses tensors and numpy arrays when transforming features. It is generally fine since TF 2.0 enables Eager mode by default.
However, turning off the Eager mode, TF will complains about this line
train_features = feature_transform(dict(train)).numpy()
because the result of feature_transform is a tensor, but we need it in ndarray form to make a dataset later from it.
Also, saving in hd5 works fine, but if you remove the .h5 extension to try to save the model in the SavedModel format, it won't save it correctly. When you load the SavedModel and try to print model.summary() it will complain:
ValueError: You tried to call `count_params` on feature, but the layer isn't built. You can build it manually via: `feature.build(batch_input_shape)`.

I think in general there is a disconnect between the FeatureColumn API and Keras API. So far all the examples I worked on which uses these two APIs fail in the non-eager mode.

@tanzhenyu

For the saving issue, I am hesitant to create a new issue, and prefer to wait for the status of these issues: #26835 and #28923 which I think are highly correlated.

For running the example in the non-eager mode, you recommended to add sess.run(tf.tables_initializer()) to the code. How do you write that in TF 2.0? I thought we are not supposed to use Sessions in this version.

@mahzoon Since you already did tf.compat.v1.disable_eager_execution() you are already in graph mode so you can also use tf.compat.v1.Session.

Thanks, this solved the table initialization problem in non-eager mode:
with tf.compat.v1.Session() as sess:
    sess.run(tf.compat.v1.initialize_all_tables())
    model.fit(train_ds)

Unable to use FeatureColumn with Keras Functional API«
Have a better solution
？？？

Any news about the saving issue?

What issue is remaining regarding this?

@tanzhenyu  when I try to save this model:
       input = [v for v in feature_layer_inputs.values()]
        x = Dense(2048, activation='relu')(feature_layer)
        x = Dropout(0.5)(x)
        x = Dense(1024, activation='relu')(x)
        x = Dropout(0.5)(x)
        x = Dense(512, activation='relu')(x)
        x = Dropout(0.5)(x)
        x = Dense(256, activation='relu')(x)
        x = Dropout(0.5)(x)
        out = Dense(1, activation='sigmoid')(x)
        self.model = Model(inputs=[input], outputs=out)
        self.model.summary()
        self.model.compile(loss='binary_crossentropy', optimizer=Adam(lr=0.0001), metrics=['binary_accuracy', 'AUC'])

I obtained the following error:
  File "classifier.py", line 165, in <module>
    main()
  File "classifier.py", line 148, in main
    c.save_model(filename, stats)
  File "classifier.py", line 108, in save_model
    self.model.save('{}.model'.format(filename))
  File "/home/sartiano/projects/suspicious-network-event-recognition-2019/models.py", line 65, in save
    self.model.save(file_model)
  File "/home/sartiano/projects/suspicious-network-event-recognition-2019/.env/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 1157, in save
    saving.save_model(self, filepath, overwrite, include_optimizer, save_format)
  File "/home/sartiano/projects/suspicious-network-event-recognition-2019/.env/lib/python3.6/site-packages/tensorflow_core/python/keras/saving/save.py", line 107, in save_model
    saved_model_save.save(model, filepath, overwrite, include_optimizer)
  File "/home/sartiano/projects/suspicious-network-event-recognition-2019/.env/lib/python3.6/site-packages/tensorflow_core/python/keras/saving/saved_model/save.py", line 85, in save
    save_lib.save(model, filepath)
  File "/home/sartiano/projects/suspicious-network-event-recognition-2019/.env/lib/python3.6/site-packages/tensorflow_core/python/saved_model/save.py", line 855, in save
    meta_graph_def, saveable_view, signatures)
  File "/home/sartiano/projects/suspicious-network-event-recognition-2019/.env/lib/python3.6/site-packages/tensorflow_core/python/saved_model/save.py", line 585, in _fill_meta_graph_def
    signatures = _generate_signatures(signature_functions, resource_map)
  File "/home/sartiano/projects/suspicious-network-event-recognition-2019/.env/lib/python3.6/site-packages/tensorflow_core/python/saved_model/save.py", line 459, in _generate_signatures
    function, mapped_inputs, resource_map)
  File "/home/sartiano/projects/suspicious-network-event-recognition-2019/.env/lib/python3.6/site-packages/tensorflow_core/python/saved_model/save.py", line 411, in _call_function_with_mapped_captures
    function.graph.captures, resource_map)
  File "/home/sartiano/projects/suspicious-network-event-recognition-2019/.env/lib/python3.6/site-packages/tensorflow_core/python/saved_model/save.py", line 333, in _map_captures_to_created_tensors
    .format(interior))
AssertionError: Tried to export a function which references untracked object Tensor("StatefulPartitionedCall/args_106:0", shape=(), dtype=resource).TensorFlow objects (e.g. tf.Variable) captured by functions must be tracked by assigning them to an attribute of a tracked object or assigned to an attribute of the main object directly.

That is not related to the original issue. Can you file another github issue for this?

For anyone landing here using tf 2.0 ... DO NOT START with the keras functional API if you are doing anything other than vanilla regression. Subclass API is first class now and much simpler. Start there.
I'm looking for tf.feature_columns examples in that context. Landed here from google search. Post refs for the next person if you have any.

i've been struggling with this as well.
I found 2 workarounds but hope there is easier way..
First method is you subclass tf.keras.models.Model there, you have inputs parameter in def call method, and you feed it to DenseFeatures
But subclassing keras model has shortcommings in that it can't do model.save()
Second method is you define input tensors for all your feature (Input(name="feature_name")) when you use functional api to build keras model. and build feature_inputs = {"feature_name" : input_tensor} dictionary, and feed the dictionary to tf.keras.layers.DenseFeatures(feature_columns)(feature_inputs) .
So you have to give input tensors to feature_layer ... not the other way around
It would be much easier if one could create input tensors from feature_columns

This is quit the only two ways I found....since there has been 6 months since you post this comment, is there any other solution?

@littlebeandog I was able to find a workaround to get past the above error (but I'm still figuring out how to initialize the table in TF2.0) I think this is a bit cleaner, but maybe still suboptimal.
    feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
    features,labels = tf.data.experimental.get_single_element(training_dataset.take(1))

    dense_inputs = feature_layer(features)

The inputs to the keras model would then be dense_inputs.

The workarounds above don't cover the case where the FeatureColumns contain trainable parameters which require the gradients during model training; e.g. embedding columns. In the case of an embedding, gradients are needed to refine the embedding during training, and precomputing via a DenseFeatures layer is not a workable solution for this use.
Are there any solutions which don't require placing the DenseFeatures outside the Model?

@jpgard I believe that the above workarounds actually include the DenseFeatures params in the model despite the funky calling. Running model.summary() on tanzhenyu's workaround gives me the output shown below. Note that the DenseFeatures layers are in the middle and have trainable params as desired.
Model: "model_2"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
age (InputLayer)                [(None, 1)]          0
__________________________________________________________________________________________________
ca (InputLayer)                 [(None, 1)]          0
__________________________________________________________________________________________________
chol (InputLayer)               [(None, 1)]          0
__________________________________________________________________________________________________
oldpeak (InputLayer)            [(None, 1)]          0
__________________________________________________________________________________________________
slope (InputLayer)              [(None, 1)]          0
__________________________________________________________________________________________________
thal (InputLayer)               [(None, 1)]          0
__________________________________________________________________________________________________
thalach (InputLayer)            [(None, 1)]          0
__________________________________________________________________________________________________
trestbps (InputLayer)           [(None, 1)]          0
__________________________________________________________________________________________________
dense_features_2 (DenseFeatures (None, 1029)         24          age[0][0]
                                                                 ca[0][0]
                                                                 chol[0][0]
                                                                 oldpeak[0][0]
                                                                 slope[0][0]
                                                                 thal[0][0]
                                                                 thalach[0][0]
                                                                 trestbps[0][0]
__________________________________________________________________________________________________
dense_6 (Dense)                 (None, 128)          131840      dense_features_2[0][0]
__________________________________________________________________________________________________
dense_7 (Dense)                 (None, 64)           8256        dense_6[0][0]
__________________________________________________________________________________________________
dense_8 (Dense)                 (None, 1)            65          dense_7[0][0]
==================================================================================================
Total params: 140,185
Trainable params: 140,185
Non-trainable params: 0

Ah, I see, thanks @JoshEZiegler

As unfortunate as this is, for now you will have to make it work by assigning tf.keras.Input to each original feature column that you have, i.e., those numeric feature column and categorical feature columns. Here's the code snippet that will make it work:
from __future__ import absolute_import, division, print_function

import numpy as np
import pandas as pd

#!pip install tensorflow==2.0.0-alpha0
import tensorflow as tf

from tensorflow import feature_column
from tensorflow import keras
from tensorflow.keras import layers
from sklearn.model_selection import train_test_split

URL = 'https://storage.googleapis.com/applied-dl/heart.csv'
dataframe = pd.read_csv(URL)
dataframe.head()

train, test = train_test_split(dataframe, test_size=0.2)
train, val = train_test_split(train, test_size=0.2)
print(len(train), 'train examples')
print(len(val), 'validation examples')
print(len(test), 'test examples')

# A utility method to create a tf.data dataset from a Pandas Dataframe
def df_to_dataset(dataframe, shuffle=True, batch_size=32):
  dataframe = dataframe.copy()
  labels = dataframe.pop('target')
  ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels))
  if shuffle:
    ds = ds.shuffle(buffer_size=len(dataframe))
  ds = ds.batch(batch_size)
  return ds

batch_size = 5 # A small batch sized is used for demonstration purposes
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

age = feature_column.numeric_column("age")

feature_columns = []
feature_layer_inputs = {}

# numeric cols
for header in ['age', 'trestbps', 'chol', 'thalach', 'oldpeak', 'slope', 'ca']:
  feature_columns.append(feature_column.numeric_column(header))
  feature_layer_inputs[header] = tf.keras.Input(shape=(1,), name=header)

# bucketized cols
age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])
feature_columns.append(age_buckets)

# indicator cols
thal = feature_column.categorical_column_with_vocabulary_list(
      'thal', ['fixed', 'normal', 'reversible'])
thal_one_hot = feature_column.indicator_column(thal)
feature_columns.append(thal_one_hot)
feature_layer_inputs['thal'] = tf.keras.Input(shape=(1,), name='thal', dtype=tf.string)

# embedding cols
thal_embedding = feature_column.embedding_column(thal, dimension=8)
feature_columns.append(thal_embedding)

# crossed cols
crossed_feature = feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000)
crossed_feature = feature_column.indicator_column(crossed_feature)
feature_columns.append(crossed_feature)

batch_size = 32
train_ds = df_to_dataset(train, batch_size=batch_size)
val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size)
test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size)

feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
feature_layer_outputs = feature_layer(feature_layer_inputs)

x = layers.Dense(128, activation='relu')(feature_layer_outputs)
x = layers.Dense(64, activation='relu')(x)

baggage_pred = layers.Dense(1, activation='sigmoid')(x)

model = keras.Model(inputs=[v for v in feature_layer_inputs.values()], outputs=baggage_pred)

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

model.fit(train_ds)

thank you so much. I also want to ask you a question，When I deploy the model in this way on tf-serving，I find that every request takes hundreds of milliseconds，I don't think it's normal. Do you have any other methods other than your one? For example, use a layer of example to accept all feature columns？

To all:
The interaction between feature columns and Keras is painful. To address this issue, we have proposed Keras preprocessing layers. If you're interested, please put your comment under this RFC

I don't believe the above workaround is effective if using this inside an estimator
`def mlp_fn(features, labels, mode, params):
"""
defines model & optimisation props
"""
feature_layer_inputs = {}
feature_layer_inputs['dense_input'] = tf.keras.Input(shape=(4,), name='dense_input', dtype=tf.string)

feature_columns = [
    tf.feature_column.numeric_column('dense_input')
]

feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
feature_layer_outputs = feature_layer(feature_layer_inputs)

# --- create NN ---

# input
input_layer = tf.keras.layers.DenseFeatures(feature_layer_outputs)

# output
logits = tf.keras.layers.Dense(units=3, activation=tf.nn.relu)(input_layer)
output = tf.keras.layers.Softmax(logits)

model = keras.Model(inputs=[v for v in feature_layer_inputs.values()], outputs=output)
softmax = model(features)
class_id = tf.argmax(softmax, axis=1)
....

In this case the errorOperatorNotAllowedInGraphError: iterating over tf.Tensor is not allowed in Graph execution. Use Eager execution or decorate this function with @tf.function.` is thrown

Any update on this thread?

@sivaharsh
Please review:
https://github.com/keras-team/governance/blob/master/rfcs/20190502-preprocessing-layers.md
and
https://github.com/tensorflow/community/blob/master/rfcs/20191212-keras-categorical-inputs.md
The work product of which would 'replace feature columns and tf.keras.layers.DenseFeatures with proposed layers'.

any update then?


v-leave transition class should be removed in 2.0

None of the example code has used it. And can someone come up with any real-world leaving animation that does not start with the normal/current state of the element? "No real use".

This is different from other deprecated API due to the symmetry and logical completeness - you wouldn't cut off a limb because you never use it.

Ridiculous justification. But you are the boss.

This is a screenshot from one of the qq group about vuejs. And I predict that such questions will come up again and again in the future, because the users commonly expect symmetric api brings symmetric usage.
We wouldn't decide to cut off a limb not just because we hardly use it, but we would cut it off if we don't use it and it is causing trouble, like the cecum.

i meet the same question.  first i think it will work as same as v-enter . but it does't。。


Feature request : Expose more JobEvents to the Notification Observers

Currently in JobNotificationObserver, only 3 JobEvent types get exposed. They are :-

JobRemoved
JobDisabled
JobRetriesExhausted

Can we have different levels for the notification observer? For instance :-
Level 1 - JobRemoved,  JobDisabled and  JobRetriesExhausted can be the default
Level 2 - Level 1 events + JobFailed + JobSkipped + JobExpired (I would argue that JobSkipped          should be in Level 1)
Level 3 - Level 2 events + JobStarted + JobFinished + JobQueued

Thanks @vineethvarghese for the proposal. Exposing more job events seems a reasonable requirement. We can make this configurable. Then users can configure which events they are interested in. Moreover, we can step further to make it possible to specify the interested job events at job level.

This is highly needed to avoid notification overload. For example, I'm only interested in being notified about failures.


[api][workers][CI/CD][SIMS #1442]Allow SIMS-API to call Camunda

PR Goals

Enable an assessment to be executed from E2E, mocking data as necessary;
Enable messages to be sent to the workflow for unblocking waiting tasks (Exceptions, PIR, CRA, and Supporting Users);
Move methods from API to Workers as needed;
Replace SIMS-API calls to Camunda API by Zeebe call to Camunda 8;

Enabled Zeebe client to be used on SIMS-API;
Associated Zeebe env variables to SIMS-API POD
Created ZeebeModule to make ZeebeClient available and injectable across all applications.

New Camunda 8 workflow current state.

PIR is now waiting for a message to be sent even for the declined status (before the workflow was canceled). The workflow will be ended once the declined is received.
The below part of the workflow is using some fake data, but the other workers are fully functional.

The generated workflow instance id is not a GUID out-of-box so the DB column was changed from a UUID to a VARCHAR(50) that would accommodate both types of instance ids.

New shared libraries

Services (@sims/services)
Utilities (@sims/utilities)

This caused many changes due to reference changes and files being moved.
Only services needed by workers were moved.
MSFAA service was split. The methods needed on API were kept on the API and the methods needed for the workers were moved.

Worker application changes

Changed the way the workers app is initialized to allow the injected ZeebeClient to be consumed.
Created a wrapper to execute the worker to allow better workers execution logging. The below logs are the result of this new wrapper that can be extended as needed.

The constants that represent workflow variables are now organized by the source workflow.

These variables are also used to create the JobInDTO/JobOutDTO, as shown below.

export interface ApplicationUpdateStatusJobInDTO {
  [APPLICATION_ID]: number;
}

export interface ApplicationExceptionsJobOutDTO {
  [APPLICATION_EXCEPTION_STATUS]: ApplicationExceptionStatus;
}

It also allows us to add nice comments to the workflow variables.

We are consuming this name in more than one place, so the idea was to keep the name centralized, we a nice comment and still provide the IntelliSense, as shown below.

Executing Camunda on local

Created make Camunda and  make camunda-stop commands to build the Camunda locally. The command includes instructions on how to execute and connect to Camunda locally, as shown below.

Once Camunda is running we still need to deploy the BPMNs/DMNs using the modeler (this process can be automated).

TODO

CRA bypass;
Zebee maxJobsToActivate configuration for all workers;
Remove hardcoded timeout for messages (timeToLive: Duration.seconds.of(30)).
Test Camunda inclusive gateway on Csamunda 8.1.
Reassessment tests.

Excellent piece of work @andrewsignori-aot. It is great to see the workflow in camunda 8 and appreciate the efforts taken to write the compose to host a local camunda as well . 👏👏
Few comments and clarifications asked.


Update main.cpp

First thanks for the tutorials so far! Brand new here so just finding any reason to get into code. Both bytes_read and bytes_written seem to be unnecessary variables as the documentation specifies NULL is usable in this instance. Figured it might help reduce complexity to newcomers like myself.

Thanks for the suggestion and glad you are enjoying the tutorials! I vaguely remember at one point in the past (Windows 7-era) that ReadProcessMemory & WriteProcessMemory would fail in certain situations if not provided with a lpNumberOfBytesRead parameter. However, when looking at the MSDN documentation, this is apparently not the case.
Let me investigate this a bit. If it turns out the parameter is unneeded in all cases, I'll drop it and update all the code.

Thought there would be a good reason. Cheers for giving an explanation! I'll try give you more feedback for further tutorials (after studying a bit more C++ XD)


Copy & Paste does not work

Firstly, I want to congratulate you on your wonderful project. Perhaps implementing the ability to copy and paste the text present in a button or chip could add more value to the project. Currently, the text displayed by a button cannot be copied. I'm developing an enterprise application used by four EU countries, and the ability to copy and paste text speeds up the work of the user.
I'm using firefox and there is no change to copy a test of the following button:
 <Button label="Hello world"   link />
Now I'm thinking of using another component or raw  to be used as a link and give the user the possibility to copy and paste the text 'Hello World'.

Hi, this is the expected behavior from the Primevue Button. The tailwind version actually mirrors the styled version. But if you want to be able to copy the text inside the button, you can easily make it.
The presets are not like rules; they are just helpers for starting to build your own presets/design systems.
So for buttons, you just need to remove "select-none" from the root. That's easy 👍


[OmniSci] Apply calcite self-join fix to modin optimization pipeline

Omnisci team found and fixed the problem with self-join queries in their calcite server.
As we have a separate optimization path for our modin-on-omnisci endpoint we should apply the same patch to it.
The patch is not open-sourced yet, but it should be open-sourced in the next release.

PR created: https://github.com/intel-ai/omniscidb/pull/158


Proxy connection implementation

Pull Request Description
This pull request closes #1677
Acceptance Test

[ ] Building the code with mvn clean install -Dintegration.tests still works.
[ ] Running mvn spring-boot:run in the strongbox-web-core still starts up the application correctly.
[ ] Building the code and running the strongbox-distribution from a zip or tar.gz still works.
[ ] The tests in the strongbox-web-integration-tests still run properly.

Questions

Does this pull request break backward compatibility?

[ ] Yes
[x] No

Does this pull request require other pull requests to be merged first?

[ ] Yes, please see #...
[x] No

Does this require an update of the documentation?

[ ] Yes, please see strongbox/strongbox-docs#{PR_NUMBER}
[x] No

@strongbox/core-developers : For implementing SOCKS proxy we have to register ConnectionSocketFactory in PoolingHttpClientConnectionManager (https://stackoverflow.com/questions/22937983/how-to-use-socks-5-proxy-with-apache-http-client-4) and we are using single PoolingHttpClientConnectionManager in entire strongbox instance, pls suggest how to achieve/implement this.
One approach is separate PoolingHttpClientConnectionManager for proxy and non-proxy connection.


#5: Crashes reporting implemented (ACRA)

#5

This seems to be necessary here as well:
https://github.com/ARK-Builders/ARK-Rate/pull/10/commits/721bcbfc50b69548d4bd65c4e2e8d80c3cff013c
android:usesCleartextTraffic="true"

This seems to be necessary here as well: ARK-Builders/ARK-Rate@721bcbf
android:usesCleartextTraffic="true"

Added


block-grid flex-grid

while playing around with block-grid and flex-grid I was wondering why there are still floats and clears - I don't think they are necessary, aren't they?

Duplicate of #8310, fixed via 4d6a363951029f1d75ed0e710fcd70572c2886a3
It'll be in the next patch release.


Adjust tests after some components have been removed from kyma

Description
Changes proposed in this pull request:

...
...
...

Related issue(s)

Resolves #1504 1504

/test pre-busola-smoke-test-docker

/cla-recheck


Recent change in device_spmv.cuh causing incorrect results

I've been testing CUDA11 integration with a recent release (7.8.0) of Cupy and found that cub's spmv was returning incorrect results. The results almost appear as though the data is being corrupted (potentially from incorrect array indexing). https://github.com/cupy/cupy/issues/3822 contains the detailed summary of the problem.
Here's a simple example of what we're seeing. a is using cub's device_spmv and b is not:
>>> a.T.dot(cupy.ones(a.shape[0])).reshape(1, a.shape[1])
array([[0.00000000e+000, 0.00000000e+000, 4.24399158e-314, ...,
        1.10383165e-307, 1.10512862e-307, 1.10636362e-307]])
>>> b.T.dot(cupy.ones(b.shape[0])).reshape(1, b.shape[1])
array([[3056.30047519, 3058.09608272, 3009.4495842 , ..., 1505.96878088,
        1460.47145535, 1505.26624145]])

The problem seems isolated to this commit: https://github.com/NVlabs/cub/commit/b2e64cf0fb4ea7ace6c86ca6765ca7c1087ef82e and when that commit is reverted, the example above returns correct results (again, a is using device_spmv and b is not):
>>> a.T.dot(cupy.ones(a.shape[0])).reshape(1, a.shape[1])
array([[3058.34350657, 3071.2141306 , 3059.92383792, ..., 1466.52647562,
        1441.08203412, 1479.81493357]])
>>> b.T.dot(cupy.ones(b.shape[0])).reshape(1, b.shape[1])
array([[3058.34350657, 3071.2141306 , 3059.92383792, ..., 1466.52647562,
        1441.08203412, 1479.81493357]])

I'm not sure if there are any automated tests for device_spmv.cuh internally, but I quickly grepped the tests in this repository for spmv and didn't get any results.

Marking unverified -- As mentioned on the PR, I'll need a way to reproduce the error with CUB-only code to create a test case.

Closing for lack of response -- feel free to reopen with more information if you still need this.

For posterity -- see comment https://github.com/NVIDIA/cub/pull/196#issuecomment-782293149 -- looks like this was fixed by the cub::DivideAndRoundUp patch.


remote "udpa." from java package paths

As discussed in #8.
@htuch

@markdroth DCO and CI :P

Fixed DCO and CI.  @htuch, please re-approve.  Thanks!


Add flask_babel dependecie

During the test on Travis and local tests via python setup.py test,
the test fails beacuse flask_babel it's not on setup.py  like
dependencie.

Codecov Report

Merging #403 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #403   +/-   ##
=======================================
  Coverage   98.73%   98.73%
=======================================
  Files          18       18
  Lines         949      949
  Branches       74       74
=======================================
  Hits          937      937
  Misses         12       12

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc78630...9449de4. Read the comment docs.

Currently, there is no test-failure - even without your addition of flask_babel.
I suppose this is a transitive dependency of an already included dependency.
But as in several locations there are imports from flask_babel, I agree it should be included in the setup.py.
Maybe @davidism can have a quick look during the current sprint?

Flask-Babel isn't required, this doesn't need to be added as a dependency. If it's not installed, nothing fails.

@davidism Could you give me a hint why it is ok to ok from third party without declaring it a dependency in setup.py?
I'd like to learn a thing or two about package maintenance.
https://github.com/lepture/flask-wtf/blob/366ff108c9ef69b3035f37688c366cbb261248de/flask_wtf/i18n.py#L5-L8


Invalid published directory structure

Hello
It seems that on version 0.1.4, the content of the published tarball is invalid:

Which is causing the following error:
> require('json-schema-to-zod');
Uncaught:
Error: Cannot find module '/some/path/node_modules/json-schema-to-zod/index.js'. Please verify that the package.json has a valid "main" entry

Link to the tarball: https://registry.npmjs.org/json-schema-to-zod/-/json-schema-to-zod-0.1.4.tgz

Thanks @gjuchault! Fixed with 0.1.5

Seems like this is happening on 0.3.0 again?
yarn run v1.22.18
$ bin/generate
Error: Cannot find module '/Users/bryanhelmig/code/zod-project/node_modules/json-schema-to-zod/index.js'. Please verify that the package.json has a valid "main" entry
    at tryPackage (node:internal/modules/cjs/loader:353:19)
    at Function.Module._findPath (node:internal/modules/cjs/loader:566:18)
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
    at Function.Module._resolveFilename.sharedData.moduleResolveFilenameHook.installedValue [as _resolveFilename] (/Users/bryanhelmig/code/zod-project/node_modules/@cspotcode/source-map-support/source-map-support.js:811:30)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/bryanhelmig/code/zod-project/bin/zodify.ts:4:13)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Module.m._compile (/Users/bryanhelmig/code/zod-project/node_modules/ts-node/src/index.ts:1618:23) {
  code: 'MODULE_NOT_FOUND',
  path: '/Users/bryanhelmig/code/zod-project/node_modules/json-schema-to-zod/package.json',
  requestPath: 'json-schema-to-zod'
}

0.2.0 confirmed to work.


Use pyautogui to perform keypresses based on sent messages

Use pyautogui to perform keypresses based on sent messages

Closed by 620e5dd35009cbf9701eaf87d09929fd2a06cd2c


Should only sync if the SyncGroup request comes from the leader member

I know that group management is in early stage, but with this change I can easily create multiple consumers belonging to the same group:
    fn is_leader(&self, group_id: &str, member_id: &str) -> bool {
        self.group_coordinator
            .groups
            .get(group_id)
            .map(|group| group.leader_id.as_deref() == Some(member_id))
            .unwrap_or(false)
    }

    fn receive_sync_group(&mut self, request: SyncGroupRequest) -> SyncGroupResponse {
        let assignments = request
            .assignments
            .iter()
            .cloned()
            .map(|assign| (assign.member_id, assign.assignment))
            .collect::<BTreeMap<String, ByteBuffer>>();

        let assignment = if self.is_leader(&request.group_id, &request.member_id) {
            warn!("leader sync group");
            let group = self
                .group_coordinator
                .groups
                .get_mut(&request.group_id)
                .unwrap();
            group.sync(assignments);
            let member = group.members.get(&request.member_id).unwrap();

            member.assignment.clone()
        } else {
            let group = self
                .group_coordinator
                .groups
                .get(&request.group_id)
                .unwrap();
            let leader = group.leader_id.as_deref().unwrap();
            let leader = group.members.get(leader).unwrap();
            leader.assignment.clone()
        };

        SyncGroupResponse {
            protocol_type: request.protocol_type.clone(),
            protocol_name: request.protocol_name.clone(),
            assignment,
            ..Default::default()
        }
    }

@aovestdipaperino Thanks for your interest in this simple server. I've sent you an invitation to the to-be-open-sourced impl of kafka-rust. Please check your mailbox and try out that software and leave suggestions if anything less than awesome.

Closed as the development has been moved to Morax under api/kafka-api.
Replied above.


FramebufferTexture' is not exported from 'three'.

"^0.134.0",
"@types/three": "^0.134.0",
"three-stdlib": "^2.8.4",

OS: MacOs BigSur M1 Chip
Problem description
running yarn build results in:
./node_modules/three-stdlib/objects/Lensflare.jsAttempted import error: 'FramebufferTexture' is not exported from 'three'.

this should be fixed in v2.8.5


Adds project requirements

adds Project requirements include  Node 4.1.2and Elm0.16. to the read.me

Could you put the requirements in a list, and requirements in a header instead of a sentence?

@eeue56 Sure thing!

:+1: Great, thanks!
If you're looking for more places you could help out, I'm happy to point you in a direction!

OK Great. I'm still getting my head around Elm, but am happy to help out where I can. :+1:

Awesome - I'll make a bunch of issues and label them with "help wanted". Feel free to pick any you fancy doing if you want!

Sounds good!


Added support for in against cookies

During our tests, we wanted to assert some cookies were not present. However, flaskclient was erroring out since it was accessing a key that didn't exist via __getitem__. As documented by __contains__, __getitem__ is invoked when __contains__ isn't defined.
https://docs.python.org/2/reference/datamodel.html#object.__contains__
self.assertIn('a', browser.cookies)
# Same as `assert 'a' in browser.cookies
self.assertNotIn('b', browser.cookies)
# Same as `assert 'b' not in browser.cookies

To remedy that, we wrote out __contains__ for each of our engines. In this PR:

Added in/not in support for cookies via __contains__
Added test

Thank you very much, mister. Merged.


Alloc stacks incorrect

Counts seem to be correct, but there's still some problem with the stacks (maybe the top is missing.)
https://github.com/jcward/hxtelemetry/blob/master/test/DisplayingABitmap/profalloc_hxcpp.png
https://github.com/jcward/hxtelemetry/blob/master/test/DisplayingABitmap/profalloc_flm.png

I believe this is now fixed due to today's GetDepth() on hxcpp and offset by 1 commits (f0db6a7f7)


openapi-react-query: support for useInfinityQuery()

@kerwanp I think it would be useful to support useInfiniteQuery().
If you agree, can I work on it?

Hi @jungwoo3490, this definitely needs to be implemented in the future.
And sure, go for it! Thanks for willing to contribute!


CRAN 0.3.0 release

@vincentarelbundock @zeileis
Just a quick update from my side. I've finally managed to wrap up some big end of year projects at work and am now hoping to submit a new version of tinyplot to CRAN before their Christmas break (23rd iirc).
Ideally, we can close (some of) the following issues:

#243
#271
#278
#279

Regardless of where we get to, I plan to submit over the weekend. Hopefully, we'll get a straight pass through pre-checks etc. and v0.3.0 will be available as a little Christmas present to our users (and ourselves!)
Any thoughts / objections? If either of you have bandwidth to take on one of the above issue, please let me. Otherwise, I'll try to tick them off myself one by one.

This sounds awesome!
Unfortunately, I won't have time to work on this before the break, as I'm finishing up a conference paper that needed to be sent out a while ago already...

If you can pull this off: Great!! If not: Perfectly fine.
Personally, I try not to submit too much stuff (be it papers, reviews, packages, ...) before Christmas because the time is so busy for everybody anyway. Instead, I do it as one of the first things in the new year. Then I can start the year productively and my submission is likely to get more attention as well.

Looks like time slipped away from me... and now I'm sticking to the Zeileis philosophy of it being better to submit things in the new year :-D


Readme doesn't reflect actual structure of Pushbullet.get_pushes() return value

The readme suggests that Pushbullet.get_pushes() returns a list of pushes. This is not true, however. It actually returns a two-length tuple, in which the first element is True and the second is the list of pushes.
Eg. the function currently returns:
>>>pb.get_pushes()
(True, [push0, push1, push2])

Instead of:
>>>pb.get_pushes()
[push0, push1, push2]

Simply updating the readme to reflect this (or changing the behavior of the function) would fix this. Some documentation of what the True means would be nice, too.

I think this can be closed now

Ah, yes, indeed it can. Pull request #43 fixed it. Thanks!


BUG: wrong implementation of toggle button

the current toggle-buttons are developed with parameters enable/disable.
In my opinion this is wrong as toggle means a on/off functionality by each pressing the button.
A parameter inside a toggle feature like enable/disable (on/off) is leading in the wrong operation.

Hi @iqtreiber, thank you for your suggestion
We looked through this and agree that the naming of the parameter sets incorrect expectations.
From the module development perspective:

We have the parameter here so users can set the action to on/off like they want and not simply toggle it. Maybe renaming the action to "Set Visibility of timer" etc. would be sufficient.
Alternatively, we can add a "real" toggle action with no parameter if this would improve the module.

It seems to me that while this is not a bug, it could be improved.
Is there a reasonable expectation that this would work in other ways? Maybe expectations are set by another module?
Or perhaps it is a matter of clarification of expectations?
I look forward to continuing this

Hello,
from the experience we have other companion modules provide always two different buttons.

toggle button (on/off) via press / release (active as long you hold press)
on button via press button (active like a "latch" as long pressing again)
off button via press button (off like "de-latched")

A good example in the audio world is the MUTE button. (e.g. Midas X32)
You will press on time MUTE an it has to stay pressed and show red coloured.
Nobody will hold the finger on the MUTE button for time being.
In video companion modules you wanna switch a INPUT via pressing the button once. (no hold)
The active INPUT will be red coloured to see the button is pressed.
Same like overlays in video where you press once.
In my opinion, both kinds of button actions are useful.
Active/Deactive buttons and Toggle buttons.
BR Stefan

Hi @iqtreiber ,
I am under the impression this has been resolved. Can you please provide some feedback so we can maybe close this ticket?

Yes, this issue is solved and the ticket can be closed


Article issue: Opera 28 supports Service Workers too

Hi - an issue with your excellent article. Opera 28 (currently in developer build, eg alpha http://www.opera.com/developer) supports Service Workers. (Currently stable Opera 27 ought to, but there's a bug).
Top article. Thanks! Bruce from Opera

Thanks Bruce, I'll amend the post.
When I try out the demo in 28 I get The Service Worker security policy prevented an action. error and the worker fails to register. Who'd be the best person to ask about this?

Hmm, I'll find out.

That bug should be gone now (and backported to Opera stable)


Suggestion: add a "year taken" field

Hey!
I think that when a user writes a review about a course he\she should fill an additional filed stating in which year he\she was enrolled in it.
Feels to me like this information is important for the relevance of the review.
BTW, your project looks great!
Ido

Great, thanks :)

@iporat08  Thanks for the advice. added it in #87.

Not yet on master


Is TestSuccessfulJobHistoryLimit flakey?

=== CONT  TestCronSuite
84
    cron_test.go:320:
85
        	Error Trace:	cron_test.go:320
86
        	            				then.go:133
87
        	            				cron_test.go:318
88
        	            				suite.go:77
89
        	Error:      	Should be true
90
        	Test:       	TestCronSuite
91
    cron_test.go:356:
92
        	Error Trace:	cron_test.go:356
93
        	            				then.go:133
94
        	            				cron_test.go:354
95
        	            				suite.go:77
96
        	Error:      	Should be true
97
        	Test:       	TestCronSuite

Closing this issue and track it with #7133

Failed more that once. Skipping.

fixed