2024-01-07T18:19:34.668588Z [debug ] meltano 3.2.0, Linux 2024-01-07T18:19:34.684721Z [debug ] /etc/timezone found, contents: Asia/Dhaka 2024-01-07T18:19:34.685030Z [debug ] /etc/localtime found 2024-01-07T18:19:34.685596Z [debug ] 2 found: {'/etc/timezone': 'Asia/Dhaka', '/etc/localtime is a symlink to': 'Asia/Dhaka'} 2024-01-07T18:19:34.692011Z [info ] The default environment 'dev' will be ignored for `meltano config`. To configure a specific environment, please use the option `--environment=`. 2024-01-07T18:19:34.693716Z [debug ] Creating DB engine for project at '/media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project' with DB URI 'sqlite://media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project/.meltano/meltano.db' 2024-01-07T18:19:34.789029Z [debug ] Found plugin parent parent=tap-ga4 plugin=tap-ga4 source= 2024-01-07T18:19:35.155979Z [debug ] Created configuration at /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project/.meltano/run/tap-ga4/tap.128b47d3-8673-482d-b8e1-a6d98d7cf17c.config.json 2024-01-07T18:19:35.156254Z [debug ] Could not find tap.properties.json in /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project/.meltano/extractors/tap-ga4/tap.properties.json, skipping. 2024-01-07T18:19:35.156422Z [debug ] Could not find tap.properties.cache_key in /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project/.meltano/extractors/tap-ga4/tap.properties.cache_key, skipping. 2024-01-07T18:19:35.156591Z [debug ] Could not find state.json in /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project/.meltano/extractors/tap-ga4/state.json, skipping. 2024-01-07T18:19:35.164938Z [debug ] Using cached catalog file 2024-01-07T18:19:35.165478Z [debug ] Visiting CatalogNode.STREAM at '.streams[0]'. 2024-01-07T18:19:35.165650Z [debug ] Setting '.streams[0].selected' to 'False' 2024-01-07T18:19:35.165689Z [debug ] Setting '.streams[0].selected' to 'True' 2024-01-07T18:19:35.165749Z [debug ] Skipping node at '.streams[0].tap_stream_id' 2024-01-07T18:19:35.165785Z [debug ] Skipping node at '.streams[0].replication_key' 2024-01-07T18:19:35.165820Z [debug ] Skipping node at '.streams[0].replication_method' 2024-01-07T18:19:35.165860Z [debug ] Skipping node at '.streams[0].key_properties[0]' 2024-01-07T18:19:35.165907Z [debug ] Visiting CatalogNode.PROPERTY at '.streams[0].schema.properties.date'. 2024-01-07T18:19:35.165952Z [debug ] Visiting CatalogNode.PROPERTY at '.streams[0].schema.properties.activeUsers'. 2024-01-07T18:19:35.165996Z [debug ] Visiting CatalogNode.PROPERTY at '.streams[0].schema.properties.report_start_date'. 2024-01-07T18:19:35.166037Z [debug ] Visiting CatalogNode.PROPERTY at '.streams[0].schema.properties.report_end_date'. 2024-01-07T18:19:35.166076Z [debug ] Skipping node at '.streams[0].schema.type' 2024-01-07T18:19:35.166110Z [debug ] Skipping node at '.streams[0].schema.required[0]' 2024-01-07T18:19:35.166143Z [debug ] Skipping node at '.streams[0].schema.required[1]' 2024-01-07T18:19:35.166174Z [debug ] Skipping node at '.streams[0].schema.required[2]' 2024-01-07T18:19:35.166206Z [debug ] Skipping node at '.streams[0].stream' 2024-01-07T18:19:35.166244Z [debug ] Visiting CatalogNode.METADATA at '.streams[0].metadata[0]'. 2024-01-07T18:19:35.166278Z [debug ] Visiting metadata node for tap_stream_id 'users_per_day', breadcrumb '['properties', 'date']' 2024-01-07T18:19:35.166321Z [debug ] Setting '.streams[0].metadata[0].metadata.selected' to 'False' 2024-01-07T18:19:35.166352Z [debug ] Setting '.streams[0].metadata[0].metadata.selected' to 'True' 2024-01-07T18:19:35.166387Z [debug ] Skipping node at '.streams[0].metadata[0].breadcrumb[0]' 2024-01-07T18:19:35.166419Z [debug ] Skipping node at '.streams[0].metadata[0].breadcrumb[1]' 2024-01-07T18:19:35.166465Z [debug ] Skipping node at '.streams[0].metadata[0].metadata.inclusion' 2024-01-07T18:19:35.166527Z [debug ] Skipping node at '.streams[0].metadata[0].metadata.selected' 2024-01-07T18:19:35.166567Z [debug ] Visiting CatalogNode.METADATA at '.streams[0].metadata[1]'. 2024-01-07T18:19:35.166602Z [debug ] Visiting metadata node for tap_stream_id 'users_per_day', breadcrumb '['properties', 'activeUsers']' 2024-01-07T18:19:35.166643Z [debug ] Setting '.streams[0].metadata[1].metadata.selected' to 'False' 2024-01-07T18:19:35.166674Z [debug ] Setting '.streams[0].metadata[1].metadata.selected' to 'True' 2024-01-07T18:19:35.166708Z [debug ] Skipping node at '.streams[0].metadata[1].breadcrumb[0]' 2024-01-07T18:19:35.166740Z [debug ] Skipping node at '.streams[0].metadata[1].breadcrumb[1]' 2024-01-07T18:19:35.166778Z [debug ] Skipping node at '.streams[0].metadata[1].metadata.inclusion' 2024-01-07T18:19:35.166811Z [debug ] Skipping node at '.streams[0].metadata[1].metadata.selected' 2024-01-07T18:19:35.166847Z [debug ] Visiting CatalogNode.METADATA at '.streams[0].metadata[2]'. 2024-01-07T18:19:35.166880Z [debug ] Visiting metadata node for tap_stream_id 'users_per_day', breadcrumb '['properties', 'report_start_date']' 2024-01-07T18:19:35.166921Z [debug ] Setting '.streams[0].metadata[2].metadata.selected' to 'False' 2024-01-07T18:19:35.166952Z [debug ] Setting '.streams[0].metadata[2].metadata.selected' to 'True' 2024-01-07T18:19:35.166994Z [debug ] Skipping node at '.streams[0].metadata[2].breadcrumb[0]' 2024-01-07T18:19:35.167026Z [debug ] Skipping node at '.streams[0].metadata[2].breadcrumb[1]' 2024-01-07T18:19:35.167063Z [debug ] Skipping node at '.streams[0].metadata[2].metadata.inclusion' 2024-01-07T18:19:35.167095Z [debug ] Skipping node at '.streams[0].metadata[2].metadata.selected' 2024-01-07T18:19:35.167131Z [debug ] Visiting CatalogNode.METADATA at '.streams[0].metadata[3]'. 2024-01-07T18:19:35.167164Z [debug ] Visiting metadata node for tap_stream_id 'users_per_day', breadcrumb '['properties', 'report_end_date']' 2024-01-07T18:19:35.167204Z [debug ] Setting '.streams[0].metadata[3].metadata.selected' to 'False' 2024-01-07T18:19:35.167234Z [debug ] Setting '.streams[0].metadata[3].metadata.selected' to 'True' 2024-01-07T18:19:35.167268Z [debug ] Skipping node at '.streams[0].metadata[3].breadcrumb[0]' 2024-01-07T18:19:35.167299Z [debug ] Skipping node at '.streams[0].metadata[3].breadcrumb[1]' 2024-01-07T18:19:35.167336Z [debug ] Skipping node at '.streams[0].metadata[3].metadata.inclusion' 2024-01-07T18:19:35.167368Z [debug ] Skipping node at '.streams[0].metadata[3].metadata.selected' 2024-01-07T18:19:35.167403Z [debug ] Visiting CatalogNode.METADATA at '.streams[0].metadata[4]'. 2024-01-07T18:19:35.167435Z [debug ] Visiting metadata node for tap_stream_id 'users_per_day', breadcrumb '[]' 2024-01-07T18:19:35.167475Z [debug ] Setting '.streams[0].metadata[4].metadata.selected' to 'False' 2024-01-07T18:19:35.167505Z [debug ] Setting '.streams[0].metadata[4].metadata.selected' to 'True' 2024-01-07T18:19:35.167548Z [debug ] Skipping node at '.streams[0].metadata[4].metadata.inclusion' 2024-01-07T18:19:35.167580Z [debug ] Skipping node at '.streams[0].metadata[4].metadata.selected' 2024-01-07T18:19:35.167612Z [debug ] Skipping node at '.streams[0].metadata[4].metadata.selected-by-default' 2024-01-07T18:19:35.167646Z [debug ] Skipping node at '.streams[0].metadata[4].metadata.table-key-properties[0]' 2024-01-07T18:19:35.167679Z [debug ] Skipping node at '.streams[0].metadata[4].metadata.valid-replication-keys[0]' 2024-01-07T18:19:35.167714Z [debug ] Skipping node at '.streams[0].selected' 2024-01-07T18:19:35.171004Z [debug ] Invoking: ['/media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project/.meltano/extractors/tap-ga4/venv/bin/tap-google-analytics', '--config', '/media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project/.meltano/run/tap-ga4/tap.128b47d3-8673-482d-b8e1-a6d98d7cf17c.config.json', '--catalog', '/media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project/.meltano/run/tap-ga4/tap.properties.json'] 2024-01-07T18:19:37.691647Z [debug ] {"type": "SCHEMA", "stream": "users_per_day", "schema": {"properties": {"date": {"type": ["string"]}, "activeUsers": {"type": ["integer", "null"]}, "report_start_date": {"type": ["string"]}, "report_end_date": {"type": ["string"]}}, "type": "object", "required": ["date", "report_start_date", "report_end_date"]}, "key_properties": ["date"], "bookmark_properties": ["date"]} 2024-01-07T18:19:38.880465Z [debug ] Deleted configuration at /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/meltano-project/.meltano/run/tap-ga4/tap.128b47d3-8673-482d-b8e1-a6d98d7cf17c.config.json 2024-01-07T18:19:38.880998Z [debug ] Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to join our friendly Slack community. Plugin configuration is invalid {"type": "SCHEMA", "stream": "users_per_day", "schema": {"properties": {"date": {"type": ["string"]}, "activeUsers": {"type": ["integer", "null"]}, "report_start_date": {"type": ["string"]}, "report_end_date": {"type": ["string"]}}, "type": "object", "required": ["date", "report_start_date", "report_end_date"]}, "key_properties": ["date"], "bookmark_properties": ["date"]} ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/meltano/cli/__init__.py:102 in _run_cli │ │ │ │ 99 │ """ │ │ 100 │ try: │ │ 101 │ │ try: # noqa: WPS225, WPS505 │ │ ❱ 102 │ │ │ cli(obj={"project": None}) │ │ 103 │ │ except ProjectReadonly as err: │ │ 104 │ │ │ raise CliError( │ │ 105 │ │ │ │ f"The requested action could not be completed: {err}", │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/click/core.py:1157 in __call__ │ │ │ │ 1154 │ │ │ 1155 │ def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any: │ │ 1156 │ │ """Alias for :meth:`main`.""" │ │ ❱ 1157 │ │ return self.main(*args, **kwargs) │ │ 1158 │ │ 1159 │ │ 1160 class Command(BaseCommand): │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/meltano/cli/cli.py:46 in main │ │ │ │ 43 │ │ │ args: Positional arguments for the Click group. │ │ 44 │ │ │ kwargs: Keyword arguments for the Click group. │ │ 45 │ │ """ │ │ ❱ 46 │ │ return super().main(*args, windows_expand_args=False, **kwargs) │ │ 47 │ │ 48 │ │ 49 @click.group( │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/click/core.py:1078 in main │ │ │ │ 1075 │ │ try: │ │ 1076 │ │ │ try: │ │ 1077 │ │ │ │ with self.make_context(prog_name, args, **extra) as ctx: │ │ ❱ 1078 │ │ │ │ │ rv = self.invoke(ctx) │ │ 1079 │ │ │ │ │ if not standalone_mode: │ │ 1080 │ │ │ │ │ │ return rv │ │ 1081 │ │ │ │ │ # it's not safe to `ctx.exit(rv)` here! │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/meltano/cli/utils.py:632 in invoke │ │ │ │ 629 │ │ enact_environment_behavior(self.environment_behavior, ctx) │ │ 630 │ │ if ctx.obj.get("tracker"): │ │ 631 │ │ │ ctx.obj["tracker"].add_contexts(CliContext.from_click_context(ctx)) │ │ ❱ 632 │ │ super().invoke(ctx) │ │ 633 │ │ 634 │ │ 635 class InstrumentedDefaultGroup(InstrumentedGroupMixin, DefaultGroup, DYMGroup): │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/click/core.py:1688 in invoke │ │ │ │ 1685 │ │ │ │ super().invoke(ctx) │ │ 1686 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx) │ │ 1687 │ │ │ │ with sub_ctx: │ │ ❱ 1688 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │ │ 1689 │ │ │ │ 1690 │ │ # In chain mode we create the contexts step by step, but after the │ │ 1691 │ │ # base command has been invoked. Because at that point we do not │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/meltano/cli/utils.py:632 in invoke │ │ │ │ 629 │ │ enact_environment_behavior(self.environment_behavior, ctx) │ │ 630 │ │ if ctx.obj.get("tracker"): │ │ 631 │ │ │ ctx.obj["tracker"].add_contexts(CliContext.from_click_context(ctx)) │ │ ❱ 632 │ │ super().invoke(ctx) │ │ 633 │ │ 634 │ │ 635 class InstrumentedDefaultGroup(InstrumentedGroupMixin, DefaultGroup, DYMGroup): │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/click/core.py:1688 in invoke │ │ │ │ 1685 │ │ │ │ super().invoke(ctx) │ │ 1686 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx) │ │ 1687 │ │ │ │ with sub_ctx: │ │ ❱ 1688 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │ │ 1689 │ │ │ │ 1690 │ │ # In chain mode we create the contexts step by step, but after the │ │ 1691 │ │ # base command has been invoked. Because at that point we do not │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/meltano/cli/utils.py:681 in invoke │ │ │ │ 678 │ │ if ctx.obj.get("tracker"): │ │ 679 │ │ │ ctx.obj["tracker"].add_contexts(CliContext.from_click_context(ctx)) │ │ 680 │ │ │ ctx.obj["tracker"].track_command_event(CliEvent.started) │ │ ❱ 681 │ │ super().invoke(ctx) │ │ 682 │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/click/core.py:1434 in invoke │ │ │ │ 1431 │ │ │ echo(style(message, fg="red"), err=True) │ │ 1432 │ │ │ │ 1433 │ │ if self.callback is not None: │ │ ❱ 1434 │ │ │ return ctx.invoke(self.callback, **ctx.params) │ │ 1435 │ │ │ 1436 │ def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]: │ │ 1437 │ │ """Return a list of completions for the incomplete value. Looks │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/click/core.py:783 in invoke │ │ │ │ 780 │ │ │ │ 781 │ │ with augment_usage_errors(__self): │ │ 782 │ │ │ with ctx: │ │ ❱ 783 │ │ │ │ return __callback(*args, **kwargs) │ │ 784 │ │ │ 785 │ def forward( │ │ 786 │ │ __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any # noqa: B902 │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/click/decorators.py:33 in new_func │ │ │ │ 30 │ """ │ │ 31 │ │ │ 32 │ def new_func(*args: "P.args", **kwargs: "P.kwargs") -> "R": │ │ ❱ 33 │ │ return f(get_current_context(), *args, **kwargs) │ │ 34 │ │ │ 35 │ return update_wrapper(new_func, f) │ │ 36 │ │ │ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/meltano/cli/config.py:420 in test │ │ │ │ 417 │ │ │ 418 │ if not is_valid: │ │ 419 │ │ tracker.track_command_event(CliEvent.failed) │ │ ❱ 420 │ │ raise CliError("\n".join(("Plugin configuration is invalid", detail))) │ │ 421 │ │ │ 422 │ click.secho("Plugin configuration is valid", fg="green") │ │ 423 │ tracker.track_command_event(CliEvent.completed) │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ CliError: Plugin configuration is invalid {"type": "SCHEMA", "stream": "users_per_day", "schema": {"properties": {"date": {"type": ["string"]}, "activeUsers": {"type": ["integer", "null"]}, "report_start_date": {"type": ["string"]}, "report_end_date": {"type": ["string"]}}, "type": "object", "required": ["date", "report_start_date", "report_end_date"]}, "key_properties": ["date"], "bookmark_properties": ["date"]} The above exception was the direct cause of the following exception: ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /media/taeef/Projects4/Sidetrek/DataEngineering/test-dagster-meltano-iceberg/venv/lib/python3.10 │ │ /site-packages/meltano/cli/__init__.py:112 in _run_cli │ │ │ │ 109 │ │ except MeltanoError as err: │ │ 110 │ │ │ handle_meltano_error(err) │ │ 111 │ │ except Exception as err: │ │ ❱ 112 │ │ │ raise CliError(f"{troubleshooting_message}\n{err}") from err │ │ 113 │ except CliError as cli_error: │ │ 114 │ │ cli_error.print() │ │ 115 │ │ sys.exit(1) │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ CliError: Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to join our friendly Slack community. Plugin configuration is invalid {"type": "SCHEMA", "stream": "users_per_day", "schema": {"properties": {"date": {"type": ["string"]}, "activeUsers": {"type": ["integer", "null"]}, "report_start_date": {"type": ["string"]}, "report_end_date": {"type": ["string"]}}, "type": "object", "required": ["date", "report_start_date", "report_end_date"]}, "key_properties": ["date"], "bookmark_properties": ["date"]} Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to join our friendly Slack community. Plugin configuration is invalid {"type": "SCHEMA", "stream": "users_per_day", "schema": {"properties": {"date": {"type": ["string"]}, "activeUsers": {"type": ["integer", "null"]}, "report_start_date": {"type": ["string"]}, "report_end_date": {"type": ["string"]}}, "type": "object", "required": ["date", "report_start_date", "report_end_date"]}, "key_properties": ["date"], "bookmark_properties": ["date"]}