Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -1485,6 +1485,10 @@ TableResult listTableData(
* Returns the requested job or {@code null} if not found. If the location of the job is not "US"
* or "EU", {@link #getJob(JobId, JobOption...)} must be used instead.
*
* <p>Unlike {@link Job#reload(JobOption...)} and {@link Job#waitFor(RetryOption...)}, this method
* returns the {@link Job} even when {@link JobStatus#getError()} is non-null, allowing callers to
* inspect job errors without catching {@link BigQueryException}.
*
* <p>Example of getting a job.
*
* <pre>{@code
Expand All @@ -1503,6 +1507,10 @@ TableResult listTableData(
* Returns the requested job or {@code null} if not found. If the location of the job is not "US"
* or "EU", the {@code jobId} must specify the job location.
*
* <p>Unlike {@link Job#reload(JobOption...)} and {@link Job#waitFor(RetryOption...)}, this method
* returns the {@link Job} even when {@link JobStatus#getError()} is non-null, allowing callers to
* inspect job errors without catching {@link BigQueryException}.
*
* <p>Example of getting a job.
*
* <pre>{@code
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -238,73 +238,97 @@ private boolean hasDoneState() {
return getStatus() != null && JobStatus.State.DONE.equals(getStatus().getState());
}

/** See {@link #waitFor(BigQueryRetryConfig, RetryOption...)} */
/**
* See {@link #waitFor(BigQueryRetryConfig, RetryOption...)}. Uses the default query retry
* configuration, which retries on {@link BigQueryErrorMessages#RATE_LIMIT_EXCEEDED_MSG}.
*/
public Job waitFor(RetryOption... waitOptions) throws InterruptedException {
return waitForInternal(DEFAULT_RETRY_CONFIG, waitOptions);
}

/**
* Blocks until this job completes its execution, either failing or succeeding. This method
* returns current job's latest information. If the job no longer exists, this method returns
* {@code null}. By default, the job status is checked using jittered exponential backoff with 1
* second as an initial delay, 2.0 as a backoff factor, 1 minute as maximum delay between polls,
* 12 hours as a total timeout and unlimited number of attempts. For query jobs, the job status
* check can be configured to retry on specific BigQuery error messages using {@link
* BigQueryRetryConfig}. This {@link BigQueryRetryConfig} configuration is not available for
* non-query jobs.
* Blocks until this job completes its execution and returns the latest {@link Job} snapshot. If
* the job no longer exists, non-query jobs return {@code null} while query jobs throw {@link
* BigQueryException}.
*
* <p>If the job completes with an execution error, this method throws {@link BigQueryException}
* (via {@link #reload(JobOption...)}) rather than returning a {@code Job} with a non-null {@link
* JobStatus#getError()}. Catch {@link BigQueryException} to handle job failures. To inspect a
* failed job's {@link JobStatus} without throwing, use {@link BigQuery#getJob(JobId,
* BigQuery.JobOption...)}.
*
* <p>By default, job status is polled with jittered backoff until the job completes or the
* default total wait timeout expires. Pass {@link RetryOption} arguments (such as {@link
* RetryOption#initialRetryDelayDuration(Duration)} and {@link
* RetryOption#totalTimeoutDuration(Duration)}) to override polling delays or the total timeout.
* If the total wait timeout expires before completion, non-query jobs throw {@link
* BigQueryException}, whereas query jobs return the current (still running) {@link Job} snapshot,
* so callers passing a short timeout should check {@link JobStatus#getState()}. For query jobs,
* status checks can also be configured to retry on specific BigQuery error messages using {@link
* BigQueryRetryConfig} and constants in {@link BigQueryErrorMessages} (by default, {@link
* #waitFor(RetryOption...)} retries on {@link BigQueryErrorMessages#RATE_LIMIT_EXCEEDED_MSG}).
* This {@link BigQueryRetryConfig} configuration is not available for non-query jobs.
*
* <p>Example usage of {@code waitFor()}.
*
* <pre>{@code
* Job completedJob = job.waitFor();
* if (completedJob == null) {
* // job no longer exists
* } else if (completedJob.getStatus().getError() != null) {
* // job failed, handle error
* } else {
* // job completed successfully
* try {
* Job completedJob = job.waitFor();
* if (completedJob == null) {
* // job no longer exists
* } else {
* // job completed successfully
* }
* } catch (BigQueryException e) {
* // job failed, polling timed out, or RPC failed
* }
* }</pre>
*
* <p>Example usage of {@code waitFor()} with checking period and timeout.
*
* <pre>{@code
* Job completedJob =
* job.waitFor(
* RetryOption.initialRetryDelay(Duration.ofSeconds(1)),
* RetryOption.totalTimeout(Duration.ofMinutes(1)));
* if (completedJob == null) {
* // job no longer exists
* } else if (completedJob.getStatus().getError() != null) {
* // job failed, handle error
* } else {
* // job completed successfully
* try {
* Job completedJob =
* job.waitFor(
* RetryOption.initialRetryDelayDuration(Duration.ofSeconds(1)),
* RetryOption.totalTimeoutDuration(Duration.ofMinutes(1)));
* if (completedJob == null) {
* // job no longer exists
* } else {
* // job completed successfully
* }
* } catch (BigQueryException e) {
* // job failed, polling timed out, or RPC failed
* }
* }</pre>
*
* <p>Example usage of {@code waitFor()} with BigQuery retry configuration to retry on rate limit
* exceeded error messages for query jobs.
*
* <pre>{@code
* Job completedJob =
* job.waitFor(
* BigQueryRetryConfig.newBuilder()
* .retryOnMessage(BigQueryErrorMessages.RATE_LIMIT_EXCEEDED_MSG)
* .retryOnMessage(BigQueryErrorMessages.JOB_RATE_LIMIT_EXCEEDED_MSG)
* .retryOnRegEx(BigQueryErrorMessages.RetryRegExPatterns.RATE_LIMIT_EXCEEDED_REGEX)
* .build());
* if (completedJob == null) {
* // job no longer exists
* } else if (completedJob.getStatus().getError() != null) {
* // job failed, handle error
* } else {
* // job completed successfully
* try {
* Job completedJob =
* job.waitFor(
* BigQueryRetryConfig.newBuilder()
* .retryOnMessage(BigQueryErrorMessages.RATE_LIMIT_EXCEEDED_MSG)
* .retryOnMessage(BigQueryErrorMessages.JOB_RATE_LIMIT_EXCEEDED_MSG)
* .retryOnRegEx(BigQueryErrorMessages.RetryRegExPatterns.RATE_LIMIT_EXCEEDED_REGEX)
* .build());
* if (completedJob == null) {
* // job no longer exists
* } else {
* // job completed successfully
* }
* } catch (BigQueryException e) {
* // job failed, polling timed out, or RPC failed
* }
* }</pre>
*
* @param bigQueryRetryConfig configures retries for query jobs for BigQuery failures
* @param waitOptions options to configure checking period and timeout
* @throws BigQueryException upon failure, check {@link BigQueryException#getCause()} for details
* @throws BigQueryException if the job completes with an error, a non-query job polling timeout
* expires, or an RPC error occurs (inspect {@link BigQueryException#getError()}, {@link
* BigQueryException#getErrors()}, and {@link BigQueryException#getCause()})
* @throws InterruptedException if the current thread gets interrupted while waiting for the job
* to complete
*/
Expand Down Expand Up @@ -585,7 +609,12 @@ public boolean shouldRetry(Throwable prevThrowable, Job prevResponse) {
}

/**
* Fetches current job's latest information. Returns {@code null} if the job does not exist.
* Fetches current job's latest information and returns a new {@code Job} snapshot, or {@code
* null} if the job does not exist.
*
* <p>If the fetched job has a non-null {@link JobStatus#getError()}, this method throws {@link
* BigQueryException} rather than returning the failed {@code Job}. To inspect a failed job's
* {@link JobStatus} without throwing, call {@link BigQuery#getJob(JobId, BigQuery.JobOption...)}.
*
* <p>Example of reloading all fields until job status is DONE.
*
Expand All @@ -607,7 +636,7 @@ public boolean shouldRetry(Throwable prevThrowable, Job prevResponse) {
*
* @param options job options
* @return a {@code Job} object with latest information or {@code null} if not found
* @throws BigQueryException upon failure
* @throws BigQueryException if the job completed with an error or an RPC error occurs
*/
public Job reload(JobOption... options) {
checkNotDryRun("reload");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -454,13 +454,15 @@ public TableResult list(Schema schema, TableDataListOption... options) throws Bi
* Job job = table.copy(datasetName, tableName);
* // Wait for the job to complete.
* try {
* Job completedJob = job.waitFor(RetryOption.initialRetryDelay(Duration.ofSeconds(1)),
* RetryOption.totalTimeout(Duration.ofMinutes(3)));
* if (completedJob != null && completedJob.getStatus().getError() == null) {
* // Job completed successfully
* Job completedJob = job.waitFor(RetryOption.initialRetryDelayDuration(Duration.ofSeconds(1)),
* RetryOption.totalTimeoutDuration(Duration.ofMinutes(3)));
* if (completedJob == null) {
* // Job no longer exists
* } else {
* // Handle error case
* // Job completed successfully
* }
* } catch (BigQueryException e) {
* // Handle error case
* } catch (InterruptedException e) {
* // Handle interrupted wait
* }
Expand Down Expand Up @@ -490,13 +492,15 @@ public Job copy(String destinationDataset, String destinationTable, JobOption...
* Job job = table.copy(destinationId, options);
* // Wait for the job to complete.
* try {
* Job completedJob = job.waitFor(RetryOption.initialRetryDelay(Duration.ofSeconds(1)),
* RetryOption.totalTimeout(Duration.ofMinutes(3)));
* if (completedJob != null && completedJob.getStatus().getError() == null) {
* // Job completed successfully.
* Job completedJob = job.waitFor(RetryOption.initialRetryDelayDuration(Duration.ofSeconds(1)),
* RetryOption.totalTimeoutDuration(Duration.ofMinutes(3)));
* if (completedJob == null) {
* // Job no longer exists.
* } else {
* // Handle error case.
* // Job completed successfully.
* }
* } catch (BigQueryException e) {
* // Handle error case.
* } catch (InterruptedException e) {
* // Handle interrupted wait
* }
Expand All @@ -523,13 +527,15 @@ public Job copy(TableId destinationTable, JobOption... options) throws BigQueryE
* Job job = table.extract(format, gcsUrl);
* // Wait for the job to complete
* try {
* Job completedJob = job.waitFor(RetryOption.initialRetryDelay(Duration.ofSeconds(1)),
* RetryOption.totalTimeout(Duration.ofMinutes(3)));
* if (completedJob != null && completedJob.getStatus().getError() == null) {
* // Job completed successfully
* Job completedJob = job.waitFor(RetryOption.initialRetryDelayDuration(Duration.ofSeconds(1)),
* RetryOption.totalTimeoutDuration(Duration.ofMinutes(3)));
* if (completedJob == null) {
* // Job no longer exists
* } else {
* // Handle error case
* // Job completed successfully
* }
* } catch (BigQueryException e) {
* // Handle error case
* } catch (InterruptedException e) {
* // Handle interrupted wait
* }
Expand Down Expand Up @@ -562,13 +568,15 @@ public Job extract(String format, String destinationUri, JobOption... options)
* Job job = table.extract(format, destinationUris);
* // Wait for the job to complete
* try {
* Job completedJob = job.waitFor(RetryOption.initialRetryDelay(Duration.ofSeconds(1)),
* RetryOption.totalTimeout(Duration.ofMinutes(3)));
* if (completedJob != null && completedJob.getStatus().getError() == null) {
* // Job completed successfully
* Job completedJob = job.waitFor(RetryOption.initialRetryDelayDuration(Duration.ofSeconds(1)),
* RetryOption.totalTimeoutDuration(Duration.ofMinutes(3)));
* if (completedJob == null) {
* // Job no longer exists
* } else {
* // Handle error case
* // Job completed successfully
* }
* } catch (BigQueryException e) {
* // Handle error case
* } catch (InterruptedException e) {
* // Handle interrupted wait
* }
Expand Down Expand Up @@ -598,13 +606,15 @@ public Job extract(String format, List<String> destinationUris, JobOption... opt
* Job job = table.load(FormatOptions.csv(), sourceUri);
* // Wait for the job to complete
* try {
* Job completedJob = job.waitFor(RetryOption.initialRetryDelay(Duration.ofSeconds(1)),
* RetryOption.totalTimeout(Duration.ofMinutes(3)));
* if (completedJob != null && completedJob.getStatus().getError() == null) {
* // Job completed successfully
* Job completedJob = job.waitFor(RetryOption.initialRetryDelayDuration(Duration.ofSeconds(1)),
* RetryOption.totalTimeoutDuration(Duration.ofMinutes(3)));
* if (completedJob == null) {
* // Job no longer exists
* } else {
* // Handle error case
* // Job completed successfully
* }
* } catch (BigQueryException e) {
* // Handle error case
* } catch (InterruptedException e) {
* // Handle interrupted wait
* }
Expand Down Expand Up @@ -636,19 +646,21 @@ public Job load(FormatOptions format, String sourceUri, JobOption... options)
* Job job = table.load(FormatOptions.csv(), sourceUris);
* // Wait for the job to complete
* try {
* Job completedJob = job.waitFor(RetryOption.initialRetryDelay(Duration.ofSeconds(1)),
* RetryOption.totalTimeout(Duration.ofMinutes(3)));
* if (completedJob != null && completedJob.getStatus().getError() == null) {
* // Job completed successfully
* Job completedJob = job.waitFor(RetryOption.initialRetryDelayDuration(Duration.ofSeconds(1)),
* RetryOption.totalTimeoutDuration(Duration.ofMinutes(3)));
* if (completedJob == null) {
* // Job no longer exists
* } else {
* // Handle error case
* // Job completed successfully
* }
* } catch (BigQueryException e) {
* // Handle error case
* } catch (InterruptedException e) {
* // Handle interrupted wait
* }
* }</pre>
*
* @param format the format of the exported data
* @param format the format of the data to load
* @param sourceUris the fully-qualified Google Cloud Storage URIs (e.g. gs://bucket/path) from
* which to load the data
* @param options job options
Expand Down
Loading